youtubedown help needed

I've been beating my head against this for a while and I've run out of time and patience, so dear Lazyweb, please lend a hand...

Something changed recently in Youtube such that many of the higher resolution video formats are no longer listed in url_encoded_fmt_stream_map but must be extracted from the dashmpd file instead. But the MP4 URLs I am finding are giving me 403 or 404 errors. So I'd like you to help me figure out where it's going wrong.

  1. Get HTML from https://www.youtube.com/watch?v=RlaoFHe6JL0
  2. Find the "dashmpd" URL, which will look like https://manifest.googlevideo.com/api/manifest/dash/...
  3. Note that it contains /signature/ rather than /s/ which means indicates that it is the real signature, not an enciphered version of it.
  4. Find the <Representation id="137"> in there.
  5. Its BaseURL should be a downloadable audio-only 1080p MP4, but instead it is 403 or sometimes 404.

What am I missing?

Update: Uh maybe I fixed it maybe.

Tags: , ,

11 Responses:

  1. Foone says:

    Looking at the recent commits in youtube-dl, I only see one that might be relevant, from May 13th:
    https://github.com/rg3/youtube-dl/commit/4c76aa06665621c7689938afd7bbdbc797b5c7ea#diff-bd8242a0122c5207531954b67e6e51f0

    They make a change involving the url_encoded_fmt_stream_map to also check for hlsvp: which seems to be something involving multi-part audio files.

    The commit message mentions restricted-embedding videos: it might be that you're not able to get the high-quailty audio-only copy because youtube doesn't generate it (or makes it 403) if embedding is disallowed.

    youtube-dl has a mode to "generate" those by pulling the combined file and ffmpeg'ing out the audio stream, so a lazy solution might be to just switch to youtube-dl in -x mode. (I assume you have reasons for not doing that, since you're doing your own downloader, though)

    • I tried using an older version of youtube-dl (2017.08.13) and it grabbed the 1080p mp4 no problem. Haven't tried with newer versions yet, though.

    • jwz says:

      My downloader predates youtube-dl; I want to keep it working because I like its features and interface, and also because a lot of other people are using it.

      hlsvp is for live streams. This is not one of those.

      The ffmpeg stuff is for combining the video-only and audio-only files together. I already do that.

  2. The BaseURL is followed by a list of SegmentURL elements. Example: SegmentURL media="sq/1/dur/5.005"

    When I add that as a suffix to the BaseURL, I get stuff.

    So I guess the file is split up into chunks?

    • jwz says:

      Yeah, I just worked that out. Sometimes using the BaseURL alone gives you the whole file. But sometimes you have to append the SegmentURLs.

      • spawn says:

        Seems like if segments are specified as: "range/#####-#####", then the base url is enough.
        However, if segments are specified as: "sq/1/dur/#.###", then the segments need

        By the way, how do you join the segments? I found that youtube-dl works for getting the desired stream. However, I also have my own downloader and would like to use that as much as possible for at least downloading the streams.

  3. neverMind says:

    Thanks for keeping your downloader working.

  4. Andrew Janke says:

    Looks fixed for me. (Failure downloading https://www.youtube.com/watch?v=B8ybR7ldXsQ is resolved.)

    Have I mentioned lately that you're a fucking hero for keeping this running, aside from all the other good you do?

    • jwz says:

      Thanks, one does what one can. But it doesn't feel very heroic, really, when there's another better-supported thingy out there. I'm supporting this one mostly out of inertia and slight curiosity.