videos to clog my tubes with

Dear Lazyweb,

Youtube is now serving up HD videos with fmt=22. (Previously, the equally undocumented fmt=18 gave you slightly-higher-quality MP4.) But has anyone figured out what to throw at the /get_video URL to actually download either the MP4 or HD versions? All the things I've tried still result in it serving up the low-res version.

    Update: The trick is to use th t and fmt parameters only; provide others and it redirects you back to the flv version. My youtubedown script works more better now. It will grab the HD version if available, else the MP4.

Also, has anyone figured out how to download video from mtvmusic.com? The video quality is pretty crappy, and (contrary to their claims) it is far from being their entire catalog, but sometimes I'll take what I can get.

Tags: , ,

27 Responses:

  1. dranon says:

    Keepvid figures out the links for the flv and the mp4 (if there is one).

  2. duskwuff says:

    If you're using Safari, Window > Activity shows all resources being used by open web pages, including Flash video assets. I don't know how to generate the URL for the HD version, but the one I captured for this HD video looks radically different from the standard get_video URL... it's under the cache.googlevideo.com domain and includes a number of new-looking parameters.

    Disappointingly, I haven't been able to find any of the crucial-looking parameters (in particular, id and signature) in the HTML source. It's possible they're being generated by the SWF somehow... :(

    • jwz says:

      The "Activity" trick notably doesn't work for the MTV stuff, and I'm not entirely clear on how that's possible, since I thought all Flash network connections went through the browser.

      • duskwuff says:

        It looks like the MTV videos are getting streamed with Flash Communication Server (macromedia-fcs, port 1935). You'll need to do something significantly more clever to capture these.

        • jayp39 says:

          They're using rtmp, which is a proprietary protocol but could be something other than FCS (such as red5).
          You can figure out what server they're connecting to with something like netcat or wireshark. Assuming they're not doing anything weird/special over rtmp, a standard rtmp ripper should work.

  3. tiff_seattle says:

    the entire whose catalog? i searched for skinny puppy videos and they showed no results. i coulda sworn i saw a skinny puppy video on mtv once.

    • jwz says:

      Yeah, there was a press release that claimed this was the entire MTV catalog. That claim is clearly bullshit, unless by "entire catalog" they mean "of stuff currently in rotation". They had about a 10% hit rate on my first couple dozen searches (of videos I'm certain I saw on MTV!)

  4. jayp39 says:

    On the topic of ripping stuff, do you know if there is an easy way to reverse the process you use to cram all the mp3s for one of your mix tapes into one file, such that I could download your mix tape and break them apart?

  5. rane500 says:

    It was my impression the MTV upload was going to be done in multiple stages, maybe they don't want to clog the tubes.

    • jesus_x says:

      You have to do it slowly, it's not a truck, you can't jsut throw 10,000 videos on it all at once.

      • karlshea says:

        Ten movies streaming across that, that Internet, and what happens to your own personal Internet? I just the other day got... an Internet was sent by my staff at 10 o'clock in the morning on Friday. I got it yesterday [Tuesday]. Why? Why? Because it got tangled up with all these things going on the Internet commercially.

  6. alogic says:

    Thanks for the script. Very useful. It even inspired me to continue study Perl.

  7. edge_walker says:

    So apparently there is now not only fmt=18 for slightly higher quality MP4, but also fmt=6 which serves a much higher video quality MP4 (30 fps 900 kbps, as opposed to 24 fps 512 kbps) though unfortunately also lower quality audio (96 kbps mono rather than 128 kbps stereo). Very few videos overall have a fmt=22 HD version, but so far I have been lucky with fmt=6 on every try.

    • jwz says:

      What the hell? Why would they turn the audio to mono?

      Is this crap actually documented anywhere yet?

      • edge_walker says:

        It appears to be meant for the iPhone client. My conjecture based on that is they assumed that since it wouldn't make a big difference in the settings where these things are watched, they could avoid doubling the size of the streams outright - just a dumb guess.

        It's aggravating that none of this appears to be documented anywhere on the YouTube site (or the wider Google mothership), much as I've searched.

    • jwz says:

      I found this which claims:

      # default: MP3, 64 kbps, 22.05 KHz, mono
      # fmt=5: MP3, 64 kbps, 22.05 KHz, mono
      # fmt=6: MP3, 66 kbps, 44.1 KHz, mono
      # fmt=13, 3GP, AMR or AAC audio
      # fmt=17, 3GP, AMR or AAC audio
      # fmt=18: AAC, 126 kbps, 44.1 KHz, stereo
      # fmt=22: AAC, 248 kbps, 44.1 KHz, stereo (only if original was 1280x720)
      # fmt=34: AAC, 68 kbps, 22.05 KHz, stereo
      # fmt=35: AAC, 112 kbps, 44.1 KHz, stereo (rare)

      • edge_walker says:

        Buried in some forum I found this table from someone who uploaded an HQ-video movie and checked the results. (Yes, really, it was an image. With the video/audio columns were interleaved. I managed to bother to shuffle the columns in Gimp, but have yet to get bothered enough to transcribe it.)

        Looks like the choice is fmt=22 when available, otherwise fmt=18 for stuff where you care about the audio quality more than the video, fmt=6 for the reverse case.

        I wonder how hard it would be to automate downloading the clip in both fmt=18 and fmt=6, demuxing the copies, and muxing the high-quality video and sound streams back together...