Youtube download escalation

10 years ago
Youtube has deployed further youtubedown countermeasures. I do not yet have working counter-countermeasures.

When extracting URLs from the url_encoded_fmt_stream_map you have to append a &signature= parameter to make it work. This used to be in the urlmap as sig=, but now it is sometimes s= and encoded differently: when use_cipher_signature=True, you have to munge it before it will work. Consensus seems to be that this is actually a cipher, not a different kind of hash. I translated the cargo-cult Javascript code I found here to Perl, but it doesn't work. (Pretty sure my translation is correct, so the algorithm is wrong.) Some discussion here.

Dear Lazyweb, any ideas?

We need a working version of the decipher_sig() function in youtubedown. Or, alternately, a URL that returns the old-style un-ciphered signature.

Some examples of videos that are no longer downloadable: Chvrches, Emika, Icona Pop, Gotye.

Previously, previously.

Tags: , , , , , , ,
Current Music: Nightmare Fortress -- Hang You on the Wall ♬

24 Responses:

  1. Srini Kumar says:
    10 years ago at 11:37 am

    your take?
  2. Chris Davies says:
    10 years ago at 12:25 pm

    I've been using a script called yturl for a while now, and nothing seems to have changed in its functionality.

    It uses http://youtube.com/get_video_info to get the bits and bobs needed to construct the actual URL, which seems different to what you're using. Of course there's no guarantee that that won't break shortly either...
    • jwz says:
      10 years ago at 12:31 pm

      Uh huh:

      % yturl 'https://www.youtube.com/watch?v=ktoaj1IpTbw'
      Traceback (most recent call last):
       File "/tmp/yturl", line 164, in <module>
         itag, url = y.getURL()
       File "/tmp/yturl", line 130, in getURL
         availableItags = dict(self.getAvailableVideoItags(videoID))
       File "/tmp/yturl", line 99, in getAvailableVideoItags
         raise YouTubeAPIError(res["reason"][0])
      __main__.YouTubeAPIError: This video contains content from VEVO. It is restricted from playback on certain sites.<br/><u><a href='https://www.youtube.com/watch?v=ktoaj1IpTbw&feature=player_embedded' target='_blank'>Watch on YouTube</a></u>
      • cc478bee6af9 says:
        10 years ago at 12:41 pm

        Just got the mixtape fine with: umph PLyALKMPGOR5cRNNSnk35R70aTT5fcZoyh | xargs cclive -s best

        Maybe recent cclive changes show the solution.
        • jwz says:
          10 years ago at 11:59 pm

          FYI, umph only lists 19 of the 23 items in that playlist.

          I just built cclive v0.9.0, which appears to be the latest, and as expected:
          cclive -s best 'https://www.youtube.com/watch?v=ktoaj1IpTbw'
          Checking ... .. .. .libquvi: error: server response code 403 (conncode=0)
      • Patrick Berry says:
        10 years ago at 12:42 pm

        youtube-dl had this interesting error on the Chvrches video:

        $ youtube-dl "https://www.youtube.com/watch?v=ktoaj1IpTbw"
        [youtube] Setting language
        [youtube] ktoaj1IpTbw: Downloading video webpage
        [youtube] ktoaj1IpTbw: Downloading video info webpage
        [youtube] ktoaj1IpTbw: Extracting video information
        [youtube] ktoaj1IpTbw: Vevo video detected.
        ERROR: Unable to decrypt signature, subkeys lengths not valid; please report this issue on GitHub.
      • Patrick Berry says:
        10 years ago at 12:47 pm

        Okay, a bit crazy but running youtube-dl twice on the Chvrches video actually worked.
        • Patrick Berry says:
          10 years ago at 12:47 pm

          Crap, forgot to link to the github issue where this is reported: https://github.com/rg3/youtube-dl/issues/920?source=c
        • Ludwig says:
          10 years ago at 3:42 pm

          youtube-dl-2013.06.26 appears to be from the future, but it works for the Chvrches video.
          • jwz says:
            10 years ago at 12:09 am

            I can't figure out how to actually run youtube-dl, because I don't know shit about installing Python, but this code from its source:

            @staticmethod
            def _decrypt_signature(s):
               """Decrypt the key the two subkeys must have a length of 43"""
               (a,b) = s.split('.')
               if len(a) != 43 or len(b) != 43:
                   raise ExtractorError(u'Unable to decrypt signature, subkeys lengths not valid')
               b = ''.join([b[:8],a[0],b[9:18],b[-4],b[19:39], b[18]])[0:40]
               a = a[-40:]
               s_dec = '.'.join((a,b))[::-1]
               return s_dec

            is far too simple to be correct / sufficient. It only handles the 87-byte case. Maybe it does the same thing in that case as the various others, or maybe not -- I'm not gonna take the time to re-learn enough Python syntax to compare them -- but obviously sizes other than 87 exist out there.

            I strongly suspect this works for you only because you lucked out by getting a back-end server that was doing things the old way and not the new way.
  3. Licaon_Kter says:
    10 years ago at 2:54 pm

    I'm using https://addons.opera.com/en/extensions/details/download-youtube-videos-as-mp4/ ( the js: http://pastebin.com/vYafALQX ) for Opera 12.15 and at least does 4 are working ok.
    • jwz says:
      10 years ago at 9:56 pm

      Well, that does not seem possible to me. I've just line-by-line compared that code to the other code and with the exception of the new "88" case at the top, it's exactly the same algorithm.

      It sounds like some people are not experiencing this problem because they are lucking out and hitting back-end servers that are still doing it "the old way". That's common with Google breakages.
    • jwz says:
      10 years ago at 10:59 pm

      In fact, I've written self-tests and verified that the Javascript code and my Perl code are doing exactly the same transformation, and it's not working for me. I dunno.
  4. a.j. says:
    10 years ago at 3:34 pm

    someone on the jdownloader forums recommended a chrome addon called "ultimate youtube downloader". it has to be installed manually but i just tried it and it deals successfully with all of the videos you mentioned above (in all their various format flavors). the problem doesn't appear to be insurmountable!
  5. Filippo Valsorda says:
    10 years ago at 6:42 am

    youtube-dl dev here.

    Our code works only on 87 bytes s, but the s contained in the ytplayer.config are almost always 87 bytes.

    Some videos have different length on first request, but get back to 87 on subsequent requests. Seriously. Don't ask me why.

    If only I could get a encrypted-decrypted pair for a non-87 s...
    • jwz says:
      10 years ago at 10:28 am

      The pastebin links above have algorithms for lengths 82 through 88 -- though as I said, I haven't yet seen any of them work in the real world.
    • jwz says:
      10 years ago at 11:01 am

      Are you able to successfully download this video with youtube-dl? Or, in fact, any video that has use_cipher_signature=True?

      If so, can you please email me a full HTTP log of what you got when loading get_video_info, and what you then sent in response that worked?

      I'd really appreciate it. Thanks!
      • Filippo Valsorda says:
        10 years ago at 11:28 am

        Yeah, I can totally download it.

        There's a catch: you have to take your s not from get_video_info but from ytplayer.config.args.url_encoded_fmt_stream_map in the /watch?v= webpage.

        Have a look at the Pi and Qi functions here.
  6. led belysning says:
    9 years ago at 11:50 pm

    Hi! I know this is somewhat off topic but I was wondering if you knew where I could locate a captcha
    plugin for my comment form? I'm using the same blog platform as yours and I'm having trouble
    finding one? Thanks a lot!

    Here is my web page ... led belysning

Comments are closed because this post is 10 years old.