Oh iTunes, why must you continue to vex me so.

Dear Lazyweb, iTunes changed its streaming behavior some time in the last 5 years and I can't figure out what it's doing now.

Create two .m3u files, one containing just the URL
and one containing just the URL

If you open the first one in iTunes 10.7, you will see that iTunes titles it "01-05.mp3", knows how long it is, and lets you move the slider to seek around in the stream.

If you open the second one, the one without the ".mp3" extension on the URL, you'll see that iTunes titles it correctly ("DNA Lounge: Bootie SF" etc.) but now it lists its length as "Continuous" and now you can't seek.

Obviously I want both titles and seekability.

And I've heard a rumor that iTunes 11 does something differently-stupid.

Ideas? Insights?

Also the actually-continuous DNA Radio stream sometimes has metadata (song titles, as they change) and sometimes does not and I can't tell why. I saw it have metadata earlier this evening and now it doesn't, so WTF.

Though really, maybe it's dumb to try and use iTunes (or WinAmp or whatever) as the way that people listen to our streams in the first place. In This Modern World, that's done with a Flash player embedded on the page, right? Trouble is, I have never been able to find one that works properly with our webcast archives which are 6+ hours long. All the players want to download the entire file to the client before they let you seek through it, because "everybody knows" MP3 files are only 4 minutes long so why would you do anything else.

E.g., the Yahoo Web Player thing that the Bootie folks use here looks pretty good, UI-wise, but as far as I can tell, it cannot be made to work with our MP3 streams even though our server supports Byte-Ranges.

If you have a suggestion of one of these players that you think works, please let me know. Because I've tried like ten of them and gotten nowhere.

Your suggestion should take the form of, "I saved the HTML of the webcast page locally, edited it to install the player, and I was able to seek to hour 5 without that taking 5 hours."

Tags: , , , , ,

40 Responses:

  1. If you want to go the flash player route, you can't use Byte-Range. You have to wrap the mp3 in an FLV, pseudo-stream it over RTMP, and then flash-based players can seek. (see e.g. here for JWPlayer).

    Isn't the future great!

  2. My friends at Voice of Geeks Network use jPlayer with success.

    Attempts to do HTML5, then Flash as a fallback. Takes MP3's in live-streaming too (MP3 demo).

    • jwz says:

      I see no evidence that this can seek at all, let alone seek without downloading the whole file first.

      • Ops! Sorry for not digging for more detail.

        It looks like jPlayer can seek, but there's server configurations for that to happen.

        The trick is on "Byte-Range Requests" which is signaled by 'Accept-Ranges' in the header. The jPlayer docs start covering the concept. This may be the pill that solves the problem for any capable player.

        • jwz says:

          I know all about byte ranges. My server implements them properly. I don't see any demos of jplayer that have seeking at all. This kind of thing is why the last paragraph of my post exists.

          • Otto says:

            In my testing with jPlayer locally, I was able to get it to seek on content that I host, but not on the links to your MP3 files. Only difference I can see is that my Apache seems to return 206 responses on a Range: bytes=0- request instead of 200 responses, which is what cerebrum:8001 returns.

            This may be useless information, I admit.

          • Got some findings and stumbled upon a differences.

            The Cro Magnon Man demo is from the jPlayer site used as the example. Seeking works without pre-cache.
            The DNA demo is using the linked above version and fails to seek. Breaks even when attempting. The only client code change is the URL to point to.

            I compared the two header outputs and found a significant difference: "Connection" The working case has "Connection: close", while the failed case has "Connection: Keep-Alive" and related headers.

            That's the only lead I have to go on for a difference. jPlayer does seek as requested when "Connection: close".

            • jwz says:

              I changed it to not to keep-alive on those URLs and it seems to still be behaving the same way. Do you still see any difference in the headers?

              • Confirming that it still doesn't behave as intended. I can also confirm that the "Connection: Close" is happening on Cerebrum.

                Right now there's two hunches I have, one better than the other. The bad hunch is that for some reason the 'x-audiocast-*' header order would be messing it up. I doubt this would be the trick.

                The better hunch is that ETags for whatever reason has something to do with it. More specifically, enabling FileETag on Apache. I added a third test by uploading the War of the Worlds to the same server and having it a go. Figured having a 81.3mb file is a better test case. While there's a few moments for it to play, seek does indeed work. Perhaps this is it?

                • jwz says:

                  WTF?? "YSlow" claims you in no uncertain terms that ETag is stupid and nobody should ever use it. Anyway, since these aren't real files when byte ranges are involved, I'd have to generate ETags by hand, so I don't want to write that code without conclusive evidence (like... seeing it in the source). Though this would definitely lean me more toward "if jPlayer depends on that, then it's an insane piece of shit" regardless.

                  I kinda doubt that's what's going on, though. Because: insane.

            • Breton Kinda says:

              I have been looking at these tests. This is what I see:
              jwplayer makes a request for the byterange 0-1, with the peculiar request header X-Playback-Session-Id. On the jwplayer hosted file, the reply is "304 not modified" (regardless of how many times I clear my cache), and then, audiodata magically materialises invisibly to my web proxy. TCPDump shows that, when this happens, a bunch of audio data is being sent from. http://futurebroadcasts.com .

              on the jwz hosted file, the same request for byterange 0-1 is made, to which the server replies with the entire contents of the mp3 file.
              This all leaves me with more questions than answers. Sorry I am not more helpful, this is fascinating to me so I shall continue investigating.

              • Breton Kinda says:

                It seems like the byterange 0-1 might be a test for support for byteranges. which jwz's server fails.

                • jwz says:

                  Yeah, maybe. I made it so that requests for 0-byte ranges return Content-Length: 0 and null data now.

              • jwz says:

                Oh, yeah, I'm actually ignoring the "end byte" in the range because, well, requesting an end-byte doesn't make any sense in this application, so I just keep sending data and assume the sender will stop reading when the user hits stop.

                This is sounding a lot like, "try to reverse engineer a server that has never been used with anything other than one particular client". That is usually not a scenario that leads to either short-term or long-term success.

                • Breton Kinda says:

                  More tests and- I know that in safari, flash is not involved on these test pages. Any seeking that is possible is a result of jwplayer's server being precisely tuned to do the exact silly things that safari expects an http "streaming" server to do. One of these things is to respond to a Range: bytes=0-1 header with exactly 2 bytes. If this is not done exactly so, safari detects a streaming mp3.

                  I tried these test pages in chrome, and neither file will seek.
                  I tried them in firefox, and it simply complained that i didn't have flash installed. Once I installed flash, and tried in firefox, I found that the flash version did not permit seeking on either file.

                  I know that seeking is possible in flash, but my earlier investigation into that, revealed that it requires your server to do the precise little proprietary dance of headers and special formats/indexes and redirects that adobe has devised. Because, here's the rub, to do this requires some way to map between a time signature, and a byte range. Safari does this by doing that little 2 byte request so it can get the total size of the file (in the Range request response header), (assuming it's CBR encoded, I presume) and then blah blah blah.

                  I'm sure this is going to give you a mini aneurism as I type this.

                  • jwz says:

                    Wait, what? Two bytes? Which two? I think range 0-1 is of length 0, isn't it?

                    Also, isn't the 3rd number the same as you'd get from Content-Length in HEAD?

                  • Breton Kinda says:

                    I can't work out how to respond directly so I'll put this here.
                    I am getting some of my information from this thread on stackoverflow:

                    This may be worth pursuing regardless of whether you plan to do this jwplayer embedding thing, as the safari code is likely the same code that iTunes uses to determine the same things.

                    The two bytes in question, in my proxy scans, are "ID" ,which I think must be the first two bytes of "ID3v[N]", so the range request is .. I guess 0-1 is byte 0 and byte 1.

                  • Breton Kinda says:

                    also content-length in head, ends up being 2, when you make a 2 byte request. what safari wants to get is the /total/ size of the file, which is only in the range response header

                  • Marcos Dione says:

                    « The first-byte-pos value in a byte-range-spec gives the byte-offset of the first byte in a range. The last-byte-pos value gives the byte-offset of the last byte in the range; that is, the byte positions specified are inclusive. Byte offsets start at zero. »

                    http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html , Section14.35.1, Byte-Ranges

            • jwz says:

              Ok, I rewrote slowcat.c to not send more data than requested, so Range: bytes=0-1 is now returning exactly 2 bytes, and it seems to be working!

              Now to see if I can make jPlayer not be hideous...

              • Hazza! Caught up on the 2 byte crap. You have no envy from me for unwrapping that onion.

                The demos came with two example styles as it's all CSS. I agree, they are hideous, but at least they had a plan put in place for skinning.

                • jwz says:

                  Ok so I got something working: Test page.

                  What I'd like to have happen is, when you click on one of the Listen links, the jPlayer box moves to be below the box containing the link you clicked (kind of like what happens here when you hit Reply and the text area is reparented) because without that, if you click play down at the bottom of the page, the actual player is scrolled off the top. But I can't make that work, I get some weird error when I try to reparent the div.

                  I also can't make it show the hour in the time displays. I don't know if I'm using timeFormat wrong, or it just doesn't work with this theme.

  3. Dusk says:

    HTML5 <audio> elements are sometimes seekable, given the right combination of browser and format.

    In particular, OGG audio is definitely seekable on (desktop) Chrome and Firefox. (But Safari won't play it at all — it prefers AAC audio, and I'm not sure if it can seek it. SO MUCH FUN.)

  4. The M3U spec on wikipedia shows that you can include titles in the playlist file itself. Not sure it iTunes will pay any attention to it, but perhaps that's worth a shot.

  5. NB says:

    I'm here only to point out that NetNewsWire showed "Enclosure: dnaradio.m3u (application/wordperfect) 41 bytes - not downloaded" for this entry, and to chuckle.

    And to point you to an iTunes 10.7 download link should you decide to downgrade after accidentally 11.0.2.

  6. jwz says:

    I just had a somewhat insane idea of how to fix this, which seems to be working... I constructed a fake id3v2 "TIT2" tag and I emit that at the front of every MP3 stream, even when byte ranges are requested. This seems to be working? It gives titles to the seekable ".mp3" version of the URL, at least.

    I'm curious if this is working properly in iTunes 11, since I'm still using iTunes 10.

    • Anonnymoose says:

      I wish I had gotten to this before you fixed it.

      Anyway, this Linux user can report that Amarok can seek and displays the title in both streams. Mplayer seeks in both streams and displays the title in neither stream.

      As an aside, if these are streams of Bootie, I'm totally kicking myself for not coming to any of the shows. When did I get so lame?

  7. M P says:

    In iTunes 11 (on OS X 10.6.8 FWIW):

    an m3u containing http://cerebrum.dnalounge.com:8001/audio/2013/01-05.mp3 shows as "Kind: MPEG audio stream", is not seekable ("Size: stream", "Time: Continuous"), and shows "01-05.mp3" as the Name

    an m3u containing http://cerebrum.dnalounge.com:8001/audio/2013/01-05 shows as "Kind: Internet audio stream", "Format: MPEG-1, Layer 3", is not seekable ("Size: stream", "Time: Continuous"), and shows "DNA Lounge: 05-Jan-2013: Bootie SF: New Years Do-Over Party." as the Name

    • jwz says:

      Well, that sure does suck. I guess iTunes 11 has made seeking impossible in both cases, then. Score another win for the new iTunes.

    • jwz says:

      Oh, and I guess this means that iTunes 11 is also ignoring the ID3v2 tag I'm now sending -- since it looks like you accessed those URLs well after I added that code. Otherwise you'd have had a real title on the first URL.

  8. ank says:

    I'd like to take a moment to laugh at you and your technical issues.

    ha ha ha ha haha HA HA HA!

    Carry on.

  9. wkrick says:

    I think the solution you are looking for is called Flash Media Server, as unpalatable as that might be.

    A buddy of mine is a developer on the Xyvid Pro broadcast platform and that's what they use. They do streaming video, but the concept is the same for audio only streams.