AppleScript

AppleScript sure is horrible. And slow. I wanted to get a list of the file names of the songs in a given playlist, and doing it like this takes hours when there are a few thousand songs in the playlist:

    tell application "iTunes"
      repeat with f in (every file track of playlist "Foo")
        set output to output & (get location of f) & "\n"
      end repeat
    end tell

Maybe I'd be better off groveling through the "iTunes Music Library.xml" file...

It's pretty crazy that there seems to be no way to write to stdout besides appending to the magic "output" variable (which writes it all at once, at the end) or invoking a sub-shell to run "echo".

Do any of the Perl AppleScript bindings (there seem to be at least two of them?) just invoke "osascript", or do they do something less stupid and more efficient?

Tags: , , ,
Current Music: (the non-music of iTunes using 78% CPU)

51 Responses:

  1. ultranurd says:

    I don't know if you're using 10.4, but I would expect that Automator actions would be faster because they're compiled Objective-C objects. This assumes that an Automator action exists for iTunes that does what you want.

  2. allartburns says:

    I think you're better off parsing the xml yourself and if you want updates, checking every N units of time to see if it's changed.

  3. darkshadow2 says:

    Here, this bit takes a lot less than hours for me. I left in a couple of debug lines, they add a listing of the time when it starts, and the number of songs, then the time when it ends. Strip those out if you're happy with the script. Also, it doesn't add things to the output variable, it adds them to a list then returns the list. This gives the output as comma separated. Oh, and I set it to output POSIX paths rather than Applescript's default Mac style paths.

    tell application "iTunes"
        set returnList to {}
        copy ((current date) as string) to the end of returnList
        set thePlaylist to playlist "Library"
        copy (count of every track in thePlaylist) to the end of returnList
        set allTracks to every track of thePlaylist
        repeat with theTrack in allTracks
            set theLocation to theTrack's location
            copy the POSIX path of theLocation to the end of returnList
        end repeat
        copy ((current date) as string) to the end of returnList
    end tell

    return returnList

    For me, it gives this output (I used the -ss option to osascript to output the list with quotes around the strings):

    {"Wednesday, December 07, 2005 9:45:06 AM", 2055, "/Users/poetman/Music/iTunes/iTunes Music/Compilations/Alice In Chains_ Greatest Hits/Would_.m4p", (lots of others deleted for brevity), "/Users/poetman/Music/iTunes/iTunes Music/Echolaylia/The 27th Letter/Pennsylvania.mp3", "Wednesday, December 07, 2005 9:45:42 AM"}

    2055 is the number of songs in my music library (OK, so I'm not big on collecting songs). More will add to the time, but it only took 36 seconds to go through all 2055, and I wouldn't think it'd be much more than a minute for double that.

    • jwz says:

      Apparently "-ss" is the fastlane to notworkingland. It slows it down a lot, and tends to do this:

        osascript: script had a result, but couldn't display it.: Cannot allocate memory

      I imagine they implemented -ss in applescript, sigh.

  4. roisnoir says:

    Is there a reason it needs to be a script?

    If it's a one-time thing, or an occasional one, what I do is go to the playlist, make it show me only the fields I'm looking for (usually just track name and artist), then select all, copy, and paste into a text file.

  5. coolerq says:

    Even simpler:

    tell application "iTunes" to get the location of every file track of playlist "Foo"

    Does it all in one go, as opposed to thousands of individual AppleEvents.

    --Quentin

    • spike says:

      Exactly.

      I've been working with AppleScript since it was knee high to a grasshopper, and one of the things I've learned is that you want to minimize the number of operations that you perform. The two biggest ways to do that, IMNSHO, are:

      (1) When you want to get the same property from every object in a set, use an approach like <lj user=coolerq>'s above:
          set valuelistvar to the property of every element of container

      (2) When you want to select certain items from a group, use a "whose" clause instead of a loop..test..accumulate setup:
          set objectlistvar to every element of container whose propertyname is value
      You can actually use any expression for the whose clause, not just "is".

      You can combine these two approaches to get selected information from selected objects all in one go:
          tell application "iTunes"
              get the location of every file track of playlist "Dark Pulse 4" whose name contains "remix"
          end tell

      Many applications have native implementation of whose-clause resolution, which lets them use their own internal indicies, data structures, caches, hacks, and so on to return the answer as fast as possible. (FileMaker is a classic example: if you say 'get every record of database "DB" whose field "area code" is "212"', the result is nearly instant since FileMaker actually uses it's 'database' data structures to find the matching records all at once.) Not all applications support native whose-clause resolution, and applications that do provide it don't have to provide it for every possible query type. If the app doesn't provide a native whose-clause resolver, the AppleScript engine runs the loop for you and accumulates the results, which is still faster than writing the loop in AppleScript yourself.

      Now, just so that you don't think I'm some kind of AppleScript apologist, here's the absolute suckiest implementation issue in the whole language at it exists today: the AppleScript "list" object does not support whose clauses. So you can't do the obvious thing you're going to want to:
          get every item of myobjectlist whose property operator value
      The omission of this single feature leads directly to the need to iterate over lists with repeat and accumulate the results "by hand", all of which slows everything down and causes most AppleScript programs to revert to their most base, vile, and slothful nature.

      • strspn says:

        What makes loops, even simple loops like iterating over a list, so slow?

        • damiancugley says:

          When you tell an application to do something, it is an interprocess communication (Apple Event); the more of these you do, the more the communications overhead slows you down.

          If you ask for information about 1000 songs one at a time, that's 1000 communications; if you can ask for all 1000 songs in one message, iTunes does the same amount of work, but you only need 1 communication.

          • spike says:

            It's not just sending and receiving the AE that takes up the time; in fact that part is often the least expensive part of the transaction.

            What can sometimes 'cost' more is that the receiving app has to unwrap the AE, convert whatever data is in there into its own internal formats, do the requested work / get the requested data, and then re-encode the results in standard data formats.

            And of course, if there's a "whose" clause involved, the whole situation is different: native whose-clause resolution can be up to 1.4 zillion times faster than iterating by hand.

            • damiancugley says:

              Well, if I'm allowed to include all the conversion work as marshalling and hence part of the communications overhead then I was still more or less right :-)

        • spike says:

          In my experience, loops themselves are not that slow; my two-year-old 1.25GHz PowerBook can do about 800,000 empty AppleScript loops per second.

          The slowest (and probably most powerful) think you can do in AppleScript is communicate with other applications, by requesting data from them and/or sending commands to them (handled internally via AppleEvents). The overhead of high-level interapplication communication is part of the lag there, but so is the processing of the requests and commands by the "other" application.

          So, for example, if you ask iTunes for a list of all songs in a given playlist, that's one round-trip AppleEvent. And if you ask iTunes for the location of all the songs in a given playlist, that's still only one round-trip AppleEvent, and only one trip through iTunes command processor.

          But if you ask iTunes for the list of all the songs in a given playlist (one AppleEvent round trip), and then loop over each item in the list, and for each item in the list, ask iTunes to get you the location of that particular song, that's N additional round trip AppleEvents, and N additional trips through iTunes' command processor.

          The ability to access the internal objects, commands, and methods of an application like iTunes (or iPhoto, or the Finder, or Mail, or FileMaker, or BBedit/TextWrangler, or iChat, or iCal, or whatever) in a relatively uniform way from the built-in system scripting language is very powerful. However, the relatively high level of abstraction that AppleEvents/AppleScript provides comes at the expense of relatively slow interapplication communication through this channel.

          And that's why asking iTunes to do all the work for you at once (get the location of every song...) is so much faster than getting the list of songs from iTunes and then asking iTunes N times for the location of one particular song.

      • jwz says:

        Cool, thanks!

        So what am I missing here? I guess I have the syntax wrong?

          tell application "iTunes" to return the POSIX path of the location of every file track of playlist "Foo"

        And how would I return multiple fields using that syntax? Like, I want to do something like this for each track:

          (get artist of f) & "\t" &
          (get album of f) & "\t" &
          (get location of f) & "\n"
        • You can right-click a playlist and Export to a tab-delimited plain-text dump of almost every attribute of each song in the playlist. It's mac-formated so you'll have to replace the ^M's with \n's with your method of choice. Then awk your way through the data.

          I can't find an iTunes applescript reference, but maybe you could get iTunes to export via osascript.

      • this sounds like Matlab. It's all about vectorizing the operations, because looping is ass slow.

  6. darkshadow2 says:

    Well, that beats my script. Gah, why didn't I think of that?

    jwz, if you like the idea of getting back the POSIX paths, then this script'll do it. Same setup on my end as before, but this only takes 6 seconds rather than 36. I'd say that's a win.

    tell application "iTunes"
        set returnList to {}
        set locationList to the location of every file track of playlist "Library"
        repeat with theLocation in locationList
            copy the POSIX path of theLocation to the end of returnList
        end repeat
    end tell

    return returnList

  7. ivan_ghandhi says:

    Maybe you'd better use some more civilized (Ruby, Python) or less, but still civilized languages (Perl)?

    • crimethnk says:

      Mac::Glue is a perl module I wrote some years ago, which is now included in Tiger. It uses AppleScript vocabulary with Perl syntax, combining the "best" of both worlds ...

      The slight downside is you need to create a "glue" for each app prior to using it. Docs are included with the module, and I can be asked for more information.

      	use Mac::Glue ':glue';
      my $itunes = new Mac::Glue 'iTunes';
      my $tracks = $itunes->obj(file_tracks => library_playlist => 1);
      my $locations = $tracks->prop('location');

      for my $location ($locations->get) {
      print "$location\n";
      }

      "whose" clauses are also possible:

      	my $tracks    = $itunes->obj(
      file_track => whose(NOT =>
      [ artist => contains => 'Kenny G' ]
      ),
      library_playlist => 1
      );


      And so on.
      • crimethnk says:

        Oh, and One More Thing ... $location ends up being automatically converted to a POSIX path for you. HA.

      • jgreely says:

        Very handy; I've been wanting something like this, but my googling always turned up less useful methods. The only minor annoyance is that Apple hides the necessary glue scripts in /System/Library/Perl/Extras/bin.

        -j

    • mutiny says:

      That's what's beautiful about Appscript.

      For example, to get the value of the first paragraph of the topmost document in TextEdit:

      app('TextEdit').documents[1].paragraphs[1].get()

      This is equivalent to the AppleScript statement:

      get paragraph 1 of document 1 of application "TextEdit"

  8. marapfhile says:

    Interesting approach--sounds almost like treating AppleScript as a functional language with techniques like map and filter.

  9. ckd says:

    You could use Python with PyObjC. Bob Ippolito's posted some sample code on his blog. Using the native XML parser to turn it into a Python dict is far, far faster than anything using AppleScript/osascript/the Python appscript bindings/etc.

    • effbot says:

      You don't really need PyObjC to do this; any XML toolkit can deal with the iTunes file. With this PLIST parsing recipe (scroll down a bit) and few lines of dictionary drilling and ugly search loops, I can pull out the files for a playlist in just over a second for a 5 megabyte iTunes file. Most of the time is spent on PLIST processing, so you could probably speed this up 2-4 times by operating directly on the XML structure.

      However, I suspect it will take more than j"ust over a second" to persuade JWZ to start tinkering with Python, so I'm not sure it would be a net win ;-)

  10. From the above code, except using a string (as you originally wanted to), taking the playlist as a commandline argument. 134 seconds for a smartplaylist with 2000 songs in it.


    #!/bin/sh
    osascript \
    -e 'tell application "iTunes"' \
    -e ' set returnString to ""' \
    -e " set locationlist to the location of every file track of playlist \"$1\"" \
    -e ' repeat with theLocation in locationList' \
    -e ' set returnString to returnString & the POSIX path of theLocation & "\n"' \
    -e ' end repeat' \
    -e 'end tell' \
    -e 'return returnString'

    Same, except using the original list from the above code and sed. It's a little more than twice as fast on my craptastic iBook2k. 63 seconds.


    #!/bin/sh
    osascript \
    -e 'tell application "iTunes"' \
    -e ' set returnList to {}' \
    -e " set locationlist to the location of every file track of playlist \"$1\"" \
    -e ' repeat with theLocation in locationList' \
    -e ' copy the POSIX path of theLocation to the end of returnList' \
    -e ' end repeat' \
    -e 'end tell' \
    -e 'return returnList' | \
    sed 's/,\ \//\
    \//g'

    Rather than use applescript's "quoted form" or other such yuck, I would continue piping through sed to transform the paths if need be.

    Too bad iTunes doesn't natively dump .m3u playlists. Or, too bad I'm too dumb to figure out how to get iTunes to dump .m3u playlists. Maybe this is why you wanted to write this script in the first place?

    • jwz says:

      BTW, the trick to writing shell scripts directly in AppleScript is:

        #!/bin/sh
        exec osascript <<\EOF
        tell application "iTunes"
        ...

      The backslash prevents sh expansion of $. You don't actually need to end the file with "EOF".

  11. baconmonkey says:

    the more applescript I see, the more I'm convinced that the design doc said nothing more than the following:

    1. Watch all 80s movies that feature computers.
    2. Create a syntax parser that will run every program or command line featured in those movies.
    3. profit.

    • jwz says:

      It's like some demented chimera of COBOL and SQL.

      • baconmonkey says:

        > FIND ALL SECRET GOVERNMENT SPY FILES

        Access Denied...

        > UNSCRAMBLE ALL SECRET GOVERNMENT SPY FILES

        Files Unscrambled.

        > DOWNLOAD ALL SECRET GOVERNMENT SPY FILES TO FLOPPY DISK

        Downloading Files...
        .
        .
        .
        Files Downloaded

        > ERASE ALL FILES AND CRASH ALL NETWORKS

        ..........................................................
      • spike says:

        AppleScript's mother was the aging town prostitute: HyperTalk, HyperCard's scripting language. My favorite HyperTalk snippet:
            get the message
            put it in the box

        HyperTalk was cool, but it didn't have progv...

        • jwz says:

          I can't decide whether line 3 is "or else it gets the hose again" or "three you catch the man".

          (Hey, help?)

          • spike says:

            I'll don't remember off the top of my head how to do what asked about in AppleScript,
                (mapcar #'namestring (itunes:get-tracks))
            but I'll poke at it and see if I can come up with something that doesn't require a repeat loop.

            • jwz says:

              Yeah, I still can't figure out how to do anything useful with these objects that AppleScript laughably calls "lists". E.g.:

                set F to every file track of playlist "Test"
                set F2 to every location of F

              gets me "syntax error: Expected class name but found property", and

                set F2 to the location of F
                set F2 to the location of every item of F

              both get me "execution error: Can't get location of {file track id 80348, file track id 80349, ...}"

              Likewise for any compound attempts like

              • spike says:

                Well, I've done my homework on this one, and the answers are, as they say, "bad".

                The short version is that the happy capability of being able to get a property of 'every' object in some container (possibly matching some filter 'whose' clause) is a unique special case.

                You cannot get a chain of properties (e.g., POSIX path of location of ...) from the objects in a container, just a single property (e.g. location).

                You cannot get a property from (or do anything else en masse to) each object in a so-called-list.

                The "get property of every object of container [whose clause]" is implemented as a unique and special convenience feature that is not shared by any other part of the whole so-called language.

                And thus, programmers have to waste their own time figuring out this giant wart on the language and end-users have to waste their time waiting for ponderous loops to iterate unnecessarily.

                Did I mention that it was "bad"?

                • jwz says:

                  Wow, what a bullshit language!

                  The next time I hear someone claim it's "based on Lisp" I'm going to beat them to death with the Platinum-Iridium Standard Cons Cell.

    • ch says:

      ding! the (bacon)monkey wins the prize.

    • avirr says:

      Close ;-) The folks who wrote it were designing an English-like scripting language based on... LISP!

      The other main issue is that they didn't like multiple verbs, so everything is just set the foo to bar, and you have to guess about foo and bar. ARGHHHHH!

  12. kitten_moon says:

    ... but in different ways.

    Behold, a stylesheet:

    <?xml version="1.0"?>
    <xsl:stylesheet version="1.0" id="stylesheet"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="text"/>
    <xsl:template match="xsl:stylesheet"/>

    <xsl:param name="playlist" select="'Bootylicious'"/>

    <xsl:variable name="plarray"
    select="/plist/dict/array[preceding-sibling::key[1]='Playlists']"/>

    <xsl:key name="tracks"
    match="/plist/dict/dict[preceding-sibling::key[1]='Tracks']/key"
    use="."/>

    <xsl:template match="/">
    <xsl:apply-templates
    select="$plarray/dict[key[.='Name' and following-sibling::string[1]=$playlist]]"
    mode="playlist"/>
    </xsl:template>

    <xsl:template match="dict" mode="playlist">
    <xsl:for-each select="array/dict">
    <xsl:variable name="trackid" select="integer"/>
    <xsl:variable name="track"
    select="key('tracks',$trackid)/following-sibling::dict[1]"/>

    <xsl:value-of select="$track/string[preceding-sibling::key[1]='Name']"/><xsl:text>
    </xsl:text>
    </xsl:for-each>
    </xsl:template>

    </xsl:stylesheet>

    Save this to your iTunes directory, then apply it to your XML file like so:

    xsltproc --novalid --stringparam playlist "Name of your playlist here" \
    makeplaylist.xsl "iTunes Music Library.xml"

    Output will be a list of file://localhost/ URLs, which approximates what you want I think?

    (This script adapted from some other stuff I had lying around, hence the wierd mode="playlist" stuff. Whatever, it works for me)

    • kitten_moon says:

      xsltproc available here

      Or use another XSLT engine.

      • joel says:

        I like xmlstarlet, I was able to use Fink on my Mac to "apt-get" it.

        In xmlstarlet-land, your stylesheet would be applied with the command:

        xml tr makeplaylist.xsl -s playlist=[Name of your playlist here] "iTunes Music Library.xml"

        xmlstarlet can do lots of other stuff. I've mainly been using it to learn XSLT, it can generate XSLT for you using just command line switches.

    • jwz says:

      That is deeply perverse.

    • sherbooke says:

      XSLT2 is way more complex than that, although I have no examples to hand. The inclusion of XML Schema and Xquery/Xpath/Xlink/Xinclude/X guarantees that it is way, way, way more complex. I dabbled in XSLT for a long while thinking it was the One True Way. Since XSLT2 hove into view, I've been recommending to anyone who will listen that they should be using XML::DOM in perl or it's equivalent in other packages. At least everything is in the same language :-)

    • brianfeldman says:

      Oh, and you can do, like... queries on it. And stuff.

      SELECT SUBSTRING(artist FROM 1 FOR 30) AS artist,
      CAST(CAST(AVG(CASE WHEN rating = 0 THEN
      (SELECT AVG(rating)
      FROM tracks r
      WHERE r.rating <> 0 AND
      r.archive_id = ot.archive_id)
      ElSE
      rating
      END) AS INTEGER)
      AS CHAR(4)) || '%' as atr,
      COUNT(*) AS considered,
      (SELECT COUNT(u.*)
      FROM tracks u
      WHERE (u.archive_id, u.artist, rating) =
      (ot.archive_id, ot.artist, 0)) AS unrated
      FROM tracks ot
      WHERE archive_id = (SELECT archive_id
      FROM archives
      JOIN users USING (user_id)
      WHERE (users.name,
      host,
      source_name,
      source_kind) =
      ('brianfeldman',
      'macintosh.green.homeunix.org',
      'Brian Feldman',
      'iPod'))
      GROUP by ot.artist, ot.archive_id
      HAVING (SELECT COUNT(*) FROM (SELECT a.album, COUNT(*)
      FROM tracks a
      WHERE a.archive_id = ot.archive_id AND
      a.artist = ot.artist
      GROUP BY a.artist, a.album) AS real_albums) > 1
      ORDER BY atr DESC, artist
      LIMIT 25;
      • brianfeldman says:

        Yes, everything you need for that comes with OS X already. Well, not the DB stuff itself. *shrugs*

        • jwz says:

          You have completely lost me. How is it that you are managing to sodomize iTunes with SQL or whatever that is?

          • brianfeldman says:

            The ITunes.rb contains everything you need to get all of the info currently available (in iTunes 6.0) for normal music playlists in proper format to do things to using a real scripting language. DB_SQL.rb is an example that does the iterating-and-ripping-track-info bit and sticks it in a postgresql database for backup or whatever you deem necessary. I've even used it to synchronize two disconnected sets of data with their ratings and playcounts and all that. Means to an end means nothing to me if not overkill :-P

  13. baconmonkey says:

    itunes:
    File > Export Song list
    Save as Type "text file"
    filename: "poop.txt"

    command line:
    > cut -f 25 poop.txt

  14. jjminer says:

    On June 9th, one of our developers (Ogden Kent) did a great presentation on XSL using the iTunes Music Library XML document as an example.. He was entirely on Windows, but the XPath examples should work with anything.

    http://www.doit.wisc.edu/webdev/archive/

    Specifically:
    http://www.doit.wisc.edu/webdev/archive/presentations/xsl_june_2005.ppt
    http://www.doit.wisc.edu/webdev/archive/presentations/XSL-Presentation.zip

    jon