On OSX, the "xscreensaver-getimage-file" Perl script is really slow. I'm trying to fix that.

This is the script that picks a random image file for the screen savers to load. It descends a directory, gathers up all the JPEG, PNG, and GIF files, and picks one at random.

On my Linux machine, running it on a directory containing 47,000 files in 750 subdirectories takes 20 seconds the first time, and 5 seconds on subsequent runs. On my Mac, it takes 49 seconds the first time, and 43 seconds the second time. As both machines have 7200RPM SATA drives, this leads me to believe that HFS+ sucks compared to Ext3fs, but that insight isn't particularly helpful.

So I thought, maybe I can speed this up by using Spotlight instead of iterating the directories myself. <LJ-CUT text="So I tried some Spotlight crap."> So after some googling, I tried:

mdfind -onlyin /Users/jwz/Pictures
      "kMDItemContentTypeTree == 'public.image'"

Oops, sorry, turns out "ContentType" is useless: it's only set on certain files, like ones created with Photoshop; I've got a ton of .jpg files that don't have that attribute (though Finder knows they're images.) Nice. So instead you have to do:

mdfind -onlyin /Users/jwz/Pictures
      "kMDItemContentTypeTree == 'public.image' ||
      kMDItemFSName == '*.jpg' ||
      kMDItemFSName == '*.jpeg' ||
      kMDItemFSName == '*.pjpeg' ||
      kMDItemFSName == '*.pjpg' ||
      kMDItemFSName == '*.png' ||
      kMDItemFSName == '*.gif' ||
      kMDItemFSName == '*.tif' ||
      kMDItemFSName == '*.tiff' ||
      kMDItemFSName == '*.xbm' ||
      kMDItemFSName == '*.xpm'"

Update:

So, it turns out that when "mdls" on a .jpg file does not list kMDItemContentTypeTree, what that means is "your Spotlight index is incomplete." Somehow, Spotlight failed to index a whole bunch of my files. So last night I nuked and re-created the index ("mdutil -E /") and now I can match files with either kMDItemDisplayName or kMDItemDisplayName. (kMDItemDisplayName is faster than the "FSName" parameters, which apparently stat() the file every time). But it's still incredibly slow. Way slower than traversing the disk directly.

And that takes... wait for it... wait for it... six minutes the first time. and seven minutes the second time.

I guess I could cache the results somewhere, and only re-list the directory once a day or something, but that's pretty lame.

Any other suggestions?

Update: I managed to speed it up a lot by reducing the number of stats (by assuming that certain file extensions are the gospel truth). It seems like using Spotlight for this is just a bad idea, which is too bad.