The file ~/Library/Cookies/Cookies.plist used to be a simple XML file of your Webkit cookies, but now that file still exists but is never updated. Now (as of November?) it's a Cookies.binarycookies file, which is apparently some exciting new form of braindamage. No, it's not a binary plist, nor is it sqlite or something.
Haven't we been down this path before? Well played, you jackasses.
Anyway. Dear Lazyweb, do you know of any Perl code that can read this?
(In my humble but correct opinion, loading the cookie files used by all of the major browsers should be a built-in feature of LWP::Simple, but it's not. You have to do it by hand.)
When I went looking I found this:
http://it.toolbox.com/blogs/locutus/understanding-the-safari-cookiesbinarycookies-file-format-49980
and said "fuck it, 'll just use Firefox instead"...
Sweet fancy moses, that's horrible.
I love the last line of that article:
…just wanted to put this sentence under the spotlight it deserves:
"a mixture of big endian and little endian formatting"
That was the precise point at which I started to feel my eyes bleed. For the love of all that's holy, why?!
At a guess: there are several independently written storage frameworks being bashed together, or maybe just one that was used by someone who reinvented half of the wheel without knowing what the other half does.
From what I inferred from the reading, the data is dropped into a container file format, some kind of nextstep reminiscent object serialization file format.
That container uses big-endian. Why? Because of the motorola and PPC architecture running big-endian and because of the network-byte-order tradition of using big-endian in file formats meant to be cross platform.
Then, the data within the container: that is just a memory dump.
And the Intel architecture uses and dumps in little-endian.
Then what's the purpose of using the container, couldn't it just dump and restore with no container whatsoever? The container also provides checksums and that makes transfers safer and tampering with the files harder. It was ready in the libraries and they used it.
It's really a fast dirty hack.
It almost seems like an improvement over the "Here's a bunch of opaque binary shit wrapped in XML" approach half the OS takes (or took, it's been a while).
But will they ever convince it that it's connected to the Internet?
...now you have two problems?
Since when were the two ever different?
Look for computer forensic tools that are designed for later versions of Safari. Cookie parsing is a key component of browser analysis. I looked through the usual places I go for that kind of tool and came up empty. However, the forensic community is quick to respond to new formats like this and I'd assume it wouldn't be too long until a verified tool is released. When something is out, I'll post back here.
In the mean time, there is a (ugh) Windows tool named "Dunk!" that claims to parse the format. I have not tested it, but there is a trial license. http://www.ccl-forensics.com/Software/dunk-web-cookie-analysis-tool.html
I'd like to thank Apple for giving me another reason to stay on Snow Leopard, which doesn't have this framework "upgrade".
-j
Perhaps David McCusker is working for Apple now? Or, worse, maybe he’s inspired people.
I will pay $1 to first person that can either: prove or disprove that it's just a dump of the in-core data structure.
That's probably true, the StackOverflow page had an answer saying that NSHTTPCookieStorage could read it.
After poking around a bit I think it might be truer to say that the in-core data structure operates directly on the on-disk representation. It's pretty ludicrous. NSHTTPCookieStorage is, naturally, a wrapper around CFHTTPCookieStorage (because Apple hates object-oriented programming), which is implemented in CFNetwork.framework using a double handful of C++ classes (because Apple hates life). Cookies seem to be stored inside instances of CompactCookieArray (multiple such per cookie storage), which store a bunch of cookies together in a big ol' contiguous byte array, which I'm guessing is exactly what's dumped to the file on disk. So of course they have a bunch of code to do things like insert, delete, expire, and look up cookies in that contiguous-byte-array thing.
You know, for efficiency!
Can't give you a Perl version with what's shipping in the OS (Though it would be possible with CamelBones), but here's a Python script to read the cookie database. The format given here is fairly rough, but may be parseable enough; if not, use the NSHTTPCookie methods to process the cookies array.
#!/usr/bin/python2.7
from Foundation import *
print NSHTTPCookieStorage.sharedHTTPCookieStorage().cookies()
Well, if you're allowed to *cheat*, this works just fine with the stock Lion perl...
use Foundation;
$e = NSHTTPCookieStorage->sharedHTTPCookieStorage->cookies->objectEnumerator;
while($ck = $e->nextObject and $$ck) { print $ck->domain->cString . ": " . $ck->value->cString . "\n"; }
You're quite right, I had forgotten how the exact bridging mechanism worked in Perl.