It's Apple Pass Type ID Certificate Renewal Day, so let me share with you what a spectacular shitshow that is.The DNA Lounge store sends you an Apple Wallet Pass when you buy a ticket. For the end user, it's a good system: your ticket pops up on your lock screen when you arrive at the club. I wish more businesses used it. (Looking at you, Alamo.)
But given how hard Apple makes it, I'm not at all surprised that more people don't.
If anyone sane were designing this, the API would be something like: "load this URL to put a JPEG in the special photo gallery that you can access without unlocking your phone. Done." But it's Apple, so you need to sign that JPEG with an Apple-issued certificate. And that certificate expires every 12 months. And you can't renew it, you have to create a new one every year by jumping through a set of hoops worthy of one of those "ninja warrior" shows.
Here are my notes on how to do that this year. Of course the procedure also changes slightly every year, and can't be sensibly automated.
This is insane. IN. SANE.
To create a new "dnapass.crt" file, which expires annually:
- developer.apple.com / Certificates / Identifiers
- Rightmost menu: "Pass Type IDs"
- Click on "pass.com.dnalounge" and Remove. (This step is new for 2020: It no longer lets you create a new pass with the name of an existing one, so we have to delete the old one first. But the name has to be the same as last year! And the error message just says "invalid", not "name already used".)
- Click the tiny plus box.
- Create new "Pass Type ID".
- Description: type "Ticket";
- Identifier: type "pass.com.dnalounge" (beware that it tries to "help" by typing out "pass." for you, so don't type it twice.)
- Keychain Access.app / Menu 0 / Certificate Assistant / Request from a CA.
- "jwz@jwz.org", "DNA Lounge Wallet Key"
- Checkbox "Save to disk"
- Back to the web site: Click on the "pass.com.dnalounge" pass.
- Choose File: "CertificateSigningRequest.certSigningRequest"
- Download the "pass.cer"
Now we need crt and key files that are not password protected, because we're doing this crazy thing of using these certs from a web server instead of typing in a password every time we use them. Here are the hoops for that:
- Open "pass.cer" in Keychain Access.app
- Select "My Certificates" (or else you can't select ".p12" on the Save dialog)
- Find the right "Pass Type ID" cert by looking for an "Expires" of today, since now there are several.
- Context Menu / Export / "Pass Type ID: pass.com.dnalounge" as "dnapass.p12" with blank password.
- openssl pkcs12 -in dnapass.p12 -clcerts -nokeys -passin pass: -out dnapass.crt
- openssl pkcs12 -in dnapass.p12 -nocerts -passin pass: -passout pass:TMPTMP -out dnapass.tmp
(Password is required, but bullshit password "TMPTMP" must be at least 4 characters!)- openssl rsa -in dnapass.tmp -out dnapass.key -passin pass:TMPTMP
- openssl pkcs12 -in dnapass.p12 -out dnapass.pem -nodes -clcerts
- rm dnapass.tmp dnapass.p12
- "scp -p dnapass.* www:" and install the new certs.
- Update your calendar reminder for this date next year: remember, the new cert expires a year from today, not a year from when it expired this year.
If it doesn't work, check:
You can create a CLI-based cloud-hosted HTTPS server in fewer steps
I'm sure it's very lucrative business to be "the guy" in any city who knows this & puts up with this every year on behalf of several clients who need it.
Steps 14-17 should be scriptable using the “security” cli tool - I’ve done some scripting with this on an older MacOS Server for handling LetsEncrypt cert updates which requires similar levels of monkeying around. Of course, it’s possible that newer macOS doesn’t allow the sort of manipulations you need any more - it seems to do a lot more prompting for approval. And it doesn’t make the whole process any less insane.
Ah, I'm glad it's not just me that finds the Apple cert rabbit hole a horrendous hell-pit of difficult-to-automate instability.
Mmm. I think all the moving parts here are some flavor of X.509 so probably that means everything other than Apple's stupid web site can be factored out with a bit of work.
Almost certainly Apple does not require CSRs to be fresh (there's no reason they should except to be annoying). In this case you can keep the result of steps 8-10 and re-use it, and also keep the result from step 21 of the same workflow and re-use that, now the certificate files you get back from Apple match your existing keys and you just need to upload to the web server in a suitable format so in future years replace steps 14-22 with something like:
openssl x509 -in pass.cer -inform DER -out dnapass.crt -outform PEM
Obviously you could do a lot more, but I focused on stuff you could just cut out without spending an hour debugging a Perl script or whatever.
The security consequence of this much lower workload is: You are using the same private key over a longer period. If you lose this key (e.g. some imbecile writes it to a Pastebin, or a
trustedemployee takes home a backup of the server and then you never see them again) arguably you should do the Apple dance immediately with fresh keys. Likewise in theory resourceful bad guys might eventually break this key after years of effort, but in reality the god-damn NSA are busy spying on people's phone sex or stealing confidential business data from foreign businesses, not making fake DNA Lounge tickets.Actually now I stare at this more closely, you end up with three output files whereas I can only explain two. You get dnapass.key dnapass.crt and dnapass.pem
So logically dnapass.key is your PEM encoded private key and thus wouldn't change from one year to the next with my shortcut
dnapass.crt is the certificate Apple gave you this year, in PEM format.
dnapass.pem seems like a duplicate but maybe the idea is that it bakes both the keys and certificate into a single PEM file? If your server software requires that then you'd need to do that each time which is another openssl call in my scheme.
dnapass.pem contains the contents of dnapass.crt followed by a private key that is not textually identical to the one in dnapass.key. That second entry has "friendlyName: DNA Lounge Wallet Key" rather than "friendlyName: Pass Type ID: pass.com.dnalounge" (which is what both copies of the .crt key have).
For creating the .pkpass bundle I only use .crt and .key. The command is:
openssl smime -binary -sign -certfile $wwdr_cert -signer $sign_cert -inkey $sign_key -outform DER < manifest.json > signature
The .pem is only used for sending push notifications (e.g. updating existing passes with an altered start time). PHP code:
$ctx = stream_context_create();
stream_context_set_option ($ctx, 'ssl', 'local_cert', $PEM_FILE);
$server = 'ssl://gateway.push.apple.com:2195';
$fp = stream_socket_client ($server, $err, $errstr, 60, STREAM_CLIENT_CONNECT | STREAM_CLIENT_PERSISTENT, $ctx);
OK, that makes sense. Well, I mean as you point the overall scheme doesn't make a whole heap of sense, but in context...
I don't see how there can be a different private key, there's nowhere for that to come from and it doesn't serve any purpose - keys come naturally in pairs, so I assume the textual difference is because OpenSSL saw fit to apply different friendlyName values which you don't care about at all while the actual keys are identical inside. I could write a whole long thing about how to check, but let's assume they are in fact identical.
In that case the PHP can be adjusted to use separate certificate and key files, keeping consistency with the bundle making code. Something like:
stream_context_set_option ($ctx, 'ssl', 'local_cert', $sign_cert);
stream_context_set_option ($ctx, 'ssl', 'local_pk', $sign_key);
So that on its own is one less step each year to make the extra file and one less source of bizarre errors as the files used are the same so they can't somehow get out of sync. I can't prove that will work but it makes sense and sounds like you could test it without fucking up real customers.
Put together with my original idea, keep dnspass.key and that CSR, do the Apple dance every year to get a new pass.cer file from them using the same CSR, and convert it to dnapass.crt you no longer need dnapass.pem
Without counting I'm going to say this halved the number of steps.
I'm a little lost about where the public/private sides of all of this are... When I generate the CSR that I send to Apple, what's in that? I thought CSRs contained a public key, but at that point I haven't generated a private key yet. Where does the private key come from? Did Apple generate the private key for me and return it in the .cer? What all is in the .cer?
The PHP change to omit the need for dnapass.pem seems to be working, thanks!
A Certificate Signing Request contains some public key, plus an identity for which its owner purportedly want a certificate, all signed using the corresponding private key to prove that the entity which created this document (requesting a certificate) has that key. It's very similar to a self-signed certificate, except it metaphorically has "Certificate Signing Request" written at the top of the document in big letters.
Where did the key come from?
Step 8 "Keychain Access.app / Menu 0 / Certificate Assistant / Request from a CA" is locally minting a key pair (both public and private keys, of whatever kind Apple decided is the default, likely 2048-bit RSA even though this is 2020 and we all have flying cars or whatever). Then it uses that pair to create and sign the CSR for you.
So yes, in some sense you actually have generated the key, it was just done for you automatically and is living in your Keychain. I am not a Mac user but my understanding is that the Keychain is at least as safe as keeping it in a file on your local computer so that's fine.
I'm actually not sure exactly what format pass.cer is in, but it can't have your private key inside it because Apple made it and they don't know that key. It's just the certificate, and when you give it to Keychain Access.app presumably it recognises the public key and says "Oh, I have the corresponding private key for this on file already". Giving it more certificates with the same key might actually confuse it, but if you follow my recommendation to skip making new CSRs you won't need to show new certificates to Keychain Access.app either.
You should in principle be able to make a CSR from the openssl command line (or a tool which doesn't suck as badly) instead of using Keychain to make it, but I have no insight into what Apple expects so that could mean wasting time experimenting, hence I recommend just re-using the one you already made forever. They don't (usually anyway I haven't seen Apple's) have a timestamp in them so Apple has no reason to reject reuse unless their goal is specifically to waste your time.
Ok, so maybe the CSR is signed with my pre-existing "Apple Development" private key, which Keychain already has, which presumably was generated as part of the Xcode "register as a developer" fire drill.
I guess I'll find out next year if this works -- since you also have to do an annual fire drill for your Apple developer account, maybe those going stale will make the old CSR not work...
Again, I do not have a Mac, but my instinct was that this can't be correct and so I looked at people's screenshots of the Keychain Access application.
The application clearly mints a new pair of keys when you do Step 8 "Keychain Access.app / Menu 0 / Certificate Assistant / Request from a CA". It even has a "Let me specify key pair information" checkbox in the screenshots I looked at so you could choose a different type of key (don't, at best this won't help and at worst it introduces new and exciting ways for things to break).
The signature on the CSR is intended to match the public key inside it (the one minted as part of step 8), Apple is not relying on this signature to verify that you're really Jamie Zawinski the developer who paid them a bunch of money to ensure the beatings continue, they know that because you logged into their web site. If they check it, they only care that it matches the key inside.
In fact Apple may not care about the signature on the CSR at all, it's only a signed document to prevent you requesting certificates for somebody else's keys and they may consider that's not relevant to their threat model. A CSR may just be a convenient (for them) way to get you to tell them which public key should be in the certificate they're about to issue. They may not care what you write in those boxes for an email address and common name either.
And you know what you're doing. I thought renewing code-signing certs with GoDaddy was a hassle.