I am attempting to use Amazon SES as my first-hop relayhost out of Postfix, since many sites are preemptively blocking connection from my EC2-hosted mail server. I have done the DNS dance, verified my domains and DKIM, and when my instance sends mail with any address ending in @dnalounge.com in the From: line, it is accepted by the SES relayhost and delivered. That's fine.
But none of my employees can receive mail, because this happens:
"From: foo@bar.com; To: employee@dnalounge.com" arrives at my mail server. The aliases file leads me to forward that message along to employee's real address, example@gmail.com. So then SES says:
"554 Message rejected: Email address is not verified. The following identities failed the check in region US-WEST-2: foo@bar.com"
How I fix.
You won’t like this, but: get out of the mail forwarding business if you possibly can. It makes you look like a spammer to others, and your real outbound traffic suffers. It’s not worth it.
Also what Ben says
Reading these two comments was exactly the right time to find the "HERP DERP" button.
Fuck off.
Its been a while since ive used postfix, but most likely a map for employees, not relaying through SES?
And when one of those employees has the bad taste to be using Yahoo for their email, who won't accept mail from my server?
Sounds like what would happen if bar.com publishes a DMARC p=reject policy. (I assume you’ve already ruled that out, but I’ve had to learn more about DMARC than a self-respecting human should and I can’t get past it without an explicit denial.)
Verify those addresses directly via SES. Go to the SES console -> Identity Management -> Email addresses, enter the employee's email address. Tell them to click on the confirm link that will arrive in their inbox. Now pour yourself a drink and do it for all of your other employees. You can at least script this using the AWS CLI tool.
Welcome to life in the Clown. :(
No, this level of fuckery with onboarding is not tractable.
If this is really how it works, then I am soliciting suggestions of other email provider relayhosts that actually function.
An old comment from an AWS representative says SES doesn't support relaying, and that still appears to be true. Looks like you need to find another email provider to avoid needing to mangle the mail that passes through your system.
Last time I dealt with this, smtp dot com did not suck as an outbound relay that got mail delivered. That was probably 5yr ago (SES sucked then too)
I'm not convinced that what you suggested would work anyway, since it's complaining about the 3rd party customer in the From line, not about either of the addresses of my employee.
Ugh, actually you're right. Sorry, it was late and I hadn't had to deal with this end of SES for a while.
SES really just isn't meant to be a general-purpose SMTP relay: as ChoHag said below it's really just meant to be a spam outflow so large and nominally accountable that the world won't auto-block it the way they do the EC2 networks.
If you wanted to attack the problem with anger, guile and a certain amount of applied stupidity, you could maybe rewrite the messages en route to have an envelope sender and From address @dnalounge.com but have a reply-to header pointing at the original sender. (Is procmail even still a thing? You could probably do that with procmail.) No guarantees that amazon wouldn't notice that kind of trickery, but at the volumes you're talking about it's probably less than line noise to them so you might get away with it.
I'm trying to think of some reasonably non-obnoxious solution to this that won't end up tacking on another $25-50/mo to your operational cost here and so far coming up empty. :(
As Dr Memory says, you need to verify addresses before you can send from them.
https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-domains.html
Also, you may have already done this, but you will need to have your account “removed from the SES sandbox” before you can send to unverified domains.
https://docs.aws.amazon.com/ses/latest/DeveloperGuide/request-production-access.html
the truly obnoxious thing is, you have to ask for the production access in each region, so if you initially did the request in us-east-2, you now get to ask for it again in us-west-1!
are you supplanting any of the From, Return-Path, Reply-To headers with a dnalounge.com address? without rewriting those, it looks like the sender is the vendor, not dnalounge.com :/
No I'm not rewriting From because I'm not the originator and so that would be insane.
And it doesn't "look like" it's from someone else because my smtp connection from my server to Amazon is authenticated with the uid and pass that Amazon generated for my account - from the IP Amazon assigned me on the virtual machine that they run.
So no. Don't try and tell me Amazon is somehow "confused" about this mail or "can't know" that the person asking that it be delivered is me, their fucking customer.
You may be in for a hard time because you're doing it wrong. Specifically you're assuming that SES is an smtp relay but in fact it's a spam relay that's simply so large the world is forced to accept it (this is a bit like the way Russia's totally not at war even though it obvious to anyone smarter than a brick that they're at war because if we called it a
spam relaywar thentheir moneyour money might go somewhere else).Specifically, https://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-email-addresses.html seems to imply that all four headers must be verified: 'You must verify each identity that you use as a "From," "Source," "Sender," or "Return-Path" address' (this being English, or seems to mean and in that sentence) but it does hint that you can stop them considering an address you've already verified by abusing the spec horribly to add '+foo' 'labels' to them.
If you can't get it to ignore the from field I think the only options are to use a real mail relay or go nuts with address rewriting. This is as dumb as it sounds. I'd like to hear that I'm wrong and SES can at least be used as an smtp relay but it literally has a thing called a "reputation dashboard".
I ran a self-hosted email server for a decade, and finally gave up. I don't have anything to offer here, except a tangential observation. Even if you run your own server, you have to relay through one of the "biggies" to get your mail accepted at any of the others, regardless of all the mechanisms that were supposed to fix this. Large corporations have, once again, seized something by fiat that should have been free and open, under the guise of "security."
I can't even figure out how to make Postfix deliver "From: *@dnalounge.com" through SES while (trying and often failing) to deliver everything else directly because the interaction between sender_dependent_default_transport_maps, sender_dependent_relayhost_maps, transport_maps and default_transport is completely incomprehensible.
I ran into this trying to get my own mail forwarded to my own Google account. The problem I kept running into was that I couldn't receive mail at Google because Google made a private agreement with example.com such that Google wouldn't accept any mail with a "From" header containing "example.com" that didn't come from example.com infrastructure. This is why so many phishers can't spell bank names correctly, and it's why you can't forward mail from example.com to anyone important enough that example.com might want to make private agreements with them.
So...that was the end of that. I gave up, and just had Postfix feed that mail to a Perl script LMTP server (alias the recipient to a virtual domain, then map the transport for that virtual domain to LMTP), which mangled the headers and fed the mail back to the outgoing queue. I can't figure out how to make Postfix do two separate things to the same email even though I have written entire MTAs from scratch.
The script mangles the addresses by appending ".$hash.fwd.mydomain.org". Any replies to that domain would be captured, authenticated, demangled, and forwarded back. The $hash was used as an authenticator, combining a secret with the original domain. Most spam filter UIs demand stable correspondent addresses. Google doesn't like seeing stuff that looks like a phisher misspelling "example.com", but at least that mail ends up in the spam folder instead of being rejected outright at MAIL FROM:.
Glad to see I'm not the only one. IMHO postfix' configuration settings are an arcane mess, though so far I managed given enough time wading through their docs. At least they understand the idea behind Hypertext.
Don’t try to think of SES as a mail server; think of it as a “do not reply to this email” mailing list origin. Every sender must be whitelisted, plus you really have to be on top of your To: list management; it doesn’t take many bounces for them to put you on prohibition.
The Internet has changed a lot in the past 20 years - it’s almost impossible to run your own mail server these days unless you’ve already been running it impeccably for 20 years on the same IP and have that history.
AWS doesn’t really have a product that is both a well-trusted email source and allows a remote origin. You’d be better off either (a) using an ISP just for your email (I use bluehost, YMMV) or (b) biting the bullet and moving your email to a hosted email provider like gmail (which may require $$$ depending on how many users you need.)
You probably don't actually want to use SES for this.
Since you're already running postfix you can just keep doing that. If you don't already have an elastic IP, get one, then cut AWS a ticket to get the reverse DNS set to something that's not AWS: https://portal.aws.amazon.com/gp/aws/html-forms-controller/contactus/ec2-email-limit-rdns-request
You'll need to set up an A record pointing to the box before they'll create it, but presumably you have an easy way to do that. I've been doing this for some number of years without having to deal with SES.
Oh, I'm shitty at reading. Maybe it's just because I've had the same elastic IP for some number of years or I got in early enough to where it didn't get blacklisted for abuse, but you can always do the annoying thing where you change the IP again if it's not deliverable.
Are you applying SRS when forwarding your mail? If not, that could be the problem; I would suggest using PostSRSd. Yes, SRS is stupid and bad, but that is the world we live in.
Well, I tried to make this happen:
• Emails sent to the Announce list, and transactional emails from the store, would go through SES;
• Other emails go direct.
I failed.
I can't figure out any combination of /etc/postfix/transport and /etc/postfix/header_checks that will allow me to do that.
I thought I could key off of ^Return-Path: .*-(request|bounce)@dnalounge\.com but that didn't work, and neither did a few other things I tried.
So I guess I'm back to not using SES, sending email directly always, and playing whack-a-mole when someone decides that my IP is ritually unclean.
"none of my employees can receive mail".
Known jwz skills:
- email client.
- app that runs on phones.
- doing convoluted things to work round corporate craziness
I should probably apologise in advance: sorry.
While I don't live under a bridge, feel free to assume I'm a troll. On the off-chance you read this and it doesn't just get automagically directed into a bit-bucket of rotten slops...
If you want a sane email provider, I can recommend https://runbox.com Their service is not free-as-in-beer, but I have had no issues with their service for many years now.
And thank-you for XScreenSaver.
I'm a runbox customer too, and I use them as a smarthost with my own domain. Things that might put you off: not cheap, and they are in Norway.