A rough week, but we survived
Whew. What a week it’s been. Two separate Microsoft issues in the span of seven days had email senders and admins scratching their heads, saw mail unexpectedly rejected, and left users temporarily locked out of their inboxes. Let’s recap what happened, and then talk about what it means for you.
First Issue: DMARC Rejections (November 6-9)
Starting Friday night, November 6th, email admins on the Mailop list and elsewhere began reporting that Microsoft was rejecting mail sent to Outlook.com and Hotmail recipients when the sending domain had no DMARC record. It didn’t matter if the message passed SPF and DKIM. No DMARC record? Bounce, with a 550 5.7.1 error stating that the message lacked SPF/DKIM/DMARC auth records.
This only hit Microsoft’s consumer mailboxes (outlook.com, hotmail.com, live.com, msn.com and friends), not Microsoft 365 business tenants. I confirmed it myself on Saturday. In my tests, mail to outlook.com bounced, while mail to a Microsoft 365-hosted domain delivered.
By Monday, November 9th, it was over. Microsoft appeared to roll back whatever change caused it, and a Microsoft support forum moderator suggested that it was unintended.
Second Issue: The Outage (November 12-13)
Next up: Starting late Thursday morning, November 12th (US central time), Microsoft services, including both consumer email (Outlook.com/Hotmail) and business email (Microsoft 365), started glitching. Users couldn’t get into webmail, and inbound mail was delayed. Senders saw a flood of temporary errors: “451 4.7.0 Temporary server error,” “421 4.3.2” concurrent connection limit messages, and “451 4.7.700 PFA agent busy.”
Microsoft’s status updates pointed to servers becoming overwhelmed by the process responsible for establishing secure connections, and noted that they were rolling back a recent network configuration change that correlated with the start of the problem. By Friday morning, November 13th, things were largely recovered, and people reported (myself included) that missing mail had started landing in their inboxes overnight.
A word for the System Admins
I want to be clear: this isn’t a post to pick on Microsoft. Running email at scale is hard. Really hard. Billions of messages. Attackers and spammers trying to get garbage through. There's a whole lot of infrastructure running a whole lot of complex code, doing a whole bunch of things, all at once. Things break sometimes. That’s true of every mailbox provider, and every email platform, too. (And I’ve written about this before.)
I know there are a lot of smart, dedicated people at Microsoft who had a very long week. They spent it investigating and fixing things, rolling things back, and getting mail flowing again, all while working to keep their users safe from bad actors. I have no bone to pick with any of them. I appreciate the work they do, and I hope this coming week is a much calmer one.
Now that we’ve recapped the week in review, and paused for a deep breath and a sip of our favorite beverage, let’s talk about what this past week means for senders.
Sender Lesson One: DMARC Isn’t Just for Bulk Senders
Google, Yahoo and Microsoft have all required DMARC (alongside SPF and DKIM) for bulk senders for a while now. But “bulk” has a definition, and a lot of smaller senders had yet to implement DMARC, because they weren’t sending enough mail to have to worry about bulk sender requirements just yet.
But, for those few days, Microsoft accidentally applied that DMARC requirement to everybody, regardless of how much mail they send. Was it intentional? No. But if you had a DMARC record in place, you sailed right through it. If you didn’t, your mail bounced.
That’s the lesson. Mailbox provider requirements change, sometimes on purpose and sometimes by accident, and you and I don’t always get a heads-up. If you use a domain to send email, you need DMARC. Even if you’re small. Even if you’re “just” sending one-to-one mail from your own domain. It’s a single DNS record, it’s easily doable, and if you’re struggling, there are lots of tools, and vendors, who can assist.
Sender Lesson Two: Issue Awareness
A day-long outage at a major mailbox provider means a day’s worth of mail that can’t be delivered. Mail that is temporarily deferred sits in a queue, on the disk of the outbound mail servers (MTAs) of whatever platform sent it, waiting to retry. Email service providers, CRMs and marketing automation platforms suddenly had to hold onto much more mail than usual, for much longer than usual.
So here’s a question for you: How did your platform handle it?
- Did their systems queue and retry gracefully?
- Did they tell you what was going on, or did you find out from a customer, a colleague, or a blog post?
- Did any of your mail end up bouncing because of this issue?
- Worse, did any of your subscribers get suppressed (marked as dead addresses), because this issue?
That last one concerns me the most. Check your reporting from those days. If you see an unusual spike in suppressed Outlook.com, Hotmail or Microsoft 365 addresses dated November 12th or 13th, talk to your platform about reactivating them.
Sender Lesson Three: Condensing Fact from the Vapor of Nuance
When a mailbox provider has a problem, they don’t always share a running play-by-play of what’s broken. Sometimes they can’t, because they don’t fully know themselves yet. Yet, email senders want answers – or at least, guidance. That’s where deliverability expertise comes in.
In the 1992 novel Snow Crash, Neal Stephenson describes the work of “condensing fact from the vapor of nuance.” That’s a big part of the deliverability job. You piece together what’s happening from the clues dancing around the edges: an unexpected bounce, a spike in deferrals, chatter on Mailop, Reddit, or industry forums, a status dashboard update, a vendor’s incident notice. Then you add context and past experience to fill in the gaps.
Maybe you figured out what was going on enough to add your own DMARC record to DNS last weekend. Maybe you stayed calm while waiting for queues to start draining during the most recent outage.
Or, maybe not. Even if you were confused by it all, now you’re a little better informed, you’ve now got a bit of that experience and history, so that you can better understand what’s happening, the next time something like this happens, whether it be at Microsoft or somewhere else.
It was a rough week. But we survived.