Hurricane Irma & Luck

As some of you may know, my wife and I (and dog) moved to the Miami area just a couple of months ago. We actually donated most of our furniture before we moved, because it was mostly junk, and because we were downsizing. So the past couple months before the hurricane, we had gone to IKEA Miami approximately nine hundred times, dragging individual pieces of flat-pack furniture home and painstakingly ignoring the silly IKEA directions. (Proving both that I can assemble a KALLAX from memory, and that a marriage is forever if it survives nearly two-dozen IKEA trips.) We had just gotten to the point where we had enough simple furniture to consider our new place home. Everything was unpacked. All the pictures were hung on the walls. No more travel for a while. We truly were where we wanted to be.

And then Hurricane Irma was coming and we had to get out of town. And we were lucky we were able to get out of town. We've been on the road, waiting for things to calm down, so we could return home. The past ten days were stressful, but we treated it mostly as an unexpected vacation and the worst thing that happened was that we overspent our food budget by eating out the whole time. My employer was not only understanding, but also provided assistance. We got through it just fine.


Unlike what other folks went through, in our case the storm mostly just did a number on the greenery around our apartment building.

Like I keep saying, we are lucky, and we really can't complain. But over on the other side of the country, my friend Karen isn't as lucky. Karen is a long time friend of mine, and past coworker, and she really knows what she is doing when it comes to email technology and deliverability. Her and her partner have fallen on hard times after she had to take medical leave. They are currently homeless. I am asking you please consider donating to Karen's GoFundMe to help them get an RV and some other stuff to get stable and so she can be able to work remotely. (And then you should hire her if you need deliverability help -- she's fantastic at it!) 

Karen is smart and sharp and I think she fell into a trap that any of us could fall into, and I think that with just the right amount of help she can get back on her feet. I hope you'll consider donating.

Full Email Headers

Email message headers have a lot of information in them. Only the tiniest amount of that header information is typically exposed to a recipient by default. There's much more to it than just the from address, to address, subject line and date. There's routing information, authentication information, IP address information, and more information hiding just beneath the surface, in the "full headers" or "internet headers" of that email message. To get at the rest of that header information associated with a given email message, a recipient will need to "view full headers" or "view the message source."

Here's a guide from MXToolbox on how to get full email header information from various email clients.

Google explains how to obtain full headers from Gmail, and even shows you how to extract valuable information from them.

Microsoft explains how to obtain full headers from various versions of the Outlook email client.

Pro tip: When somebody asks you to send them email headers, don't send them a screen shot of the from address or email body. That's not what they're asking for. (And screen shot images aren't searchable; if somebody is trying to help you troubleshoot something, it is painful to have to manually read each line of a header in an image to try to find the right header.) Instead, send them a text file attachment containing all of the copied-and-pasted email header info, or full message source.

Hotmail UK MX Change

The MX servers for Hotmail UK changed in DNS overnight. For those who are caching DNS or have hard coded DNS entries, you're going to run into problems with false positive "user unknown" bounces. Time to flush the cache and make sure you're relying on what's currently in DNS.

I'd include the specific domains here, and their new MX records, but I don't want to lead people down the wrong path of hard coding MX record entries, when they really should be relying on DNS.

Haiku Break

So, what do we do?
Deliverability!
How? Nobody knows!
-- Courtesy of my friend Sarah

Finding DMARC when it isn't there?

If you're running a mailing list manager, or dealing with a reply-handling mail system, like many ESPs have in place, you've learned that DMARC adds a number of challenges to this process. To deal with that, you have probably modified your software to watch for any domain that has a p=reject DMARC policy and you special case those, rewriting the from address and perhaps preserving the original sender information elsewhere in the message.

This works great...until it doesn't. I've started to notice that message forwarding can be problematic for some domains that don't actually publish a restrictive DMARC policy. AOL and Yahoo's consumer webmail domains already have a p=reject policy in place, but Microsoft and Google don't have similar restrictions applied to gmail.com and hotmail.com/outlook.com.

But, you might as well start treating them as if they had p=reject, because ISPs practically seem to be doing that already. Mailing list messages from users at those domains fail authentication, and in many cases, end up blocked or diverted to the spam folder, for that authentication failure alone. It almost seems as if Google, in particular, watches for any sort of DMARC policy (even p=none) and treats a message more harshly if authentication or DMARC checks fail, for any of those domains.

So what I've done in my mailing list manager a bit more aware. If a domain has ANY type of DMARC policy in place, it will rewrite the from header, to minimize chances of any issue with delivery of that mailing list message.

I also spent a few minutes today compiling a list of "MAGY" domains. What are MAGY domains, you might ask? I think I got this term from Return Path -- it's shorthand for "Microsoft, AOL, Google, and Yahoo" and is a good way to quickly refer to the top webmail providers. I took that list of corporate and webmail domains for those companies, some of which have p=reject in place, and some do not, and set that up so that I'll just proactively special case mail from those domains, rewriting from addresses as if the domain had a restrictive p=reject DMARC policy, to save myself the headache of wondering how the end hop ISP is going to treat mail from that domain.

So far, my treating any DMARC policy as if it is p=reject is working fine for me. It might not be a good solution for everyone, but it has saved me some troubleshooting. Will it help to add in the MAGY primary domain list and treating all of those domains similarly? Time will tell.

16 Years And Counting

LinkedIn just reminded me: Spam Resource is just about sixteen years old, give or take a week or two. After sixteen years (and over 1.6 million page views), I don't always get the chance to post as often as I used to; life is busy and fast moving. I really can't complain, though, and I'm glad to be able to share what I can, when I can. I hope you all still find it valuable.

People still seem to be visiting; looking for information on topics like backscatter, how list-unsub works in iOS, how to use Microsoft SNDS, or what domains are hosted by Yahoo. I'll keep trying to add to this knowledge base as time permits.

Thanks to everyone who was kind enough to drop me a note and thanks to everyone who has visited the site, left a comment, or been kind enough to author a guest post at some point over these past years.

-- Al Iverson

DMARC will (not fully) fix it!

Over on the Word to the Wise blog, Laura Atkins raises the concern that in response to the recent "prank" email scandal, email technologists are likely to say that this couldn't have happened if the White House had implemented DMARC. She's correct in that DMARC wouldn't have helped in this case, but that doesn't mean we should write off DMARC just yet. It's actually part of the solution to this very issue. Just not the whole solution.

The use for DMARC here is to close a door before this or some other prankster or hacker decides to open it. The White House and other government agencies should be using DMARC, because keeping bad mail from using the legit entity's exact domain name is part of a best practice anti-spoofing strategy.

Internet privacy and security guru Jim Fenton tackles the user interface issue at play here, that only the "friendly from" text was displayed, and not the actual email address. He suggests that mail clients and webmails should change their user interface to show some level of trust-like information, when it's available. Is the email address of this sender in your address book? That's a good start. I'd add to it: Is this email address one you have emailed previously? If you initiated conversation to them previously, that's a potential starting point for trust. (And of course, did each message received authenticate properly, and does it comply with DMARC policy for the domain?)

So that's two considerations so far: preventing mis-use of a specific domain name, and deciding when you can or should trust the friendly from of a sender.

There's a third: The end user, or how much do you trust the end user to do the right thing? In the screen shot shown on CNN, the email message has a big [SUSPECTED SPAM] tag right on it. I wonder why -- because it didn't authenticate properly? If I were a super elite hax0r trying to twiddle the friendly from field, I'd probably be doing this from a linux box with a script, or via some open source email application, and I'd probably be trying to hide tracing messages back to me, maybe by using "open relay" insecure mail servers or something along those line. Something in the process made the receiving server suspicious of that message. But the server delivered the message to the recipient anyway. Should it have? And after it did, shouldn't the end recipient been suspicious and shouldn't they have looked into it further?

I imagine the ISP's point is that they're loathe to block a message that could be legitimate, even though it appears not to be. But I think we just found a gap in that process, trusting that the end user knew what this meant or that they knew what they should do about it. Maybe this message shouldn't have been delivered to begin with? There's a third consideration right there.

And there's a fourth consideration as well, the cousin domain problem. Agari's Bob Boucneau touches on that here. That problem is a real one and can be tricky. But ultimately, email authentication still helps here, too. That other "White House" can authenticate, but that authentication gives them a steady identifier to attach reputation to. If somebody registers whitehouseemail2.com, they can easily apply SPF, DKIM and even DMARC, but if they do something bad with that domain, that domain's reputation is going to suffer, and it'll be easier for smart ISPs to block mail using that domain. (And at the same time, don't confuse users by using cousin domains for legitimate reasons if you can prevent it. The more variables you legitimize for end users, the more you diminish their ability to understand how to know whether or not that message is legitimate.)

There's probably even some other fifth aspect of this that I haven't considered. But off the top of my head, it sure looks to me like email authentication and anti-spoofing measures would be part of any comprehensive solution to try to prevent or mitigate this type of thing, and it looks to me like that ought to include DMARC.

So, yeah, I'm going to keep pushing people to implement DMARC. I still think it's the right thing to do.

Most federal departments aren’t using DMARC: Wyden

Found on the Sophos Naked Security blog:  Senator Ron Wyden (D-Oregon) contacted the US Department of Homeland Security in a July 18 letter where he 'asked the agency to “take immediate steps” to mandate that all federal agencies implement DMARC (Domain-based Message Authentication, Reporting and Conformance), an email authentication, policy, and reporting protocol launched in 2012 that helps prevent email domain spoofing.' He noted that DMARC has been implemented by very few government agencies to date.

This is good to see, and one hopes it helps drive DMARC adoption. It's not a phishing cure-all, but I still think it's an important step in the fight to reduce the risks around email forgery.