Finding and deciphering email headers

Here's an article from Lifewire (formerly about.com) on how to get at and decipher full email headers. This link is worth saving for later, if you struggle with this.

H/T: Brian Krebs

Krebs: Phishers Are Upping Their Game. So Should You.

Brian Krebs recently published a most excellent article on the current state of phishing and what you can and should do to try to protect yourself as an email user. It's definitely worth a read.

The 12 Days of Listmas

🎵 On the first day of Listmas
   🎵 my data showed to me
🎵 something’s causing slow delivery…

Over on Spamtacular, Mickey Chandler is  publishing a series of posts for the 12 days of Listmas, talking about the things that can go wrong with deliverability and list hygiene during this most wonderful time of the year. Here's day one. And here's day two.

Help! I'm blocked at Verizon.net!

Hey! Are you finding your mail rejected when trying to send to verizon.net?

Remember that verizon.net email users now have their mail handled by AOL's systems. That means, if you're blocked at verizon.net, you're blocked at aol.com (as well as aim.com, cs.com, compuserve.com, netscape.net, wmconnect.com, wow.com, ygm.com).

So what you have here is an AOL issue, not a "Verizon" issue. What to do about AOL issues, I've blogged about right over here.

For a full list of all AOL domains, click here.

Note: AOL and Yahoo Mail are now owned by the same company, but as of December, 2017, their systems have not been combined. Thus, you should continue to handle AOL and Yahoo issues completely separately.

Vodafone (New Zealand) Email Closure

Vodafone (NZ) is reporting that they have shut down their email service as of November 30, 2017.

This affects the following domains: clear.net.nz, es.co.nz, ihug.co.nz, paradise.net.nz, pcconnect.co.nz, quik.co.nz, vodafone.co.nz, vodafone.net.nz, and wave.co.nz.

However, don't purge those addresses from your lists just yet. Vodafone is allowing users to (at least temporarily) continue to receive mail at those accounts, and the mail will be forwarded on to a new email address as configured by the user.

What should senders do? Watch for bounces; bounces will definitely increase at those domains. Those bounces mean that users have not set up email forwarding, and their accounts are thus now closed. You may want to attempt to mail those subscribers, to find the ones whose accounts still forward, to request that they update their email addresses registered with you.

Top MAGY Domains

Sometimes it is handy to have a list of the top MAGY (Microsoft, AOL, Google, Yahoo) email domains. Here's what I've compiled based on current and prior research, plus input from readers.

AOL and Yahoo are merging, but I've left them distinct for now. Also, some of these domains might not be common email domains, but they all seem to have valid MX records and thus if you're looking for a list of domains to, for example, exempt from validity checking so that they never falsely appear to be dead domains, it'll still come in handy.

If you use this to group domains to count sending mail volume by ISP, it will be slightly imperfect, as I've included the corporate domains (microsoft.com, google.com, teamaol.com, oath.com, yahoo-inc.com) that may use separate mail systems. However, your send volume to those domains is probably pretty low compared to the consumer mailboxes, so for "back of napkin" kind of math in a pinch, it might do just fine.

Note that the intent here is to list email domains owned and used by that given provider; not to list every possible customer domain that points at the ISP's mail servers. I'm not capturing Google Apps domains, Yahoo Business Hosting domains, or anything like that. That's out of scope for this exercise.

Holiday Season is Here: What to do

Hey big sender, it's that time of the year again! Lots of companies generate lots of revenue during the run up from about now until Christmas, and email is a very important channel for them to do that.

That's why it's important for senders to follow these five deliverability recommendations to ensure they minimize deliverability issues during this important time.

October 11: Spam in the News

Wondering if government action against spammers still happens? Here's an example from the UK. MediaPost's EmailMarketingDaily reports that "two UK firms were fined thousands of pounds by the Information Commissioner’s Office (ICO) for allegedly spamming consumers." Read all about it here.

HTML Email: JWZ's fault?

One-time Netscape Navigator developer Jamie Zawinski suggests that HTML support in email might have been his fault.

Whoops: iOS 11 Mail Microsoft Issues

If you've upgraded your iPhones to the just-released iOS version 11 and your email is hosted by Microsoft's Outlook.com, Office 365 or Exchange 2016, you may now find yourself unable to send email from your iPhone. It sounds like a known issue and that it is being worked on.

In the mean time, the most commonly suggested workaround seems to be that you should download Microsoft's Outlook email client for iOS and use that instead.

Full Email Headers

Email message headers have a lot of information in them. Only the tiniest amount of that header information is typically exposed to a recipient by default. There's much more to it than just the from address, to address, subject line and date. There's routing information, authentication information, IP address information, and more information hiding just beneath the surface, in the "full headers" or "internet headers" of that email message. To get at the rest of that header information associated with a given email message, a recipient will need to "view full headers" or "view the message source."

Hotmail UK MX Change

The MX servers for Hotmail UK changed in DNS overnight. For those who are caching DNS or have hard coded DNS entries, you're going to run into problems with false positive "user unknown" bounces. Time to flush the cache and make sure you're relying on what's currently in DNS.

I'd include the specific domains here, and their new MX records, but I don't want to lead people down the wrong path of hard coding MX record entries, when they really should be relying on DNS.

Haiku Break

So, what do we do?
Deliverability!
How? Nobody knows!
-- Courtesy of my friend Sarah

Finding DMARC when it isn't there?

If you're running a mailing list manager, or dealing with a reply-handling mail system, like many ESPs have in place, you've learned that DMARC adds a number of challenges to this process. To deal with that, you have probably modified your software to watch for any domain that has a p=reject DMARC policy and you special case those, rewriting the from address and perhaps preserving the original sender information elsewhere in the message.

This works great...until it doesn't. I've started to notice that message forwarding can be problematic for some domains that don't actually publish a restrictive DMARC policy. AOL and Yahoo's consumer webmail domains already have a p=reject policy in place, but Microsoft and Google don't have similar restrictions applied to gmail.com and hotmail.com/outlook.com.

But, you might as well start treating them as if they had p=reject, because ISPs practically seem to be doing that already. Mailing list messages from users at those domains fail authentication, and in many cases, end up blocked or diverted to the spam folder, for that authentication failure alone. It almost seems as if Google, in particular, watches for any sort of DMARC policy (even p=none) and treats a message more harshly if authentication or DMARC checks fail, for any of those domains.

So what I've done in my mailing list manager a bit more aware. If a domain has ANY type of DMARC policy in place, it will rewrite the from header, to minimize chances of any issue with delivery of that mailing list message.

I also spent a few minutes today compiling a list of "MAGY" domains. What are MAGY domains, you might ask? I think I got this term from Return Path -- it's shorthand for "Microsoft, AOL, Google, and Yahoo" and is a good way to quickly refer to the top webmail providers. I took that list of corporate and webmail domains for those companies, some of which have p=reject in place, and some do not, and set that up so that I'll just proactively special case mail from those domains, rewriting from addresses as if the domain had a restrictive p=reject DMARC policy, to save myself the headache of wondering how the end hop ISP is going to treat mail from that domain.

So far, my treating any DMARC policy as if it is p=reject is working fine for me. It might not be a good solution for everyone, but it has saved me some troubleshooting. Will it help to add in the MAGY primary domain list and treating all of those domains similarly? Time will tell.

16 Years And Counting

LinkedIn just reminded me: Spam Resource is just about sixteen years old, give or take a week or two. After sixteen years (and over 1.6 million page views), I don't always get the chance to post as often as I used to; life is busy and fast moving. I really can't complain, though, and I'm glad to be able to share what I can, when I can. I hope you all still find it valuable.

People still seem to be visiting; looking for information on topics like backscatter, how list-unsub works in iOS, how to use Microsoft SNDS, or what domains are hosted by Yahoo. I'll keep trying to add to this knowledge base as time permits.

Thanks to everyone who was kind enough to drop me a note and thanks to everyone who has visited the site, left a comment, or been kind enough to author a guest post at some point over these past years.

-- Al Iverson

DMARC will (not fully) fix it!

Over on the Word to the Wise blog, Laura Atkins raises the concern that in response to the recent "prank" email scandal, email technologists are likely to say that this couldn't have happened if the White House had implemented DMARC. She's correct in that DMARC wouldn't have helped in this case, but that doesn't mean we should write off DMARC just yet. It's actually part of the solution to this very issue. Just not the whole solution.

The use for DMARC here is to close a door before this or some other prankster or hacker decides to open it. The White House and other government agencies should be using DMARC, because keeping bad mail from using the legit entity's exact domain name is part of a best practice anti-spoofing strategy.

Internet privacy and security guru Jim Fenton tackles the user interface issue at play here, that only the "friendly from" text was displayed, and not the actual email address. He suggests that mail clients and webmails should change their user interface to show some level of trust-like information, when it's available. Is the email address of this sender in your address book? That's a good start. I'd add to it: Is this email address one you have emailed previously? If you initiated conversation to them previously, that's a potential starting point for trust. (And of course, did each message received authenticate properly, and does it comply with DMARC policy for the domain?)

So that's two considerations so far: preventing mis-use of a specific domain name, and deciding when you can or should trust the friendly from of a sender.

There's a third: The end user, or how much do you trust the end user to do the right thing? In the screen shot shown on CNN, the email message has a big [SUSPECTED SPAM] tag right on it. I wonder why -- because it didn't authenticate properly? If I were a super elite hax0r trying to twiddle the friendly from field, I'd probably be doing this from a linux box with a script, or via some open source email application, and I'd probably be trying to hide tracing messages back to me, maybe by using "open relay" insecure mail servers or something along those line. Something in the process made the receiving server suspicious of that message. But the server delivered the message to the recipient anyway. Should it have? And after it did, shouldn't the end recipient been suspicious and shouldn't they have looked into it further?

I imagine the ISP's point is that they're loathe to block a message that could be legitimate, even though it appears not to be. But I think we just found a gap in that process, trusting that the end user knew what this meant or that they knew what they should do about it. Maybe this message shouldn't have been delivered to begin with? There's a third consideration right there.

And there's a fourth consideration as well, the cousin domain problem. Agari's Bob Boucneau touches on that here. That problem is a real one and can be tricky. But ultimately, email authentication still helps here, too. That other "White House" can authenticate, but that authentication gives them a steady identifier to attach reputation to. If somebody registers whitehouseemail2.com, they can easily apply SPF, DKIM and even DMARC, but if they do something bad with that domain, that domain's reputation is going to suffer, and it'll be easier for smart ISPs to block mail using that domain. (And at the same time, don't confuse users by using cousin domains for legitimate reasons if you can prevent it. The more variables you legitimize for end users, the more you diminish their ability to understand how to know whether or not that message is legitimate.)

There's probably even some other fifth aspect of this that I haven't considered. But off the top of my head, it sure looks to me like email authentication and anti-spoofing measures would be part of any comprehensive solution to try to prevent or mitigate this type of thing, and it looks to me like that ought to include DMARC.

So, yeah, I'm going to keep pushing people to implement DMARC. I still think it's the right thing to do.

Most federal departments aren’t using DMARC: Wyden

Found on the Sophos Naked Security blog:  Senator Ron Wyden (D-Oregon) contacted the US Department of Homeland Security in a July 18 letter where he 'asked the agency to “take immediate steps” to mandate that all federal agencies implement DMARC (Domain-based Message Authentication, Reporting and Conformance), an email authentication, policy, and reporting protocol launched in 2012 that helps prevent email domain spoofing.' He noted that DMARC has been implemented by very few government agencies to date.

This is good to see, and one hopes it helps drive DMARC adoption. It's not a phishing cure-all, but I still think it's an important step in the fight to reduce the risks around email forgery.

AOL: Reputation corrected and request denied

Check out the reply I received in response to a recent AOL Whitelist Request submission:
Subject: Reputation corrected and request denied
Your Whitelist request, with the confirmation code X, has been denied. 
The requested IP address(es) is receiving temporary failures due to poor reputation. We have corrected the reputation and this should help in better delivery of mails. Please monitor the spam complaints via the feedback loop and re-apply for Whitelist after you have built a good 20-day history on your IPs. Also, check the reputation of the IP before opening the ticket at: https://postmaster.aol.com/ip-reputation.
Feedback Loop Request form can be found at: https://postmaster.aol.com/fbl-request
Have you seen this one before? I haven't. I think it's really cool, though. It explains what they've done and what you need to do, if you want to get whitelisted at AOL.

Here's what they're saying:
  1. Your sending IP address doesn't have a great sending reputation today.
  2. But, AOL has reset the sending reputation of your IP address, giving you another chance to build a good reputation.
  3. They're telling you to keep your nose clean (build a good reputation) for at least 20 days before applying for whitelisting.
  4. They're reminding you to sign up for AOL's ISP Feedback Loop.
Seems pretty straight forward to me. I wish all ISP responses were this clear and easy to understand. Good job, AOL!

Text to Image ratios in email

Laura Atkins of Word to the Wise explains: "The text to image ratio is not going to make or break delivery." I've certainly had people try to tell me that they think the secret to inbox placement is based on a certain specific text-to-image ratio. Like Laura, I know that this is not true, and I am happy to link to her excellent explanation of how this all works.

Verizon Email Transition Update

From Network World, here is an update on the winding down of Verizon's email service, which I previously reported on here. When is the transition happening?
"Once customers are notified, they are presented with a personal take-action date that is 30 days from the original notification. If you happen to miss the deadline and still want to retain your address, you can choose Option 1 and switch over to AOL.  
"Based on the current rate of migration it looks like Verizon will probably get through all of the customer notification waves by mid-summer. At that point, the company will assess when the platform might be entirely wound down."
Note: Like I mentioned before, keep in mind that subscribers can indeed keep their verizon.net email if they like. It'll just be handled by AOL's systems and user interface going forward.

Network World says that "Verizon controls 4.5 [million] Verizon.net email accounts, and [Verizon] figures about 2.3 million of them are active." Active meaning that they have been accessed in the last 30 days.

June 26, 2017 Update: As related to me and others by AOL, the Verizon mailboxes that remain have now been transitioned to AOL's mail servers.

New Anti-Phishing Protection in Gmail on Android

Gmail app users on Android, rejoice -- Google just added phishing protection. If you try to click on a link deemed to be problematic, you'll get warned: “The site you are trying to visit has been identified as a forgery intended to trick you into disclosing financial, personal, or other sensitive information,’ the notice states. “You can continue to [the link] at your own risk.”

Read more about it over at the Consumerist.

Why list-unsub doesn't let you "opt-down"?

If you're familiar with the "list unsubscribe" functionality, support for which is implemented in Apple's iOS Mail Client, as well as Gmail and Outlook.com, you might wonder why these implementations might not allow you to land at a preferences page when clicking on the link. Clients have certainly asked me why they aren't allowed to add a step in the middle of this process -- instead of just logging the unsubscribe request, can't they ask the subscriber if they might want to receive fewer emails (opt-down instead of opt-out) or otherwise adjust their preferences, instead of losing them?

The problem here, is differing expectations between marketers and internet service providers (ISPs).

Microsoft's Terry Zink explains specifically why Outlook.com does not support the HTTP method of list-unsubscribe (which would potentially allow driving to a preferences page instead of just capturing an opt-out): Because it's their user interface, and it's up to them (Microsoft) to ensure that their users have the best experience possible, and they really intend this to be a simple "unsubscribe" and nothing more. He explains that #1 the experience is really supposed to be "you are unsubscribed," not click this checkbox or hit this button, and #2, he explains their concerns over the potential for a third-party interface not necessarily spinning up properly, resulting in a poor subscriber experience, and an unlogged unsubscribe request.

Jump on over to Terry Zink's blog post where you can read it in his own words. That, in a nutshell, is why it works the way it does, at least as far as Microsoft's Outlook.com platform is concerned.

Orange UK Email Closure

United Kingdom-based ISP Orange (now part of Everything Everywhere aka EE) has announced that they are shutting down their email service as of May 31, 2017. This affects users at these domains: orange.net, orangehome.co.uk, wanadoo.co.uk, freeserve.co.uk, fsbusiness.co.uk, fslife.co.uk, fsmail.net, fsworld.co.uk, fsnet.co.uk. This does not affect non-UK Orange email users.

Follow this link for more details.

New DMARC Record Lookup Tool

If you use the DNS tools over on XNND, you might notice that the DMARC record lookup feature now links to a new DMARC record lookup tool, kindly provided by Steve Atkins of Word to the Wise. Thanks, Steve!

Senders: What should you do about verizon.net?

As mentioned in a previous post, Verizon recently announced that it is getting out of the email hosting business.

Verizon users are being given a choice. They can:
  • Give up their verizon.net email address; or
  • Continue to use their verizon.net email address, but it will now be hosted by AOL.
Note also that Verizon has yet to announce a deadline as to when users will have to make this choice.

What does this mean for senders? What should senders "do" about verizon.net subscribers?

Truth be told, senders really shouldn't have to do anything here. The email domain verizon.net is not going away. Current users are going to be able to keep their existing email addresses. It is just that going forward, those email addresses will be hosted by AOL.

If a user chooses to give up their verizon.net email address, mail to that address with eventually start bouncing. Any good list management software or ESP platform should be able to denote those bounces and suppress that email address, after it goes dormant. There really shouldn't be any negative impact to email deliverability.

Even if senders wanted to do something with this segment of subscribers -- there isn't much they could do. You could email all of your verizon.net subscribers today, but what would you tell them? You don't know which ones plan to keep their email address and which ones plan to move.

I think all senders really need to do here is:
  1. Keep your regular mailing cadence;
  2. Make sure you have an easy "update your email address" process as part of your email footer; and
  3. Make sure your email platform automatically recognizes and suppresses no-longer-valid email addresses.
Beyond that, really, no special action is needed.

ETA: There's certainly no harm if you want to send all verizon.net users an email message asking them if they want to update their email address. I think it all boils down to how much effort you want to put forth to try to save a percentage of a percentage of users. How big the value you'll get from that is really impossible to measure without doing it. That's why my guidance is to just let the email platform work it out for you.

Do you care about WHOIS?

There's an effort underway within ICANN where the net result could be that publicly-available domain ownership info is no longer available under any circumstances. Does that strike you as the best way to go? I personally don't think it is. WHOIS is a valuable forensic tool for security researchers and spam filterers.

Does a business have a right to privacy on the internet? I say no. If you're a company in the real world, your company registration is public knowledge. If you're a company on the internet, shouldn't that registration information be at least as available?

If WHOIS matters to you, please consider joining the "Next-Generation gTLD Registration Directory Services to Replace Whois" ICANN group and sharing your opinion, voting in the surveys they publish, and responding to the questions people may ask about why WHOIS matters.


Verizon Users: Leave, or move to AOL

As reported last May, the Verizon email user transition to AOL continues, with Verizon now announcing that they have decided to "close down our email business."

Existing verizon.net email users have two options:

  1. Continue to use their verizon.net email address, but it will be hosted by AOL Mail.
  2. Give up their verizon.net email address and find a new email/webmail provider.

From a deliverability perspective, I think it is safe to assume that most of the Verizon email infrastructure is being retired, and that the verizon.net email address domain lives on a just another of the AOL email domains (like cs.com, wmconnect.com, aim.com, wow.com, etc.) There likely won't be any difference in reputation or filtering systems between verizon.net and aol.com mailboxes, when the transition is fully complete. This could be subject to change, so stay tuned.

Update: Laura Atkins of Word to the Wise rightly points out that there's no timeline or deadline published anywhere in this Verizon notice. When exactly will Verizon shut it all down? We shall see.

June 26, 2017 Update: As related to me and others by AOL, the Verizon mailboxes that remain have now been transitioned to AOL's mail servers.

Password Reset Emails: Best Practices

I've been thinking about best practices for password reset emails lately. Instead of trying to re-invent a wheel that other folks have already capably designed, I'll just highlight a couple of thoughts and link to some more detailed info from a couple of folks with have good insight to share.

The most important thing to remember, I think, might be this: Always reset, never remind. Meaning, don't email a password to the user. It could spit out the password to the wrong person, if abused. Also, aren't your passwords one-way encrypted? Don't store it in the clear, don't send it out in the clear.

A close second: Make sure your emails don't look like phishing. Everything should properly authenticate with SPF and DKIM. Your domain should have a DMARC policy in place. Lock that domain down, to make it harder for faked password resets (or other notifications) to get through to the inbox.

And finally, delivery speed really matters -- though busy email systems can often still deliver emails pretty quickly, you will find that delivery delays due to poor reputation will absolutely kill you here. This highlights why you need to keep your nose clean with your marketing emails -- so your reputation is stellar enough that the same communication channel is open and available to you for very quick delivery of very important user notifications, like password reset emails.

AOL, Yahoo, Gmail (and possibly other ISPs) seem to delay delivery of inbound email when a sender's reputation is only so-so. And you can't always try to segregate that mail to work around the issue. You might not have enough transactional mail volume to warrant a dedicated IP address just for notifications. And your domain name is going to be the same across all types of email, assuming you want to stick solidly to your primary brand and its domain.

Microsoft's Troy Hunt has put together an excellent number of suggestions on the topic of resetting your password,  and Postmark's Garrett Dimon dives deeper into the email side of this equation. They're both worth reading.

New Outlook.com/Hotmail IP ranges

Microsoft's Terry Zink announced yesterday on the Mailop list that Microsoft is combining the infrastructure for Outlook.com (Hotmail) and Office 365. As part of this infrastructure update, Microsoft is letting the world know that soon, all outlook.com (hotmail.*, live.*, msn.*, etc.) consumer email traffic will originate from IP addresses in the 40.92.0.0/14 (40.92.0.0 - 40.95.255.255) network range. Folks running spam filters will want to update their systems accordingly.

AOL User Mike Pence

I usually try to avoid getting too political around these parts, but this one I just can't resist.


Turns out, then-Indiana governor Mike Pence was a big fan of using his personal AOL email account for state business.

That can't be right, can it? After all, that party sure made a big fuss about another party's presidential candidate using personal email for business purposes.

Indeed, the Indy Star and USA Today report that "Pence fiercely criticized Clinton throughout the 2016 presidential campaign, accusing her of trying to keep her emails out of public reach and exposing classified information to potential hackers."

Yet, "[While] Indiana Gov. Eric Holcomb's office released more than 30 pages from Pence's AOL account, ... [the office] declined to release an unspecified number of emails because the state considers them confidential and too sensitive to release to the public."

Strange how politicians attack rivals over things they themselves engage in, isn't it?

IBM Patents Out-of-Office Reply functionality

The United States Patent and Trademark Office has granted patent number 9,547,842 to IBM for an "Out-of-office electronic mail messaging system." What? I don't even.

Guest Post: A Guide to Microsoft SNDS

Deliverability Expert Chris Truitt was kind enough to reach out to me after he noticed my prior post lamenting the lack of guidance on what you should do with Microsoft SNDS data. Here are his suggestions for what you should be thinking about when looking at Microsoft SNDS data. Thanks, Chris! -- Al Iverson

Sean Spicer and WHOIS

ARS Technica reports on Trump minister of propaganda (I think that's his title) Sean Spicer and how he registered a handful of domains a few years ago with his still-current contact information, including his still-current phone number.

Howto: Maximize Inbox Delivery to Yahoo

What is the path to deliverability success at Yahoo! Mail? Here are five simple considerations that will help point you in the right direction.

What can SNDS tell you?

Microsoft Hotmail / Outlook.com's SNDS (Sender Network Data Services) portal is a useful interface where you register to be able to look up reputation data specific to your sending IP addresses. In my "How to troubleshoot Microsoft Outlook.com / Hotmail Delivery Issues" article, I suggest signing up for it.

The Scunthorpe Problem

Have you heard of the Scunthorpe Problem? It highlights the side effects associated with trying to filter out profanity.

What is UCENET?

Formerly known as the London Action Plan, UCENET (Unsolicited Communication Enforcement Network) exists to promote international spam enforcement cooperation and address spam related problems, such as online fraud and deception, phishing, and dissemination of viruses. Its members include government and internet industry representatives and public agencies from 27 countries.

You can learn more about UCENET's recent activities here.

(H/T to the Mainsleaze blog.)