Pre-order Spam Nation by Brian Krebs

From Brian Krebs: "The backdrop of the story is a long-running turf war between two of the largest sponsors of spam. A true-crime tale of political corruption and ill-fated alliances, tragedy, murder and betrayal, this book explains how the conditions that gave rise to this pernicious industry still remain and are grooming a new class of cybercriminals.

But Spam Nation isn’t just about junk email; most of the entrepreneurs building and managing large-scale spam operations are involved in virtually every aspect of cybercrime for which there is a classification, including malware development, denial-of-service attacks, identity theft, credit card fraud, money laundering, commercial data breaches and extortion."

I'll be ordering my copy soon!

Does Gmail use Spamhaus blacklists?

Probably, implies Return Path based on a correlation between a typical Spamhaus blacklisting and drop in inbox delivery rates at Gmail. I think it's safe to assume that Google does use Spamhaus data for some sort of reputation calculation impacting Gmail deliverability.

Ask Al: Should I add a DMARC record to fix the Yahoo issue?

A friendly representative of a company who helps small businesses sell products asked: "We're having problems forwarding mail from our customers back to our users due to the new Yahoo and AOL restrictive DMARC policy. If we add a DMARC record for our own domain name, would that help address the Yahoo/AOL bouncing issue? Would that explain to the ISPs that we're not spoofing when we forward on that mail?"

No, this wouldn't fix your issue. It's probably not a bad idea for you to implement a DMARC record for your domain, especially if the domain is one you use for email marketing or online retail and want to make it harder for bad guys to spoof it. (But be sure you learn more about DMARC before proceeding; I would recommend partnering with somebody like Return Path or Agari to use their tools and benefit from their expertise with regard to anti-phishing/spoofing and DMARC.)

The reason this wouldn't fix your issue is because the Yahoo and AOL DMARC policies affect only mail that has a Yahoo or AOL domain in the from address. Also, they have the potential to affect all/any mail with a Yahoo or AOL domain in the from address. What other domain you might have in the message or message headers has no bearing on that fact. Whatever DMARC policy setting you publish wouldn't override whatever policy setting the owner of those domains may have published. In other words, if it's AOL.com in the from address, it's always going to be the AOL.com policy that applies, no matter what.

The real fix for the issue is to figure out how to get it so only your own domain name shows up in the from address. That might necessitate a change in your message flow process. It might make you have to reconsider whether or not you forward on messages through your system at all. Or you might have to rewrite headers, if you still want to be able to forward on that mail.

Need to contact Live.com/Hotmail?

On a mailing list I subscribe to, someone recently asked for assistance with getting mail delivered to Microsoft's Live.com and Hotmail.com domains. Apparently the poster works for an internet provider that had a compromised account or two, and Microsoft was blocking their mail as a result.

A kind soul posted this reminder in response:

"The most efficient and effective way to address any deliverability problem is by submitting the issue to our dedicated deliverability support team. Senders can do this by filling out a form with detailed information necessary to diagnose the problem. The form can be found here: https://support.msn.com/eform.aspx?productKey=edfsmsbl&ct=eformts&st=1&wfxredirect=1. Going directly to deliverability support will ensure that we have the right information to investigate the cause and recommend the right solutions quickly."

Those of us who have been around a while already have that URL bookmarked, but I thought I would share it here for new folks who might not already know it.

Check out this neat thing: Email Privacy Tester

Smart web developer and systems admin Mike Cardwell put together something neat that I think you'll want to check out: Email Privacy Tester. Plug in your email address, and his system will send you an email message. That message will contain a whole bunch of different types of active content in various different wrappers and encodings, to see which of them are triggered by your MUA (mail user agent; aka email client). I did a quick test to a Gmail account and I already see one thing that I don't like, something Gmail executes automatically without asking me.

DMARC does only one thing (but pretty well)

DMARC (specifically, DMARC with a p=reject policy) does one thing: It makes it very difficult for somebody to use your domain in a from address without your permission. It doesn't stop spam from open relays, open proxies, snowshoeing network blocks, nor does it prevent people from ignoring CAN-SPAM and continuing to mail you after you have unsubscribed.

It doesn't do all those other things; but the one thing it does, it seems to do it very well.

It has limitations; yes. It doesn't specifically do anything the cousin domain problem (deceptively-registered domains with similar, but not exactly the same names). It makes mailing list management more complex. And it makes it so for the most part, users of email service providers (ESPs) cannot use Yahoo or AOL addresses as from addresses (because Yahoo and AOL publish strict DMARC policies).

Is anybody saying that DMARC is a magic bullet? I don't think so, but a better question is -- Does DMARC help? Yes, says Yahoo.

But spammers eventually moved on to forging one of Yahoo's domains, so it was a pointless exercise, others have said. Pointless exercise? I don't agree with that at all. It's true that Yahoo only published a strict policy for one of their domains; it's almost like they decided to lock only one door first, either because it was the most abused, or perhaps because you have to start somewhere (and maybe want to see how that goes before adding others).

But if I were a Yahoo or AOL, or a financial institution, or well known retail brand, would I think it is a win; a gain, quite likely a long term one, if I could keep bad guys away from using my domain in the from address of their bad mail? Yes, I would. DMARC helps. And I think when you look at who is implementing DMARC, I suspect that I'm not the only one who feels that way.

This reminds me so much of the days when I first got involved in spam fighting. Open relaying mail servers were a big bad problem. Spammers exploited them regularly and with great vigor. I was so tired of dealing with spam from open relaying mail servers that I started to block it, an effort that ended up becoming the MAPS Relay Spam Stopper, a coordinated third party blacklist that easily allowed mail system administrators to block mail from open relaying mail servers.

Spammers cried about being blocked. But not just spammers; there were also a non-zero number of people who claimed that I was part of some big conspiracy to wreck email. People called my employer, trying to get me fired. A number of people threatened to sue. A few high-profile savvy internet users like John Gilmore, explained that they needed to be able to legitimately relay mail for other people, and that blocking mail from those servers was akin to censorship.

Somehow, we survived all of that. Eventually, people seemed to accept this change. We got to a point where, while you do still see open relays out there from to time, there's a general consensus that it's a bad way to configure a mail server. But for us to get there, a number of us had to first band together and take action; personally implementing policies wherein we block mail from misconfigured servers used to shovel abusive, unwanted mail, while people who didn't understand why it mattered gave us hell for at the time.

I find a lot of parallel there; my point, other than I'm old and 1998 was a long time ago, is that I have a feeling that this will be a non-issue in a few years, and even if sites don't broadly implement DMARC with a p=reject policy, enough of them will do so that you're truly going to have to just deal with it.

OpenDKIM & SpamAssassin Gotchas on Ubuntu 12.04

Allow me to share with you my rough notes compiled during my recent configuration of email and DKIM on a new Ubuntu VPS installation. Hopefully these helpful hints will help the next poor soul trying to get DKIM up and running on the first try:
  • SpamAssassin seems to want to fail DKIM keys from Gmail or Google Apps with an error of T_DKIM_INVALID. There's some noise online about this perhaps being an issue with clock synchronization, but it is more likely that you don't have one or more perl modules installed necessary for SpamAssassin to properly decode the DKIM key. The fix turned out to be simple; installing these packages solved that issue:
    sudo apt-get install libmail-dkim-perl
    sudo apt-get install libcrypt-openssl-random-perl
    sudo apt-get install libcrypt-openssl-rsa-perl
    (Thank you, Henrik Schack, for the tip!)
  • If you are new to OpenDKIM's Authentication Results header, you're going to be confused by this. You'll see a lot of DKIM as having passed, but with a reason of "1024-bit key; insecure key." This made me start poking around, looking at file permissions for my various keys (I set up signing for multiple domains). I assumed I had done something wrong, but I couldn't find any issue no matter where I looked. It turns out that it is not really an issue at all. What the error message actually means is that the domain that send you the message isn't using DNSSEC. Long term? Sure, yeah, everybody should look at DNSSEC, but one thing at a time.
  • Here's what nobody tells you if you're DKIM signing multiple domains on the same server, using OpenDKIM: It is possible to interpret the opendkim.conf configuration file in a way that would lead you to add multiple sections starting with "domain," setting a selector for each and linking to different keys. Truth be told, OpenDKIM will only honor the LAST one of these sections, signing mail for only one of your domains. It won't generate any sort of error message, either, so it can be frustrating to understand what is happening. Remember, if you want to set up signing for multiple domains, look at how to configure the SigningTable and KeyTable settings in OpenDKIM.
Even with the challenges I've run into here and there as I set up my new server, I'm amazed at how easy overall it was to get everything working. Last time I tried to enabled DKIM locally was a few years ago, and I got stuck in some arcane technical issue and gave up. Either the process got a lot easier and more stable, or I am smarter today than I was then. For my ego's sake, I'll pretend it's the latter.

(Originally posted to my personal blog, but I think it makes more sense here since it's talking about email, spam filtering and authentication.)

106 miles to Chicago


If you've ever set up catch-all domain names, that accept mail destined to any address, or set up your MTA in a way where all mail is accepted before checking to see if the recipient is valid, then you've probably seen emails containing this quote: "It’s 106 miles to Chicago, we got a full tank of gas, half a pack of cigarettes, it’s dark… and we’re wearing sunglasses."

The movie isn't new to me, but this is the first time I've run across the quote in this context. Apparently, for a while now, some spammer has been trying to find open relaying mail servers and seems to use this quote in almost every open relay attempt. I guess they just got around to trying it some place where I would notice the attempts.

Blast from the past: Challenge/Response Filtering?

A few weeks ago, somebody on the Mailop list asked about an email they received. Turns out, it was a Challenge/Response spam filter. You know, one of those emails from a robot that says "click here to prove you're not a robot, otherwise I'm not going to deliver your email on to the intended recipient." That doesn't work very well -- do you think Amazon has people waiting to click on that link each time they send out an emailed order receipt? Yeah, yeah, supposedly you (the user with the C/R spam filter) can whitelist that stuff, but you're going to miss something.

Challenge/Response filtering seems to have mostly disappeared; this was the first time I had heard about it in many years. But it's not the first time I've blogged about it; look all the way back to this MediaPost article from 2006, and you can see me raise still-extant concerns relating to how C/R works.

Click here to jump in the time machine and read what else I've written about C/R in the past.

Signing outbound list mail with OpenDKIM

For my server running the discussion mailing lists that I host, I'm running OpenDKIM, OpenDMARC and Postfix on Ubuntu. I run a custom mailing list manager that I wrote myself; I keep telling myself that eventually I'll polish the code enough to share it with the world.

I wanted all posts to the mailing list to be signed with the DKIM signature of my mailing list domain name. Any DKIM signature the message was initially signed with when submitted to the list is rendered invalid by the header changes my mailing list software makes before distributing the posted message to all subscribers. Thus, mail would have been distributed to subscribers either with no signature or with a now failing signature. I feel strongly enough about email authentication to be dissatisfied by that; I want all mailing list messages to clearly demonstrate to the world that my server, my domain is responsible for the list mail.

To do this, I would in theory want to add OpenDKIM's "SenderMacro" directive to opendkim.conf. Except, support for SenderMacro is not compiled into OpenDKIM by default. No big deal, right? My brain shouldn't have atrophied too much from when I had a day job as a unix system administrator; I can just recompile it. But, it turns out, compiling OpenDKIM from source on Ubuntu is a hugely frustrating pain in the ass, with a ton of dependencies that I had to solve for manually, and package names that correspond to the absent library names in almost no way whatsoever. And even after I got it all compiled, there still ended up being a path issue with locating libraries at runtime that I just was not able to figure out. In theory I know how to fix that, but in practice, it just didn't work no matter what I did.

But then I stumbled across a much simpler way to do this: The "SenderHeaders" directive. The OpenDKIM README has a helpful section on mailing lists, where it explains that you can add this directive and set it to "Sender, From" to tell OpenDKIM, when deciding whether or not to sign the mail, look first at the domain in the sender header, then if that doesn't match the list of domains you sign, look at the From address domain to find a match.

My mailing list manager software doesn't use a sender header. I could easily add one, but I don't really like how that displays in the user interface of certain MUAs. Thankfully, the OpenDKIM documentation suggests an alternative header: "List-Post." Most mailing list manager software, including mine, includes a header designating what address you send to if you want to post a message to the mailing list. (The header looks like this: "List-Post: <mailto:LISTNAME@example.com>.")

I decided to give that a try and updated the SenderHeaders directive to read "SenderHeaders List-Post,Sender,From" and it worked! OpenDKIM looks at the List-Post header, sees a domain there that I can sign mail for, and it signs the mail with the key for that domain. The net result, all messages to the mailing list a signed with my DKIM key automatically.

Yahoo on Yahoo's new DMARC Policy

What impact has Yahoo's new "p=reject" DMARC policy had on spam claiming to be from Yahoo.com? "DMARC has reduced spam purported to come from yahoo.com accounts by over 90%," says Alex Stamos, Yahoo's Vice President of Information Security, in his recent testimony before Congress.

H/T: Return Path's Ken Takahashi.

(Note: Alex's title may be Chief Information Security Officer; I've seen him referred to both ways. I decided to use the title referenced in the testimony.)

The Current State of TLS over SMTP?

Michael Adkins, Mail Integrity Engineer at Facebook summarizes it thusly:

"STARTTLS encryption is widely supported and has achieved critical mass despite some issues with certificate management. A system deploying STARTTLS support for the first time can expect more than half of its outbound email to be encrypted. Also, the majority of deployments provide Perfect Forward Secrecy. We see two high priority areas for improvement. First, we encourage the industry to work together to develop better tools for preventing mismatched certificates. Second, we encourage everyone to deploy support for opportunistic encryption via STARTTLS."

TL;DR? Turn on opportunistic TLS, and if your results are like Facebook's results, at least half of the time your mail will be encrypted in transit. This is a very good thing, and adoption is only going to grow, especially when you've got a big site like Facebook who sends a lot of mail, helping to gently nudge folks in the right direction.

Want to learn more? Steve Atkins explains how to Protect your email with TLS over at Word to the Wise.

Yahoo Groups rewriting from addresses to handle DMARC policy

As reported on the Mailop list today, Yahoo has modified their Yahoo Groups discussion group service to "play nice" with posts from subscribers at domains behind a restrictive DMARC policy. Google recently implemented very similar changes to its Google Groups service. Both are believed to have done so in response to Yahoo and AOL having recently both implemented strict "p=reject" DMARC policies, affecting Yahoo users' and AOL users' ability to participate in some mailing list discussion forums.


Yahoo is modifying the original from address header to make the sending email address the Yahoo Group address and moving the original from address header to an X-Original-From header.

Yahoo Groups, Google Groups, mailman, and others services and software packages have been updating their handling of from addresses recently, in response to the challenges posed by restrictive DMARC policies that prevent off-network, unauthenticated uses of certain domain names in a from address. I've updated my own mailing list manager software similarly, and I'm sure you'll be seeing more of the same from other mailing list services and software packages in the near future.

Operators of discussion mailing lists and similar services pretty much have two possible choices for how to respond to this type of issue: Modify their software to write the from address to cooperate with this policy change, or lock Yahoo and AOL users out of participating in these various services and discussion groups; this course of action that doesn't seem widely accepted or adopted. Most list managers want their subscribers to be able to continue to participate, and most commercial service providers seem to not want to deny participation to a large swath of internet users. And certainly, one concern I've hear is, what happens when the next large ISP or webmail provider implements a similar policy?

May 15, 2014 update: Yahoo explains their changes to Yahoo Groups here.

(May 13, 2014 author note: I had an unfinished, half-written blog post in my draft queue trying to answer the question "What of Yahoo Groups?" I was basically going to say that it wouldn't surprise me to see Yahoo Groups update their service similar to how Google has done. Guess there's now no need to finish that one up.)