Dealing with spam to your abuse desk?

Among other things, I run the abuse desk for a large service provider with lots of clients. We get a handful of complaints a day. For example, over the past three days, we’ve received about sixteen complaints. And about two hundred spams.

The “fun” part of our job (for various values of “fun”) is going through the abuse mailbox and separating the wheat from the chaff every day. More than 90% of that inbound mail stream is spam. Just random, stupid spam emails from people dumb enough to send spam to an abuse desk. We take turns taking out the trash, moving this mail out of the way so that we can focus on the actual, actionable reports that need to be reviewed and investigated.

How can I reduce the amount of spam our abuse desk receives? I’ve used a lot of different blacklists over the years to reduce the amount of spam received. Problem is, most of them have some level of false positives associated with them. I don’t ever want to knowingly reject a complaint from somebody trying to report abuse from one of our users.
Time to do a bit of testing. On February 2nd, I wrote a script that tags all inbound mail sent to our abuse desk. The sending IP is checked against the Spamhaus ZEN combined list. If the sending host is on the ZEN list, our script adds [SPAM] to the subject line. This helps us sort the mail faster. We spend less time looking at the mail with [SPAM] in the subject line, and more time reviewing the mail that isn’t tagged.
Reviewing the over 2,200 spams I’ve received to our abuse desk from February 2nd through today, Spamhaus has successfully tagged 79.3% of them as spam. I’m very happy with that rate – this correct classification significantly reduces the amount of spam I have to deal with in the long term.
But what about false positives? Since I’m tagging mail, and not rejecting it, it’s very easy for me to find and note false positives. (A false positive in this instance would be a spam report that I wanted to receive, but might have missed because it was tagged as spam.) To date, I haven’t had a single false positive! I’ve saved all the mail in question, and reviewed it multiple times, looking for mail that I might want, but could have missed previously. There doesn’t seem to be any. Score another point for Spamhaus!

If you run an abuse desk that gets a lot of spam, how do you deal with it? I’d love to hear your thoughts. And if you’re in the same boat as me, and wondering what to do? It might be worth your while to tag the mail with Spamhaus ZEN. I think you’ll find that it’ll correctly identify most of the spam, and that false positives, if any, will be few and far between.

The Changing Definition of Spam

Over on CIO Magazine's website, capable jounalist Esther Schindler posted an interesting article on the topic of spam defined, and how that definition has been changing over the years. The spark that led to her writing this came from a discussion on an anti-spam mailing list we're both members of, and it was a topical discussion that I myself delved into.

I perhaps don't agree with her conclusions 100%, but I credit her for tackling a tough topic, and stirring up discussion and debate. It is true that the definition of spam is changing. It's also true that there's a hard-core group of anti-spam advocates who are resisting this change. Anti-spam mailing lists are sliding from the center out to the edge of the anti-spam universe; they once were the core and forefront of development and discussion relating the latest anti-spam technology, blocking tools, best practice methodology, etc. Nowadays, that's all shifted away, to discussions internal to ISPs and industry groups, spam filtering device manufacturers, and other areas, far from the view of the folks who used to call for "heads on pikes" as the only reasonable response to a single piece of (perceived) spam received.

To me, it highlights that the world is changing, and the Linux users with their access control lists don't hold the keys to the inbox like they once did.

What it does it mean “We do not relay?”

Have you ever received a bounce message that says “We do not relay” or “Mail loops back to myself”? Wondering what that means? I get this question often enough, that I thought it would be useful to break it down for you here.

Both of these error messages mean that the site you’re trying to send mail to is misconfigured. Neither of these errors typically indicates a problem with you or your ISP.

We do not relay” (or "Relaying denied") means “It looks like you’re trying to relay mail through me, and I don’t allow that.” In this instance, the destination mail server is incorrectly configured. It doesn’t know that it’s supposed to handle mail to the domain you’re contacting, so it’s thinking that it’s being asked to forward mail on for an unknown party. Most mail servers don’t allow this, because “open relaying” was a popular method of sending spam for many years. This doesn’t mean you’re a spammer. It just means that the remote server is configured incorrectly. They are probably (accidentally) blocking all inbound mail due to this issue.

Mail loops back to myself” is a similar issue. In this case, the mail server is smart enough to have looked up the MX (mail exchange) records for the remote domain. It notes that it in fact is the destination server for the mail. But, it hasn’t been configured to receive mail for this domain. So, the server is telling you, “I know that mail to this site is supposed to end here, but I’m not configured for that, so I don’t know what to do with this mail.” This is another example of an error message that probably indicates that all mail to that site is being bounced, because their server is not configured correctly.

If the server can tell that DNS records indicate that it’s supposed to accept mail for a given domain, why doesn’t it just automatically accept the mail? A mail server wouldn’t automatically accept mail for domains pointed at it via DNS, because that would be a risk to the server’s security and stability. Bad guys all around the world could point their MX (mail exchange) record toward a server, and the server could then be overwhelmed by mail its administrators didn’t ask for and don’t want.

Got any other questions about bounce messages and what they mean? Feel free to contact me, and I’ll do what I can to help.

CAN-SPAM Roundup

The US Federal Anti-Spam law (CAN-SPAM) has been in effect for just over three years now, yet I still get questions about it constantly. Here’s a quick roundup of links to resources relating to the CAN-SPAM act:
If you send commercial email, here’s an important thing to remember:

As you probably noticed while doing research online, some people recommend that if the list you are sending to isn’t opt-in, you should label your mail with a clear notice that it’s an advertisement or solicitation. They’re wrong, and here’s what you should be doing instead:

If you don't have clear consent, don’t send it.

“But the law allows it” isn’t a good enough reason. ISPs can and do block 100% legal emails all day long. Label your message as an advertisement and send it to a list that isn't opt-in and you’re asking, practically begging, for an ISP to filter or block your mail.

If it’s not opt-in, if you don’t have affirmative consent, your mail will be blocked as spam, and you’re going to create an email deliverability problem. It’s that simple.

Microsoft using Spamcop and Spamhaus? Yes!

5/22/2007 Update: Some of this information is out of date. Please click here for my latest thoughts on Spamcop and the Spamcop SCBL.

Recently, while doing a bit of research to find other opinions on blacklists, I ran across this seemingly random anti-blacklist blog post. He’s mad at Spamcop, but he thinks there’s some sort of collusion with Microsoft (and he even accused me of working for Microsoft—ha). Then he throws in a bunch of stuff about CAN-SPAM (missing the bits where it says that ISPs are free to block whatever they want in their best efforts to stop spam) and MAPS (which hasn’t blacklisted him and has no connection to the issue; he just wants to highlight that MAPS is free to define spam “outside of accepted standards,” as basically anyone in a society with a right of free press is allowed to do.)

Anyway. In this rant, he points out that Microsoft is using Spamcop. (February 2008 update: Previously I included links and info on Spamcop being controversial and causing significant false positive issues. Since then, I've made my own measurements that suggest Spamcop is fairly conservative and to be trusted.)

Microsoft is indeed using both the Spamcop Blacklist (SCBL) and the Spamhaus SBL-XBL combined feed. I emailed an address referenced in a bounce snippet posted on that other guy’s blog, and got an autoreply back indicating that Microsoft is indeed using both blacklists. Click here to see for yourself. Note that this is for mail sent to microsoft.com users only. This does NOT MEAN that MSN Hotmail is using Spamcop or Spamhaus. In other words, Microsoft the company is using it on their own mail, not as a filter on your Hotmail account.

Blast from the past: Scott Richter on the Daily Show



It's time for a Friday night funny. Remember this video of Scott Richter on the Daily Show, from back in 2004?

Well, check this out. I was doing a bit a Googling today, and I ran across this article on Mediadpost.com, from shortly after that interview, where Scott talks about how it was all a planned put on. "I know how to pronounce the word clitoris," Scott made a point of telling author Bill McCloskey. Ha.

I'm not sure I buy it, but I got a good laugh out of it nonetheless, two years after the fact. How come nobody forwarded me the Mediapost link way back when?