What are spamtraps?

In short, spamtraps are bad addresses that you don’t want on your list. They’re old email addresses that haven’t been used for real people for a long time, or addresses that are put out to ensnare bad guys who are obtaining addresses in ways other than opt-in.

If you have spamtraps on your list, you’re going to be labeled a bad sender and blocked as a spammer. That’s why it’s important that you have good signup process to prevent fraudulent signups, and good bounce processing so that you expire out invalid addresses before an ISP would ever turn them into spamtraps.

For more about spamtraps, check out my recent blog posts on the topic (part one here and part two here), where I talk in detail about what they are, how they end up on your list, how you prevent them from getting there, and how to clean your list and get rid of spamtraps in the process.

Spamtraps are addresses that drive you directly into ISP spam filters and anti-spam blacklists. Consider them the express route to having your email blocked. Stefan Pollard recently wrote an excellent article on the topic of blacklists for ClickZ (find that here). Since many blacklisting issues are spamtrap-driven, there's great overlap between best practices on how to keep your list clean and what you should be doing to prevent and remove spamtraps.

On the Glossary page of anti-spam blacklist group Spamhaus, you'll find their definition of "spamtrap," one that I find generally to be accurate.

Dealing with spam to your abuse desk?

Among other things, I run the abuse desk for a large service provider with lots of clients. We get a handful of complaints a day. For example, over the past three days, we’ve received about sixteen complaints. And about two hundred spams.

The “fun” part of our job (for various values of “fun”) is going through the abuse mailbox and separating the wheat from the chaff every day. More than 90% of that inbound mail stream is spam. Just random, stupid spam emails from people dumb enough to send spam to an abuse desk. We take turns taking out the trash, moving this mail out of the way so that we can focus on the actual, actionable reports that need to be reviewed and investigated.

How can I reduce the amount of spam our abuse desk receives? I’ve used a lot of different blacklists over the years to reduce the amount of spam received. Problem is, most of them have some level of false positives associated with them. I don’t ever want to knowingly reject a complaint from somebody trying to report abuse from one of our users.
Time to do a bit of testing. On February 2nd, I wrote a script that tags all inbound mail sent to our abuse desk. The sending IP is checked against the Spamhaus ZEN combined list. If the sending host is on the ZEN list, our script adds [SPAM] to the subject line. This helps us sort the mail faster. We spend less time looking at the mail with [SPAM] in the subject line, and more time reviewing the mail that isn’t tagged.
Reviewing the over 2,200 spams I’ve received to our abuse desk from February 2nd through today, Spamhaus has successfully tagged 79.3% of them as spam. I’m very happy with that rate – this correct classification significantly reduces the amount of spam I have to deal with in the long term.
But what about false positives? Since I’m tagging mail, and not rejecting it, it’s very easy for me to find and note false positives. (A false positive in this instance would be a spam report that I wanted to receive, but might have missed because it was tagged as spam.) To date, I haven’t had a single false positive! I’ve saved all the mail in question, and reviewed it multiple times, looking for mail that I might want, but could have missed previously. There doesn’t seem to be any. Score another point for Spamhaus!

If you run an abuse desk that gets a lot of spam, how do you deal with it? I’d love to hear your thoughts. And if you’re in the same boat as me, and wondering what to do? It might be worth your while to tag the mail with Spamhaus ZEN. I think you’ll find that it’ll correctly identify most of the spam, and that false positives, if any, will be few and far between.

The Changing Definition of Spam

Over on CIO Magazine's website, capable jounalist Esther Schindler posted an interesting article on the topic of spam defined, and how that definition has been changing over the years. The spark that led to her writing this came from a discussion on an anti-spam mailing list we're both members of, and it was a topical discussion that I myself delved into.

I perhaps don't agree with her conclusions 100%, but I credit her for tackling a tough topic, and stirring up discussion and debate. It is true that the definition of spam is changing. It's also true that there's a hard-core group of anti-spam advocates who are resisting this change. Anti-spam mailing lists are sliding from the center out to the edge of the anti-spam universe; they once were the core and forefront of development and discussion relating the latest anti-spam technology, blocking tools, best practice methodology, etc. Nowadays, that's all shifted away, to discussions internal to ISPs and industry groups, spam filtering device manufacturers, and other areas, far from the view of the folks who used to call for "heads on pikes" as the only reasonable response to a single piece of (perceived) spam received.

To me, it highlights that the world is changing, and the Linux users with their access control lists don't hold the keys to the inbox like they once did.

What it does it mean “We do not relay?”

Have you ever received a bounce message that says “We do not relay” or “Mail loops back to myself”? Wondering what that means? I get this question often enough, that I thought it would be useful to break it down for you here.

Both of these error messages mean that the site you’re trying to send mail to is misconfigured. Neither of these errors typically indicates a problem with you or your ISP.

We do not relay” (or "Relaying denied") means “It looks like you’re trying to relay mail through me, and I don’t allow that.” In this instance, the destination mail server is incorrectly configured. It doesn’t know that it’s supposed to handle mail to the domain you’re contacting, so it’s thinking that it’s being asked to forward mail on for an unknown party. Most mail servers don’t allow this, because “open relaying” was a popular method of sending spam for many years. This doesn’t mean you’re a spammer. It just means that the remote server is configured incorrectly. They are probably (accidentally) blocking all inbound mail due to this issue.

Mail loops back to myself” is a similar issue. In this case, the mail server is smart enough to have looked up the MX (mail exchange) records for the remote domain. It notes that it in fact is the destination server for the mail. But, it hasn’t been configured to receive mail for this domain. So, the server is telling you, “I know that mail to this site is supposed to end here, but I’m not configured for that, so I don’t know what to do with this mail.” This is another example of an error message that probably indicates that all mail to that site is being bounced, because their server is not configured correctly.

If the server can tell that DNS records indicate that it’s supposed to accept mail for a given domain, why doesn’t it just automatically accept the mail? A mail server wouldn’t automatically accept mail for domains pointed at it via DNS, because that would be a risk to the server’s security and stability. Bad guys all around the world could point their MX (mail exchange) record toward a server, and the server could then be overwhelmed by mail its administrators didn’t ask for and don’t want.

Got any other questions about bounce messages and what they mean? Feel free to contact me, and I’ll do what I can to help.

CAN-SPAM Roundup

The US Federal Anti-Spam law (CAN-SPAM) has been in effect for just over three years now, yet I still get questions about it constantly. Here’s a quick roundup of links to resources relating to the CAN-SPAM act:
If you send commercial email, here’s an important thing to remember:

As you probably noticed while doing research online, some people recommend that if the list you are sending to isn’t opt-in, you should label your mail with a clear notice that it’s an advertisement or solicitation. They’re wrong, and here’s what you should be doing instead:

If you don't have clear consent, don’t send it.

“But the law allows it” isn’t a good enough reason. ISPs can and do block 100% legal emails all day long. Label your message as an advertisement and send it to a list that isn't opt-in and you’re asking, practically begging, for an ISP to filter or block your mail.

If it’s not opt-in, if you don’t have affirmative consent, your mail will be blocked as spam, and you’re going to create an email deliverability problem. It’s that simple.

Microsoft using Spamcop and Spamhaus? Yes!

5/22/2007 Update: Some of this information is out of date. Please click here for my latest thoughts on Spamcop and the Spamcop SCBL.

Recently, while doing a bit of research to find other opinions on blacklists, I ran across this seemingly random anti-blacklist blog post. He’s mad at Spamcop, but he thinks there’s some sort of collusion with Microsoft (and he even accused me of working for Microsoft—ha). Then he throws in a bunch of stuff about CAN-SPAM (missing the bits where it says that ISPs are free to block whatever they want in their best efforts to stop spam) and MAPS (which hasn’t blacklisted him and has no connection to the issue; he just wants to highlight that MAPS is free to define spam “outside of accepted standards,” as basically anyone in a society with a right of free press is allowed to do.)

Anyway. In this rant, he points out that Microsoft is using Spamcop. (February 2008 update: Previously I included links and info on Spamcop being controversial and causing significant false positive issues. Since then, I've made my own measurements that suggest Spamcop is fairly conservative and to be trusted.)

Microsoft is indeed using both the Spamcop Blacklist (SCBL) and the Spamhaus SBL-XBL combined feed. I emailed an address referenced in a bounce snippet posted on that other guy’s blog, and got an autoreply back indicating that Microsoft is indeed using both blacklists. Click here to see for yourself. Note that this is for mail sent to microsoft.com users only. This does NOT MEAN that MSN Hotmail is using Spamcop or Spamhaus. In other words, Microsoft the company is using it on their own mail, not as a filter on your Hotmail account.

Blast from the past: Scott Richter on the Daily Show

It's time for a Friday night funny. Remember this video of Scott Richter on the Daily Show, from back in 2004?

Well, check this out. I was doing a bit a Googling today, and I ran across this article on Mediadpost.com, from shortly after that interview, where Scott talks about how it was all a planned put on. "I know how to pronounce the word clitoris," Scott made a point of telling author Bill McCloskey. Ha.

I'm not sure I buy it, but I got a good laugh out of it nonetheless, two years after the fact. How come nobody forwarded me the Mediapost link way back when?

The AOL email tax: Is the sky falling?

Remember the famous internet rumor that made the rounds in 2006: “AOL is trying to tax your email, write in now!

Was it a rumor? Let’s take a look how things have shaped up, since that hit the internet.

One year ago, Cindy Cohn of the Electronic Frontier Foundation “blogged a piece” about this. How AOL and Yahoo were going to use the Goodmail system to allow senders to choose to certify some inbound email. How it was going to create a “pay to speak” environment and this is the first step toward killing the “basically free” landscape of email. A site called DearAOL was created, decrying Goodmail as a threat to a free and open internet. And if you believe the opinions of those behind the site, like MoveOn and the EFF, and others, they mean that -- literally!

One year later, let's take a look at the facts:
  • The Dear AOL site seems to have vanished (as of February 7, 2007). Poof goes the astroturf.
  • You can still deliver mail to AOL just fine, “basically free,” as EFF put it, without signing up for Goodmail.
  • Those that choose to utilize Goodmail and find themselves successful with it can see significant increases in clicks, for example. (Meaning, if it matters to you, and you want to sign, users probably trust your mail and read it first.)
Amazingly, EFF implied then that Goodmail means you’re going to get more spam. The implication is that bad guys will be able to pay, and AOL and Yahoo will want to give you this new, bad mail “served alongside your ordinary spam.”

Fast forward to a year later -- today. Again, the facts:
  • Goodmail has stringent guidelines you have to comply with to enter the program, and to stay in the program. Generating too many spam complaints? You don't get accepted into the program. Generate too many complaints while in the program? You get kicked out. (These are typical "are you spamming" measures used by ISPs, and not new.)
  • AOL is all about user experience. The whole point of AOL's massive spam blocking effort is to get you to be happier with them than you'd be with Gmail or Lycos or somebody else. They were the first large ISP to offer a "report spam" button, which they use as the guiding data behind every spam filtering decision they make. That's called listening directly to their users. Do you really think they're about to screw that up by serving you up some sort of weird paid spam after doing all that?
  • The DearAOL site talked about how the signatories there "have always been happy working together with you to fight spam and phishing." Well, so is AOL. AOL was then, and AOL is today. For years (since at least 2003), AOL’s had this useful site called “Postmaster.Info,” specifically geared toward helping senders address issues sending mail to AOL users.
I’m not here to sell you on Goodmail. I have no vested interest regarding whether or not you use it or don’t use it. I think it might work well for some people, and less well for others. That’s not the point.

The point is this: Go over to your window. Look out. Look up. See the sky? It’s still there. It hasn't fallen, regardless of what the EFF wanted you to think.

I guess it was just a rumor after all.

Backscatter: What is it? How do I stop it?

What is it?
Backscatter is a certain kind of mail you receive that you didn't ask to receive. If you've ever received a “Your mail could not be delivered” bounce notification, a “Your mail contained a virus” notice, or a request to confirm your signup request for a mailing list you've never heard of, you've probably received backscatter. The backscatter problem is inherently linked to the spam problem, as most backscatter received is due to somebody else on the internet doing something bad and spam-related.

Types of backscatter:
  • Misdirected bounces from spam runs, from mail servers who “accept then bounce” instead of rejecting mail during the SMTP transaction.
  • Misdirected virus/worm “OMG your mail was infected!” email notifications from virus scanners.
  • Misdirected “please confirm your subscription” requests from mailing lists that allow email-based signup requests.
  • Out of office or vacation autoreplies and autoresponders.
  • Challenge requests from “Challenge/Response” anti-spam software. Maybe C/R software works great for you, but it generates significant backscatter to people you don't know.
How bad is this problem?
Let's do some quick math on the back of a napkin. A quick check of my personal spamtrap account finds 2,039 pieces of backscatter, just by searching for a few common phrases found in bounce messages and challenge/response requests. Out of the 320,000 recent pieces of spam I've got in that account, that may not seem like much. But that's just me and my tiny domains. Think about a big site, like say, Outblaze, with millions of users across hundreds of domains.

Spam lists contain a high percentage of invalid addresses, driving a high bounce rate. A normal mailing to a legitimate list will result in 3-10% of the mail bouncing, per mailing. And that's if you handle bounces properly. When working with senders, I know that if they haven't tracked bounces properly in the past, a list can have anywhere from 30-50% bounce rate. Spammers rarely handle bounces properly, so let's assume a 40% bounce rate on a spam run. If a spammer sends two million pieces of spam, that leaves 800,000 bounces. Where do those bounces go? I know from helping clients legitimately handle bounces, maybe 7-10% of email servers accept the mail then bounce it back later.

Do the math based on all of these assumptions: 2,000,000 spams, 40% bounce rate, 9% of mail servers send backscatter... That means that for that two million spam run, 72,000 bounce notifications (NDRs) are going to be sent back to the sender address. Since spammers forge the sender's address, this mail is going to be be received by people who had nothing to do with the spam. This, in a nutshell, is backscatter. And there's a lot of it floating around.

How can you stop backscatter?

If you're an end user, there's probably not a ton you can do to prevent the receipt of backscatter. Some anti-spam blacklists (such as backscatterer.org) actively block servers that generate backscatter. Spamcop used to do so in the past, but I don't believe they actively target these kind of servers at the moment. Though, any spamtrap-based DNSBL is going to (intentionally or not) catch servers sending backscatter, because the backscatter will hit their spamtrap addresses just like it's hitting you and everybody else.

Whatever you do, don't use a “Challenge/Response” anti-spam application or service. It makes the problem worse for everybody else on the internet – your challenge requests are just another kind of backscatter.

The same goes for those anti-spam applications that promise to send fake bounces on your behalf. The reasoning is that the spammers will realize your address is dead, and stop sending you mail. That's simply not true. You're going to bounce off the lists of good senders, who actually process bounces. The bad guys (whom are responsible for far more of the mail you receive) don't process bounces. They're the ones creating this whole backscatter problem to begin with. Yours will be just another bogus bounce notification bothering some innocent, unrelated third-party.

Don't set an “out of office” reply, either. Besides contributing to the backscatter problem yourself, you're sending random notes out to the world telling whoever receives the notice that you have a live email address and you'll be back soon to receive it. Guess what? You're confirming for spammers that your email address is valid! That means you're going to get even more spam. I see over 800 out of office replies in my spamtrap mailbox. If I were a sleazy spammer, it would be very easy to write a script to save those addresses in a file, and wait and see how many more I get today, tomorrow, next week, etc.

It might be useful to report backscatter as spam via your favorite spam-reporting service. It helps to collect stats on the problem, and helps to nudge sites to fix their backscatter-generating problems.

If you administer a mail server, here's what you can do to minimize your contribution to the backscatter problem:
  • Don't allow email-based signup requests for your email list. I've got more than 330 “please confirm your subscription” requests in my spamtrap account. If you do this, you're going to end up blacklisted, because these misdirected requests are going to hit spamtraps run by anti-spam groups, too. Instead, only allow web-based signups. People go to your website, fill out a web form to sign up, then send them the confirmation request.
  • Don't use Challenge/Response, and don't allow your users to, either. As I mention above, it fixes your spam problem at the cost of everybody else's spam problem. Few, if any implementations of C/R are smart enough to prevent responses to spam messages and other random junk. Justin Mason puts it succinctly: C/R is “broken and abusive.” Over on his blog, he rounds up a bunch of opinions on why this technology is flawed. I've also made my opinions known on the subject, as well.
  • Don't run autoresponders, out-of-office notifications, etc. You'll annoy people and your mail could end up being blocked. I've seen people complaining that anti-spam blacklists have listed them because of this kind of backscatter. This generally impacts your mail far beyond out of office responses. Spamcop has a FAQ with more information.
  • Don't bounce mail after accepting it from a remote site. If you run an unpatched version of Qmail, or some other MTA that accepts all mail, then later sends bounces for some of that mail, then you are at the heart of the problem. Those who came before you learned a long time ago that this behavior contributes to the internet's spam problem. That includes AOL (the site with possibly the largest percentage of end-user mailboxes in the US), who has spent tons of time and money over the years, ensuring that as much as their rejected mail as possible is rejected during the SMTP transaction, and not later. Before they made this change, they were a significant source of backscatter. They may not be perfect at this today, but they've made huge strides, and are to be commended.
    Jeremy Harris was kind enough to pass along this support note from Microsoft on how to configure Exchange 2003 so that it does not work in "accept, then reject" mode.
    You might ask yourself, "What's the point? If I reject the mail, somebody will be sending a bounce." That's usually not true -- most spam is sent by infected zombie computers, and the software spammers are running doesn't bother with things like bounces.
  • Configure your virus scanner to silently strip or discard viruses/worms instead of sending a notification back to the sender. It's a fact: Viruses and worms sent via email do not have valid sender addresses. If you send a notification back, it goes to the wrong person, an innocent third-party. This is backscatter, and your server just succeeded in annoying somebody.
  • If you run an ISP, rate limit outbound mail or detect usage of large numbers of different envelope senders. This helps to combat “outscatter,” which is what happens when users on your network have virus-infected desktops utilizing your SMTP servers to send outbound spam. Since they're on your network, they'd normally be allowed to relay mail through your server. But, if left unchecked, you'll find your outbound mail servers blacklisted for sending spam, as the place it's being transmitted from, as far as the rest of the internet is concerned, is your outbound mail server.
  • Read and learn. Anti-spam groups like Spamhaus have posted and linked to useful information about the backscatter problem and what to do about it. Justin Mason's blog covers this topic, as well. The Spam Links site has an entire section devoted to backscatter. It's worth reading.
I hope you found this information useful. If you see anything out of date, or think there's more information I should consider adding, don't hesitate to contact me with your thoughts and feedback.

Joe Jared Wins

Joe Jared used to run an open-relay blacklist called Osirusoft. I respected what Joe’s intent was, though I was of the opinion that Joe ran things unwisely. Both of these things I mentioned way back in 2001, when an SF Gate Article slammed Joe (and blacklisting in general).

I recall that Joe has been defensive and combative in the past when people with affected/blocked mail reach out to him. He was also not very good at record keeping, something I knew from the past, and that fact came up in this case (Pallorium vs Jared).

Pallorium, a private detective agency, sued Joe, saying more or less that they weren’t running an open relaying mail server, they weren’t spammers, that Joe was careless, and that he offered no recourse to resolve the listing, because his website was periodically unavailable.

The court documents indicate that Joe admits that he had no idea how the server came to be listed, and that Joe’s tests after the lawsuit saga began show that the server was not actually an open relay. If records were kept showing where and why something was listed, I suspect it would’ve saved him some effort and perhaps prevented the lawsuit. It probably didn’t help that Joe was rude to the plaintiff on the phone about the issue, too.

Regardless, this case was clearly resolved in favor of spam blocking. I've always said that ISPs and anti-spam groups are pretty much free to block whatever mail they want, in their best efforts to block spam. The court and I seem to be in agreement on that point.

On one of the public anti-spam lists I’m, this spurred a discussion positing that this allows folks to probe for open relays and add them to a blacklist, as long as the person doing so has been receiving objectionable spam. Perhaps, but this isn’t quite precedent-setting. My understanding is that this would only apply in California courts, and that the decision is unpublished anyway. Not to mention, open relays (which I used to help people block myself back when I ran blacklists) are no longer the primary source of spam that they once were.

(If you don’t know what an open relay is, check my recent post on the topic.)