Microsoft using Spamcop and Spamhaus? Yes!

5/22/2007 Update: Some of this information is out of date. Please click here for my latest thoughts on Spamcop and the Spamcop SCBL.

Recently, while doing a bit of research to find other opinions on blacklists, I ran across this seemingly random anti-blacklist blog post. He’s mad at Spamcop, but he thinks there’s some sort of collusion with Microsoft (and he even accused me of working for Microsoft—ha). Then he throws in a bunch of stuff about CAN-SPAM (missing the bits where it says that ISPs are free to block whatever they want in their best efforts to stop spam) and MAPS (which hasn’t blacklisted him and has no connection to the issue; he just wants to highlight that MAPS is free to define spam “outside of accepted standards,” as basically anyone in a society with a right of free press is allowed to do.)

Anyway. In this rant, he points out that Microsoft is using Spamcop. (February 2008 update: Previously I included links and info on Spamcop being controversial and causing significant false positive issues. Since then, I've made my own measurements that suggest Spamcop is fairly conservative and to be trusted.)

Microsoft is indeed using both the Spamcop Blacklist (SCBL) and the Spamhaus SBL-XBL combined feed. I emailed an address referenced in a bounce snippet posted on that other guy’s blog, and got an autoreply back indicating that Microsoft is indeed using both blacklists. Click here to see for yourself. Note that this is for mail sent to microsoft.com users only. This does NOT MEAN that MSN Hotmail is using Spamcop or Spamhaus. In other words, Microsoft the company is using it on their own mail, not as a filter on your Hotmail account.

Blast from the past: Scott Richter on the Daily Show



It's time for a Friday night funny. Remember this video of Scott Richter on the Daily Show, from back in 2004?

Well, check this out. I was doing a bit a Googling today, and I ran across this article on Mediadpost.com, from shortly after that interview, where Scott talks about how it was all a planned put on. "I know how to pronounce the word clitoris," Scott made a point of telling author Bill McCloskey. Ha.

I'm not sure I buy it, but I got a good laugh out of it nonetheless, two years after the fact. How come nobody forwarded me the Mediapost link way back when?

The AOL email tax: Is the sky falling?

Remember the famous internet rumor that made the rounds in 2006: “AOL is trying to tax your email, write in now!

Was it a rumor? Let’s take a look how things have shaped up, since that hit the internet.

One year ago, Cindy Cohn of the Electronic Frontier Foundation “blogged a piece” about this. How AOL and Yahoo were going to use the Goodmail system to allow senders to choose to certify some inbound email. How it was going to create a “pay to speak” environment and this is the first step toward killing the “basically free” landscape of email. A site called DearAOL was created, decrying Goodmail as a threat to a free and open internet. And if you believe the opinions of those behind the site, like MoveOn and the EFF, and others, they mean that -- literally!

One year later, let's take a look at the facts:
  • The Dear AOL site seems to have vanished (as of February 7, 2007). Poof goes the astroturf.
  • You can still deliver mail to AOL just fine, “basically free,” as EFF put it, without signing up for Goodmail.
  • Those that choose to utilize Goodmail and find themselves successful with it can see significant increases in clicks, for example. (Meaning, if it matters to you, and you want to sign, users probably trust your mail and read it first.)
Amazingly, EFF implied then that Goodmail means you’re going to get more spam. The implication is that bad guys will be able to pay, and AOL and Yahoo will want to give you this new, bad mail “served alongside your ordinary spam.”

Fast forward to a year later -- today. Again, the facts:
  • Goodmail has stringent guidelines you have to comply with to enter the program, and to stay in the program. Generating too many spam complaints? You don't get accepted into the program. Generate too many complaints while in the program? You get kicked out. (These are typical "are you spamming" measures used by ISPs, and not new.)
  • AOL is all about user experience. The whole point of AOL's massive spam blocking effort is to get you to be happier with them than you'd be with Gmail or Lycos or somebody else. They were the first large ISP to offer a "report spam" button, which they use as the guiding data behind every spam filtering decision they make. That's called listening directly to their users. Do you really think they're about to screw that up by serving you up some sort of weird paid spam after doing all that?
  • The DearAOL site talked about how the signatories there "have always been happy working together with you to fight spam and phishing." Well, so is AOL. AOL was then, and AOL is today. For years (since at least 2003), AOL’s had this useful site called “Postmaster.Info,” specifically geared toward helping senders address issues sending mail to AOL users.
I’m not here to sell you on Goodmail. I have no vested interest regarding whether or not you use it or don’t use it. I think it might work well for some people, and less well for others. That’s not the point.

The point is this: Go over to your window. Look out. Look up. See the sky? It’s still there. It hasn't fallen, regardless of what the EFF wanted you to think.

I guess it was just a rumor after all.

Backscatter: What is it? How do I stop it?

What is it?
Backscatter is a certain kind of mail you receive that you didn't ask to receive. If you've ever received a “Your mail could not be delivered” bounce notification, a “Your mail contained a virus” notice, or a request to confirm your signup request for a mailing list you've never heard of, you've probably received backscatter. The backscatter problem is inherently linked to the spam problem, as most backscatter received is due to somebody else on the internet doing something bad and spam-related.

Types of backscatter:
  • Misdirected bounces from spam runs, from mail servers who “accept then bounce” instead of rejecting mail during the SMTP transaction.
  • Misdirected virus/worm “OMG your mail was infected!” email notifications from virus scanners.
  • Misdirected “please confirm your subscription” requests from mailing lists that allow email-based signup requests.
  • Out of office or vacation autoreplies and autoresponders.
  • Challenge requests from “Challenge/Response” anti-spam software. Maybe C/R software works great for you, but it generates significant backscatter to people you don't know.
How bad is this problem?
Let's do some quick math on the back of a napkin. A quick check of my personal spamtrap account finds 2,039 pieces of backscatter, just by searching for a few common phrases found in bounce messages and challenge/response requests. Out of the 320,000 recent pieces of spam I've got in that account, that may not seem like much. But that's just me and my tiny domains. Think about a big site, like say, Outblaze, with millions of users across hundreds of domains.

Spam lists contain a high percentage of invalid addresses, driving a high bounce rate. A normal mailing to a legitimate list will result in 3-10% of the mail bouncing, per mailing. And that's if you handle bounces properly. When working with senders, I know that if they haven't tracked bounces properly in the past, a list can have anywhere from 30-50% bounce rate. Spammers rarely handle bounces properly, so let's assume a 40% bounce rate on a spam run. If a spammer sends two million pieces of spam, that leaves 800,000 bounces. Where do those bounces go? I know from helping clients legitimately handle bounces, maybe 7-10% of email servers accept the mail then bounce it back later.

Do the math based on all of these assumptions: 2,000,000 spams, 40% bounce rate, 9% of mail servers send backscatter... That means that for that two million spam run, 72,000 bounce notifications (NDRs) are going to be sent back to the sender address. Since spammers forge the sender's address, this mail is going to be be received by people who had nothing to do with the spam. This, in a nutshell, is backscatter. And there's a lot of it floating around.

How can you stop backscatter?

If you're an end user, there's probably not a ton you can do to prevent the receipt of backscatter. Some anti-spam blacklists (such as backscatterer.org) actively block servers that generate backscatter. Spamcop used to do so in the past, but I don't believe they actively target these kind of servers at the moment. Though, any spamtrap-based DNSBL is going to (intentionally or not) catch servers sending backscatter, because the backscatter will hit their spamtrap addresses just like it's hitting you and everybody else.

Whatever you do, don't use a “Challenge/Response” anti-spam application or service. It makes the problem worse for everybody else on the internet – your challenge requests are just another kind of backscatter.

The same goes for those anti-spam applications that promise to send fake bounces on your behalf. The reasoning is that the spammers will realize your address is dead, and stop sending you mail. That's simply not true. You're going to bounce off the lists of good senders, who actually process bounces. The bad guys (whom are responsible for far more of the mail you receive) don't process bounces. They're the ones creating this whole backscatter problem to begin with. Yours will be just another bogus bounce notification bothering some innocent, unrelated third-party.

Don't set an “out of office” reply, either. Besides contributing to the backscatter problem yourself, you're sending random notes out to the world telling whoever receives the notice that you have a live email address and you'll be back soon to receive it. Guess what? You're confirming for spammers that your email address is valid! That means you're going to get even more spam. I see over 800 out of office replies in my spamtrap mailbox. If I were a sleazy spammer, it would be very easy to write a script to save those addresses in a file, and wait and see how many more I get today, tomorrow, next week, etc.

It might be useful to report backscatter as spam via your favorite spam-reporting service. It helps to collect stats on the problem, and helps to nudge sites to fix their backscatter-generating problems.

If you administer a mail server, here's what you can do to minimize your contribution to the backscatter problem:
  • Don't allow email-based signup requests for your email list. I've got more than 330 “please confirm your subscription” requests in my spamtrap account. If you do this, you're going to end up blacklisted, because these misdirected requests are going to hit spamtraps run by anti-spam groups, too. Instead, only allow web-based signups. People go to your website, fill out a web form to sign up, then send them the confirmation request.
  • Don't use Challenge/Response, and don't allow your users to, either. As I mention above, it fixes your spam problem at the cost of everybody else's spam problem. Few, if any implementations of C/R are smart enough to prevent responses to spam messages and other random junk. Justin Mason puts it succinctly: C/R is “broken and abusive.” Over on his blog, he rounds up a bunch of opinions on why this technology is flawed. I've also made my opinions known on the subject, as well.
  • Don't run autoresponders, out-of-office notifications, etc. You'll annoy people and your mail could end up being blocked. I've seen people complaining that anti-spam blacklists have listed them because of this kind of backscatter. This generally impacts your mail far beyond out of office responses. Spamcop has a FAQ with more information.
  • Don't bounce mail after accepting it from a remote site. If you run an unpatched version of Qmail, or some other MTA that accepts all mail, then later sends bounces for some of that mail, then you are at the heart of the problem. Those who came before you learned a long time ago that this behavior contributes to the internet's spam problem. That includes AOL (the site with possibly the largest percentage of end-user mailboxes in the US), who has spent tons of time and money over the years, ensuring that as much as their rejected mail as possible is rejected during the SMTP transaction, and not later. Before they made this change, they were a significant source of backscatter. They may not be perfect at this today, but they've made huge strides, and are to be commended.
    Jeremy Harris was kind enough to pass along this support note from Microsoft on how to configure Exchange 2003 so that it does not work in "accept, then reject" mode.
    You might ask yourself, "What's the point? If I reject the mail, somebody will be sending a bounce." That's usually not true -- most spam is sent by infected zombie computers, and the software spammers are running doesn't bother with things like bounces.
  • Configure your virus scanner to silently strip or discard viruses/worms instead of sending a notification back to the sender. It's a fact: Viruses and worms sent via email do not have valid sender addresses. If you send a notification back, it goes to the wrong person, an innocent third-party. This is backscatter, and your server just succeeded in annoying somebody.
  • If you run an ISP, rate limit outbound mail or detect usage of large numbers of different envelope senders. This helps to combat “outscatter,” which is what happens when users on your network have virus-infected desktops utilizing your SMTP servers to send outbound spam. Since they're on your network, they'd normally be allowed to relay mail through your server. But, if left unchecked, you'll find your outbound mail servers blacklisted for sending spam, as the place it's being transmitted from, as far as the rest of the internet is concerned, is your outbound mail server.
  • Read and learn. Anti-spam groups like Spamhaus have posted and linked to useful information about the backscatter problem and what to do about it. Justin Mason's blog covers this topic, as well. The Spam Links site has an entire section devoted to backscatter. It's worth reading.
I hope you found this information useful. If you see anything out of date, or think there's more information I should consider adding, don't hesitate to contact me with your thoughts and feedback.

Joe Jared Wins

Joe Jared used to run an open-relay blacklist called Osirusoft. I respected what Joe’s intent was, though I was of the opinion that Joe ran things unwisely. Both of these things I mentioned way back in 2001, when an SF Gate Article slammed Joe (and blacklisting in general).

I recall that Joe has been defensive and combative in the past when people with affected/blocked mail reach out to him. He was also not very good at record keeping, something I knew from the past, and that fact came up in this case (Pallorium vs Jared).

Pallorium, a private detective agency, sued Joe, saying more or less that they weren’t running an open relaying mail server, they weren’t spammers, that Joe was careless, and that he offered no recourse to resolve the listing, because his website was periodically unavailable.

The court documents indicate that Joe admits that he had no idea how the server came to be listed, and that Joe’s tests after the lawsuit saga began show that the server was not actually an open relay. If records were kept showing where and why something was listed, I suspect it would’ve saved him some effort and perhaps prevented the lawsuit. It probably didn’t help that Joe was rude to the plaintiff on the phone about the issue, too.

Regardless, this case was clearly resolved in favor of spam blocking. I've always said that ISPs and anti-spam groups are pretty much free to block whatever mail they want, in their best efforts to block spam. The court and I seem to be in agreement on that point.

On one of the public anti-spam lists I’m, this spurred a discussion positing that this allows folks to probe for open relays and add them to a blacklist, as long as the person doing so has been receiving objectionable spam. Perhaps, but this isn’t quite precedent-setting. My understanding is that this would only apply in California courts, and that the decision is unpublished anyway. Not to mention, open relays (which I used to help people block myself back when I ran blacklists) are no longer the primary source of spam that they once were.

(If you don’t know what an open relay is, check my recent post on the topic.)

Senders: How to avoid false positives

As I often say, listen to as many sources of information as possible, and learn as much as you can from all of them. Don't be afraid to search for a second or third opinion on how to handle a deliverability problem or list management issue.

To that end, it's worth checking out this page. This information from the makers of the popular SpamAssassin spam filter offers valuable insight into how their system works. If you’re a mail sender, it’ll help guide you on how to finesse your legitimate mail, to reduce the likelihood that it'll be incorrectly classified as spam.

I don't agree with every suggestion it makes (many non-spammy senders use open detection that sets off their "web bug" detector, for example), but overall, there's some good info here. Read it, learn from it, and see what easy steps you can take to comply with these guidelines.