On the APEWS Blacklist

Lots of talk about the "anonymous" APEWS blacklist lately. Over on DNSBL Resource, I summarize everything I've seen on the topic, and include some info regarding its effectiveness as an anti-spam filter against my own spamtrap and hamtrap.

Additionally, I've added a page with tips on what to do if you find yourself blacklisted by APEWS.

The Virtumundo/Jim Gordon Affair

Internet email and security guru John Levine sums it up a lot better than I ever could, so I'll simply point you in his general direction.

Update: John Levine pulled his post down, replacing it with this text: "This post has been withdrawn due to objections from Virtumundo's lawyers."

He links to a copy of the judge's order, which can be found here.

Also, SpamSuite.com has more information on the topic, which can be found here. In addition to commentary, SpamSuite highlights the following excerpts from the order:
"the Court begins by expressing serious doubts about the accuracy with which Defendants’ attorneys recorded and billed both costs and fees in this litigation."
"Furthermore, the prospect that ... well over 1,000 hours—was spent on the Linke Log is absurd."
"Having seen the results of this project, the Court finds that spending the equivalent of over thirteen 40-hour weeks on this process is far more than was reasonable."
"Moreover, the inaccurate documentation presented with the instant motion reinforces the Court’s separate conclusion that the hours requested exceed the reasonable time spent on this case. Given that in making the instant motion Defendants have inexplicably inflated the total hours for which they request compensation by almost 27% beyond what was even recorded in their own billing records, the Court finds it entirely appropriate to cut their requested senior attorney hours by at least that much to account for other inflation that likely occurred in daily billing and overcharges to their clients, which may or may not have been partially balanced out by bill cuts and discounts."
"it appears to the Court that Defendants have deliberately doubled the requested compensation"
"It is unclear how Defendants arrived at the total of $26,338.01 requested in their motion. Moreover, as discussed, the individual expense requests that total $28,839.36 here also are inexplicably inflated when compared with the actual billing records submitted to the Court."

Blah on Challenge Response

Richi Jennings breaks it down: Peter Brockman, and open questions on C/R success rate determination methodology. As Richi puts it,

"Statistics aside, asking C/R users if they're happy isn't the be-all and end-all of anti-spam research. C/R users may indeed be happy -- happily unaware that their spam filter is sending spam by replying to innocent third parties who's addresses have been forged by spammers."

Spot on.

Justin Mason's take on it is accurate and insightful, as well:

"Now, here’s the first problem. The “Spam Index” therefore considers a false negative as about as important as a false positive. However, in real terms, if a user’s legit mail is lost by a spam filter, that’s a much bigger failure than letting some more spam through. When measuring filters, you have to consider false positives as much more serious! (In fact, when we test SpamAssassin, we consider FPs to be 50 times more costly than a false negative.)"

Justin hits the nail on the head. Part of the problem a number of anti-spam "researchers" have in common is discounting the damage done (or even inaccurately counting FPs) by doing things like relating the number of "hits" a blacklist or spam filter gets and assuming that the more hits you get, the better.

Then add in the, um, awesomeness of C/R, in that you're bouncing unwanted spam back to unrelated parties who were forged in from lines. C/R is a good way to block spam, by bouncing it off your bad filter and in to somebody else's inbox. That's like keeping criminals away from you by helping them break into your neighbor's home. Yuck.

Happy Friday from...the Baron!

I've apparently been dubbed "the Baron of Blacklists" for "waxing lyrically" on the subject of DNSBLs. If you're wondering what that's all about, Melinda Krueger published some information about blacklists in a recent Email Diva column. A long time subscriber myself, I thought it would be helpful to provide some more detail and clarification. So, I dropped her an email, which landed in a follow-up Diva column with my blessing. Neat!

Of course, to see what the Baron of Blacklists will be waxing lyrically about next, head on over to my other site, DNSBL Resource.

Where was the consumer?

My friend Neil Schwartzman asked me a question during the FTC Spam Summit a couple of weeks ago. He asked me, “Where's the consumer?”

Neil, executive director of CAUCE (the Coalition for Unsolicited Commercial Email) in North America, had a point. The whole point of this exercise is figuring out how to answer the question, how do we protect the consumer? Problem is, there were a lot of consumer groups completely unrepresented at the event. It's great that they got Consumer Reports and Consumer Action to participate. In particular, Consumer Reports teased us with an upcoming review of spam filtering applications. Good stuff!

But, there was still a glaring omission: Where were the consumer groups actually focused on dealing with the spam problem? Where were the blacklists? How come CAUCE wasn't on a panel?

These are the groups actively fighting behind the scenes to preserve email. Working across countries, across boundaries, to solve the spam problem. The blacklists work hard to identify bad actors (often at significant personal legal liability), enabling receiving sites to more easily reject unwanted mail. Not everybody agrees with their methodology, and not everybody agrees with their goals. That's OK-- the same can be said of just about anybody else who was represented at the event. That doesn't mean they don't deserve a seat at the table.

That seat is important, for two simple reasons. One, so they can educate the rest of us of their point of view and all the valuable information they have. Two, so we can educate them. Put everybody in as room, get them to listen to each other, and something rubs off in both directions – usually for the better.

By not including CAUCE, or any of the blacklist groups like Spamhaus, SURBL, NJABL, PSBL, etc., in any of the panel discussions, we all lost out on that opportunity.

I'm very disappointed.

Blacklist notifications? Think again.

Infacta's "Messaging Times" posted a generally good article today on what you should be doing to minimize blacklistings. Except...

The article posits that "blacklist agents" should "contact senders that were reported prior to listing them with a plain-English explanation of what was reported and give them an opportunity to respond appropriately prior to being blacklisted. This process should be clear with instructions that are easy to follow."

Whoa. This is untenable on every possible level. Why?
  • The vast majority of this spam is coming from forged addresses, overseas IPs, or infected machines (or all of the above). Notification to the listee is far from trivial and it will send bogus notifications to the wrong person 99% of the time. It is not worth it just to notify the 1% person who is actually reading his postmaster/abuse mailbox and speaks English.
  • It just doesn't scale. Consider: My tiny random site receives, on average, ten thousand spams each day. Of the (approximately) 807,998 spams I've received since March 10, they came to me from 532,958 unique IP addresses. You expect me to send out over five hundred thousand notifications? Now explode that out exponentially to the real levels that blacklists deal with (which reveal my volumes to be puny).
  • Smart senders check their bounces. The default configuration for blacklist usage includes a clear message with every bounce containing a link to a site or reference code with more information. This is notification. Do your due diligence and you'll notice a blacklisting within minutes or hours of it taking place. In most cases it is then easily and simply resolved.
  • Smart senders periodically check blacklists to see if their IP addresses are listed. Any good email service provider (ESP) offer this service. Sites like DNS Stuff and Open RBL make it easy to check a bunch of lists at once.
  • Good email actually doesn't get blacklisted very often. Sure, there are badly run blacklists out there (and I catalog both good and bad ones over on www.dnsbl.com), but most lists are not run by bad guys and are not out to attack people sending regular opt-in mail. If you are regularly ending up on lists like Spamhaus, NJABL, CBL, etc., then you're probably doing something wrong. If you're regularly getting blocked at Yahoo, Hotmail, or AOL, then you're probably doing something wrong. Fix your list. Stop trying to blur the lines of permission. Stop mailing to bounced email addresses repeatedly. Confirm new signups. Re opt-in your existing lists. Be proactive. It's not up to some external third party to tell you that you screwed up; if you let it go and got bitten by a blacklisting, you've usually got nobody to blame but yourself. The real problem is whatever caused the blacklisting, not the lack of a notification.
Notifying everybody listed on a blacklist is a noble goal. It was a goal of mine, back when I created the RRSS blacklist in 1999 (that later went on to become the MAPS RSS). Back then, I found that notifications did nothing but annoy unrelated parties and generate more bounces back to my own mailbox. It's telling that today, no blacklist I'm aware of notifies somebody before placing them on the list. For a lot of these lists, the point is to mitigate the potential damage of spam being received from listed hosts, while the host's owner or ISP is asleep at the wheel, not to prod the host owner to be friends with them.

Next, the article mentions "email authentication systems" referring to things like Goodmail and Sender Score Certified. These are actually email certification services, not authentication systems. You can choose to participate in a certification system, but it's not required on any level to get your mail delivered. Email authentication systems are actually things like SPF (Sender Policy Framework), Sender ID, DomainKeys, and DKIM. These all make it easier for receivers to identify senders and help their efforts to improve their ability to discern the good mail apart from the bad mail. They don't cost anything. SPF and Sender ID are things you set up in your DNS and can be done in about five minutes if you're technically inclined. DK/DKIM require support at the mail server sending side. Sometimes this is free, sometimes it might require an upgrade. This is like upgrading any piece of software, though, and it it's part of some conspiracy to make you pay to have to send email. (I think in the future you'll find just about every free or commercial mail server software will support DK or DKIM.)

And finally, the article asks the question, "Since when did the world "free" become a bad word?" The answer is: It didn't. It's not. The vast majority of spam content filters don't do anything so simplistic as to filter or block a message just because it contains the word "free." Don't be afraid to use the word "free." If you're not sending spam, it's not likely to get you blocked.