Seedlist testing: It's useful, even at Gmail


I've spent a lot of time lately explaining what exactly seedlist testing is, how it works, and why it's valuable. I like it. I have a bias, after all, in that my day job is product manager for a suite of deliverability tools, and that includes seedlist-based inbox testing (shameless plug: find more info about the Kickbox Deliverability Suite here). Occasionally somebody will tell me that they've heard that seedlist testing can't be trusted or that it's not useful now in 2022, because mailbox provider spam filters are so individualistically focused on user feedback. Yeah...they have a point...sort of. But not quite.

It is very true that spam filters are very user-centric. Gmail, in particular. Your Gmail spam filter is different than mine. Gmail tracks individual user feedback and tailors the spam filtering experience, if you will, based on the different inputs that you and I provide to it. That means that a message from XYZCO could land in the spam folder in my Gmail account, but land in the inbox in your Gmail account, because we could have given Gmail different feedback about our opinion of email from that sender.

It also explains how your CEO got a particular email marketing campaign (from your own company) in the spam folder, while you, lowly marketing manager, did not. Both of you have given Gmail feedback about how you feel about marketing messages. You might even be the builder and tester of those campaigns, so you read each one very closely, clicking on various links. Positive signals sent to Gmail. But then your boss perhaps rarely interacts with the marketing messages. This could be either "no" feedback, or it could be neutral-to-negative feedback signals to be recorded by Gmail. Gmail believes that you want these messages more than your CEO does.

It happens, it is common, it is expected, and it is not something that seedlist-based inbox placement testing will capture.

Seedlist testing doesn't know your customized Gmail filter settings; nor does it know the customized filter settings of your boss. But it does know the default; how Gmail will treat messages for people who do not engage. And those default settings apply to almost all of the subscribers on your list. Think of your best click rates ever. Maybe 2%? 5%? Maybe 40%? That means 60% - 98% of your subscribers aren't clicking aren't sending a positive feedback indication to Gmail. I'm oversimplifying a bit here (as there are more individual feedback signals besides clicking), but if Gmail's getting no feedback from a subscriber, Gmail's going to fall back to the default filter setting -- the overall sending reputation -- of that sender, when it comes to decide whether or not that mail merits inbox placement, or spam folder placement.

In other words, if you send a Gmail message to 10 people, the filtering decisions for those ten people could look like this:

  • Subscriber 1: Hates your messages, reported the last one as spam. This new one is going to the spam folder.
  • Subscriber 2: Loves your messages, reads and clicks on each one. Your new message is very likely to go to the inbox.
  • Subscribers 3-10: Rarely interact with your messages, so they rarely provide feedback. Whether or not your mail gets to the inbox is based on Gmail's overall view of your sending reputation. (Fed by everyone's votes overall, and other factors.)

And that -- right there -- is the value of inbox placement. To find a mailbox provider's view of your sending reputation. Quite valuable, if you ask me, even though it does not (and is not meant to) capture the individual nature of mailbox provider spam filters as they adjust over time. It likely shows you inbox placement results for almost all subscribers, not every subscriber.

I blogged about this with even more information over at the Kickbox blog, if you really want to know everything I have to say on how seedlist testing works and why I think it's useful. But even after that, I had more to say about Gmail in particular, and I thought I would share it here.

Post a Comment

Comments