Gmail bounce issues on 12/14-12/15

Google's Gmail platform appears to have been throwing intermittent false "user unknown" bounces for just about the past 24 hours. The raw bounce would suggest that an address does not exist, when it does actually exist. Example:

550-5.1.1 The email account that you tried to reach does not exist. Please try
550-5.1.1 double-checking the recipient's email address for typos or
550-5.1.1 unnecessary spaces. Learn more at
550 5.1.1  https://support.google.com/mail/?p=NoSuchUser

Google's initial application status information suggested that the issue was not long lived, starting at around 5:30 pm US EST on Monday evening, for perhaps 90 minutes, based on reports from others. However, intermittent issues with "false positive" bounces continued to be observed throughout Monday evening into and throughout Tuesday afternoon and early evening.

If you invalidate addresses due to a "user unknown" bounces, data cleanup after resolution can be tricky. Not every single bounce was invalid, so you perhaps don't want to simply "zero out" all bounces from that time period. If possible, look for Gmail subscribers that just bounced but have previous open or click activity -- those addresses are most likely to still be valid. If that's not easily doable, then I'd suggest perhaps resetting any bounces for gmail.com subscribers that occurred  from 5:00 pm EST on 10/14 until 7:00 pm EST on 10/15. There may not be a "one size fits all" remediation for this issue.

Update: As of 6:15 pm US CST on Tuesday, the issue appears to be resolved. From the Google Workspace Status Dashboard: As of 5:51 pm US CST: "The problem with Gmail has been resolved. We apologize for the inconvenience and thank you for your patience and continued support. Please rest assured that system reliability is a top priority at Google, and we are making continuous improvements to make our systems better. If you are still experiencing an issue, please contact us via the Google Help Center."

Final Update: Google later published a root cause analysis explaining what went wrong. You can find that here. I thank them for providing this good level of detail. One little nit that I'll pick is that they lay it out as if the issue was occurring during one small window on each of two days -- myself and others did observe false positive bounces in the period in-between the two windows. Thus from where I sit, that was seemingly more like one long window over 12/14 and 12/15.

No comments:

Post a Comment

Comments policy: Al is always right. Kidding, mostly. Be polite, and you're welcome to join in, even if it's a differing viewpoint.