Receiving Duplicate List Messages?

The other day, somebody asked me what causes a recipient to receive the same message more than once. I run into duplicate message issues perhaps once or twice a year; not too often, but often enough that a recipient gets really angry at the sending ESP, assuming it's they're fault, because it doesn't seem to be happening with other email.


To that end, I've compiled a few tips I think you might find useful when troubleshooting a recipient receiving multiple messages.

First, review the full headers of the duplicate messages as found on the recipient system. The received headers will be insightful. Which headers are the same across the duplicates and which are not? Compare "received" headers, noting which ones have the exact same time stamps and message IDs. For example, if the "received" header showing receipt at the recipient's mail server is unique for each message, then that tells you that the recipient's mail server is being handed the same message over and over. It may or may not tell you which server is to blame, but the information it provides will be useful.

Here's what usually causes duplicate message issues:

  • The receiving site is running a CISCO PIX firewall with "SMTP fixup" enabled. There's a long standing bug in this firewall that will make it squeeze the SMTP connection shut without telling the sending server that the message was received. The net result is that the sending server will send the same message over and over and over, until the message expires.

    To tell if the receiving site is running a PIX firewall, telnet to port 25. If you get an SMTP banner that is mostly unreadable, containing dozens of asterisks, then you have found a PIX firewall in use.

    To fix the issue, the receiving site needs to turn off SMTP fixup and disable Mailguard. And let's be clear - this is the receiving site's fault. Don't blame the sender. In this scenario, the PIX firewall is not RFC compliant.

  • Perhaps the sending MTA has too short a time out setting, and the receiving MTA is taking too long to reply with its acknowledgement of receipt of the message. The net result is that the sending MTA "hangs up" on the receiving MTA, not realizing that the message was successfully delivered. The solution there would be to configure the sending MTAs to increase the SMTP session timeout setting.
     
  • A disk error on some mail server in the email delivery chain could result in a corrupt, non-deletable lock file, resulting in that machine handing that message off to the next one repeatedly.
Back in the old days, 99% of the time when this was happening, it was due to the receiving site running a PIX firewall. Those things were the bane of my existence. This issue comes up a lot less nowadays, but I know there must still be some old, poorly configured firewalls out there, so that's the first thing I'd look for when attempting to troubleshoot this issue.

3 comments:

  1. I've been working with Cisco firewalls for eleven years and it's only very rarely that I've found an SMTP inspection issue that caused duplicate messages. I believe that current Cisco firewall firmware doesn't have this problem.

    For any suspected firewall problem, my advice would be to engage a firewall expert to diagnose it.

    ReplyDelete
  2. It certainly is likely that current versions of firmware don't suffer from this issue. But, I've been working in email for, oh, somewhere around 13 years now, and I can tell you that 99% of the times this has happened throughout my long lifespan around these parts, it was due to a Cisco PIX.

    As I'd mentioned, this tends to happen less and less often nowadays. But, it does still happen occasionally, I am sad to say.

    ReplyDelete
  3. Yet again, why a decent ability to do telnet SMTP tests is required by anyone who does email.

    ReplyDelete

Comments policy: Al is always right. Kidding, mostly. Be polite, and you're welcome to join in, even if it's a differing viewpoint.