Those who have been in the Deliverability world for some time have seen almost constant change to the point that half their job is just keeping up with the latest new standards and requirements. It’s nothing new to say that IP reputation is gradually being replaced by domain reputation in a lot of filtering decisions. Still, there’s something new on the horizon:
While the MBPs have been factoring domain reputation into their reputation systems for some time, their MTAs have historically provided feedback through tempfail and permfail messages in the context of the incoming IP address. This practice is in some ways traditional: in the early days of SMTP each entity had its own sending MTA with its own domain and IP address, so all you needed to do was tell that sending MTA how it should behave and you were telling that entity how to behave.
Things started changing as ESPs were introduced: those multitenant senders built up large numbers of domains and IPs sending from their MTAs, and unless their tenants were using dedicated IPs the historically IP-based feedback to the MTAs was no longer about how a given entity was sending, but how the IP itself was sending. For the most part this didn’t really change things, because the MBPs were still providing IP-based feedback to the senders. This could range from responses such as “Too many connections” to “Too many messages per connection” and everything in between. These usually arrive as 4xx temporary failures, and there’s no real consistency between the MBPs as to how these messages are formatted (but ideally most will adhere to using the IANA status codes so that there’s some consistency).
This situation has recently started to change, with Google and Yahoo! publishing revised standards and practices for senders, we’re now seeing new responses introduced that are based on the sending domain rather than on the sending IP address. One example of a new response is:
550, "5.7.26", "This message does not have authentication information or fails to pass authentication checks (SPF or DKIM). To best protect our users from spam, the message has been blocked. Please visit Prevent mail to Gmail users from being blocked or sent to spam for more information."
This response breaks from tradition in that it’s not specifically related to the IP address (outside of SPF purposes), nor is it specifically related to the message itself (outside of DKIM), instead it’s related to the domain that the message is sent from, essentially the tenant itself in an ESP context.
The MBPs expect that the sending MTA will react appropriately to the response that it generates. For example, if the sending MTA gets a response of “too many connections” it’s pretty obvious that the MBP would like the sending MTA to open fewer connections. The authentication response shown above is not generally something that the sending MTA can automatically fix, so most MailOps administrators would prefer to pause sending in this case so that they can resolve the issue before more messages are rejected.
The challenge is that historically even those MTAs that could take automated actions based on the responses from the MBPs do so in the context of a sending IP: again, in the past most feedback that you could automatically act on was in the IP context around sending rates, connection counts, etc. In a shared IP environment one can’t pause all sending on a given shared IP just because one tenant doesn’t have authentication properly configured. The result of this situation is that the only outcome available to the MailOps team is to let the rest of the affected tenant’s messages be attempted and for them all to encounter the same permanent failure message.
Many “multi-tenant” MTAs are based on some variation of the “Virtual MTA” concept where each IP is treated like its own server with its own throttles and log information, but this falls apart in the common shared IP model that is prevalent at ESPs because a pool of these virtual MTAs no longer represents a tenant, it just represents a collection of IPs with no way to operate on the underlying tenant.
We knew this was a challenge for ESPs, so we built KumoMTA on the concept of tenants and campaigns as first-class queue elements, not just metadata for the logs. As a result, KumoMTA users can granularly configure and manage the queues of individual tenants and individual campaigns, without the MTA running expensive functions that walk the entire queue looking for the desired metadata. More importantly it means that certain traffic shaping rules can be applied at the tenant level instead of at the IP level.
KumoMTA users can set throttles to control how fast a given tenant can send for QOS purposes, as well as write automation rules that reflect the sending domain instead of the sending IP. MailOps can deploy a rule like this to help them catch configuration issues and fix them without having to let all of the impacted tenant’s messages bounce:
[["gmail.com".automation]] regex = "This message does not have authentication information" action = "SuspendTenant" duration = "3 hours"
By monitoring these automated actions, the MailOps team can investigate these new domain-specific responses and resolve them, all while leaving the IP itself active so that all other tenants using that same shared IP are unaffected by the misconfiguration of one of their peers.
Those who have been in the Deliverability world for some time have seen almost constant change to the point that half their job is just keeping up with the latest new standards and requirements. It’s nothing new to say that IP reputation is gradually being replaced by domain reputation in a lot of filtering decisions. Still, there’s something new on the horizon:
While the MBPs have been factoring domain reputation into their reputation systems for some time, their MTAs have historically provided feedback through tempfail and permfail messages in the context of the incoming IP address. This practice is in some ways traditional: in the early days of SMTP each entity had its own sending MTA with its own domain and IP address, so all you needed to do was tell that sending MTA how it should behave and you were telling that entity how to behave.
Things started changing as ESPs were introduced: those multitenant senders built up large numbers of domains and IPs sending from their MTAs, and unless their tenants were using dedicated IPs the historically IP-based feedback to the MTAs was no longer about how a given entity was sending, but how the IP itself was sending. For the most part this didn’t really change things, because the MBPs were still providing IP-based feedback to the senders. This could range from responses such as “Too many connections” to “Too many messages per connection” and everything in between. These usually arrive as 4xx temporary failures, and there’s no real consistency between the MBPs as to how these messages are formatted (but ideally most will adhere to using the IANA status codes so that there’s some consistency).
This situation has recently started to change, with Google and Yahoo! publishing revised standards and practices for senders, we’re now seeing new responses introduced that are based on the sending domain rather than on the sending IP address. One example of a new response is:
550, "5.7.26", "This message does not have authentication information or fails to pass authentication checks (SPF or DKIM). To best protect our users from spam, the message has been blocked. Please visit Prevent mail to Gmail users from being blocked or sent to spam for more information."
This response breaks from tradition in that it’s not specifically related to the IP address (outside of SPF purposes), nor is it specifically related to the message itself (outside of DKIM), instead it’s related to the domain that the message is sent from, essentially the tenant itself in an ESP context.
The MBPs expect that the sending MTA will react appropriately to the response that it generates. For example, if the sending MTA gets a response of “too many connections” it’s pretty obvious that the MBP would like the sending MTA to open fewer connections. The authentication response shown above is not generally something that the sending MTA can automatically fix, so most MailOps administrators would prefer to pause sending in this case so that they can resolve the issue before more messages are rejected.
The challenge is that historically even those MTAs that could take automated actions based on the responses from the MBPs do so in the context of a sending IP: again, in the past most feedback that you could automatically act on was in the IP context around sending rates, connection counts, etc. In a shared IP environment one can’t pause all sending on a given shared IP just because one tenant doesn’t have authentication properly configured. The result of this situation is that the only outcome available to the MailOps team is to let the rest of the affected tenant’s messages be attempted and for them all to encounter the same permanent failure message.
Many “multi-tenant” MTAs are based on some variation of the “Virtual MTA” concept where each IP is treated like its own server with its own throttles and log information, but this falls apart in the common shared IP model that is prevalent at ESPs because a pool of these virtual MTAs no longer represents a tenant, it just represents a collection of IPs with no way to operate on the underlying tenant.
We knew this was a challenge for ESPs, so we built KumoMTA on the concept of tenants and campaigns as first-class queue elements, not just metadata for the logs. As a result, KumoMTA users can granularly configure and manage the queues of individual tenants and individual campaigns, without the MTA running expensive functions that walk the entire queue looking for the desired metadata. More importantly it means that certain traffic shaping rules can be applied at the tenant level instead of at the IP level.
KumoMTA users can set throttles to control how fast a given tenant can send for QOS purposes, as well as write automation rules that reflect the sending domain instead of the sending IP. MailOps can deploy a rule like this to help them catch configuration issues and fix them without having to let all of the impacted tenant’s messages bounce:
[["gmail.com".automation]]
regex = "This message does not have authentication information"
action = "SuspendTenant"
duration = "3 hours"
By monitoring these automated actions, the MailOps team can investigate these new domain-specific responses and resolve them, all while leaving the IP itself active so that all other tenants using that same shared IP are unaffected by the misconfiguration of one of their peers.
Comments
Post a Comment
Comments policy: Al is always right. Kidding, mostly. Be polite, please and thank you.