OK, I suppose I wasn’t clear in my previous posts…sorry!
I have two sites:
SiteA Network: 192.0.2.0/28
SiteA VIP: 192.0.2.10
SiteA “Dummy” VIP: 192.0.2.11
SiteA Internal Nodes: 172.16.10.0/24
SiteB Network: 192.0.2.16/28
SiteB VIP: 192.0.2.24
SiteB “Dummy” VIP: 192.0.2.25
SiteB Internal Nodes: 172.19.10.0/24
We’re load balancing SMTP between 6 physical servers at two geographically dispersed locations (3 at each site). Primary MX is 192.0.2.10 and secondary MX is 192.0.2.24. Because little traffic should actually go to the backup during normal operation, we want to use these servers in the primary MX pool so we don’t have these boxes sitting around twiddling their thumbs, so to speak. These are obviously just example IPs and networks, BTW…
SiteA has two VIPs configured, as does SiteB. SiteA is the primary MX and SiteB is the backup MX. SiteA is load-balanced to a pool configured with three local servers (172.16.10.11, .12, and .13) and one remote server (192.0.2.25). The second VIP, what I call a “dummy” VIP, is configured with only the three local servers. SiteB is configured similarly.
The dummy VIP exists as an loop-prevention mechanism. It exists because if I use the secondary MX as a node in my primary MX VIP, the possibility exists that the backup MX will choose the primary MX node upon receipt of a TCP session. To avoid this, we made a fourth node (the dummy VIP) which consists solely of the three remote servers at each location.
We cannot use SNAT on the sessions which are load-balanced to the servers local to each BIGIP - the sessions appear to come from the BIGIP and the servers start denying connections because there’s too many from a single address. This could be changed in the server configs, but there’s side-effects to that which I won’t go into.
We need to use SNAT for the sessions which are load-balanced to the *remote* nodes, because if we don’t, the original IP is sent in the packet, so the remote dummy VIP attempts to send the ACKs to the original IP - which obviously doesn’t work because no pre-existing TCP session exists with the remote dummy VIP.
I’m sorry for the long explanation - I hope this clears things up at least a little.