Loadbalance SMTP relay on F5 and exchange 2010

Do you have BIG-IP doing SNAT (source address translation)? This would be set as a property on the virtual server.

My first guess is that you have SNAT enabled. If you disable SNAT, the remote Hub server should see the connection as originating with the IP address of the Hub server(s) and should allow a connection to the custom receive connector.

You may have to adjust routing in your environment to get this to work without SNAT; the important configuration detail is that each destination server needs to have a route back to the source server though the BIG-IP.

Please let us know if SNAT is enabled and if disabling it (and adjusting routing, if needed) solves the problem. If that doesn't work, please open a support case with F5 so we can take a look at more-detailed aspects of your configuration.

When you disable SNAT, as Dayne mentioned, you must configure some additional routing so the pool members know to respond back through the F5. It sounds like right now your pool members are responding to the client, bypassing the VIP on the return path. The client doesn't expect a SYN from the pool member so send a RST.

We worked around this by using GTM instead of LTM. Hope this helps.

Actually, there may be an easier solution.

With SNAT still enabled, what happens if you just specify the BIG-IP's self IP address (or range of SNAT pool addresses, depending on how you have things configured) as allowed when setting up your custom receive connector?

Although the link you provide is a valid and helpful configuration for some scenarios, I don't believe it helps with what you're trying to do.

The scenario outlined on that blog talks about determining which SNAT address (of two available) to use based on the origin IP of the connection. In both cases, though, the BIG-IP still does SNAT, and the receive connector has to be configured to accept connections from the SNATed addresses.

In general, the two approaches you have are:

1) Disable SNAT, so the source IP is still seen to be the original IP of the sender(s). You tried this and were not able to connect at all anymore, even via telnet, which shows that your Hub servers don't have a route back to the origin via the BIG-IP. If you fix the routing, this solution should work (the BIG-IP essentially works as a "smart router" in this scenario).

2) Keep SNAT enabled, but make sure your receive connector can accept connections from the address(es) the BIG-IP is using for SNAT. This will either be one of the self IP addresses, or addresses you define explicitly in a SNAT pool. To confirm which addresses are being used for SNAT, you could do a Wireshark/tcpdump/Netmon capture on the Hub servers to check the source(s) of the connections. To make the captures easier to read, you can disable monitoring first--monitors will always come from the self IP of the BIG-IP, whereas the SNATed connections may come from some other address. You want to make sure you're looking only at addresses from legitimate source "clients".

If none of that works, please let us know the case you open, and I'll take a closer look at your details.