F5 sending all synthetic transactions to one host

Can you check if you have persistence enabled on pool? There must be a source address persistence or cookie persistence which causing the issue. Also please check if you have any one-connect profile enabled on virtual servers as well.

We found the root cause. The extensive logging was enabled on the script and we found the script actually made 2 connections. One to test the connection and the 2nd one actually sent the SOAP request. Since these devices are not live yet, the only traffic on them are the synthetic transactions. So it appears that the F5 was only sending traffic to one device.

We proved this by looking at the logs and finding which device was getting the soap request from the monitoring tool. We then sent a SoapUI request so it would hit the same device.

On the next synthetic transaction soap request the direction would change and started hitting the other device.

Another indication that an initial test connection was being made by the monitoring script was that we were seeing SSL handshake errors and peer did not send certificate errors on the device that appeared not to receive any traffic at almost the exact same time the synthetic transaction SOAP request was hitting the other device.

Hence monitoring script would do a test connection without sending a certificate to device A and then a second connection to send the actual SOAP request with the certificate hitting device B.