Forum Discussion
kash_49328
Nimbostratus
NimbostratusBig IP LTM sending tcp Resets due to SSL handshake time out ?
Hi F5 gurus, We have a https file transfer going on a daily basis and we are experiencing a big problem here. Client is a java program and server is behind the F5. We are offloading ssl on F5 so we use client ssl profile with default settings ( Version 11.2.1 LTM, ssl handshake time out = 10 sec ) . Tcpdump is saying that RST are generated from F5. As per F5 Handshake time more than 10sec will make system vulnerable to DoS attack. Also client route through many network devices before hitting the G5 big ip. I have enabled the rstcause.log and rstcause.pkt which gives me below logs.
ltm 12-15 17:00:13 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14720, [0x147b9e1:962] SSL handshake timeout exceeded
ltm 12-15 17:00:13 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14716, [0x147b9e1:962] SSL handshake timeout exceeded
ltm 12-15 17:00:13 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14718, [0x147b9e1:962] SSL handshake timeout exceeded
ltm 12-15 17:00:13 err lb1 tmm[9546]: RST sent from server ip :443 to client ip :14719, [0x147b9e1:962] SSL handshake timeout exceeded
ltm 12-15 17:00:14 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14716, [0x147db9a:4315] TCP 3WHS rejected
ltm 12-15 17:00:14 err lb1 tmm1[9547]: RST sent from server ip :443 to client ip :14718, [0x147db9a:4315] TCP 3WHS rejected
ltm 12-15 17:00:14 err lb1 tmm[9546]: RST sent from server ip :443 to client ip :14719, [0x147db9a:4315] TCP 3WHS rejected
Please help me if you have seen this problem before.
Thanks in advance
nitassEmployee
isn't 10 seconds for ssl handshake enough? it is only handshake (not data transfer), isn't it?
kash_49328Not sure earlier versions used to have 60 sec timeout
afedden_1985Cirrus
You might try using ssldump to capture the connection between the F5 and the Server. It might help find root cause. https://support.f5.com/kb/en-us/solutions/public/10000/200/sol10209.html
- kash_49328Yeah i did that but its not actually giving me any specific reason why BIG ip is sending RST. 2 4.7914 (0.0000) S>C TCP FIN 2 258 4.8733 (0.0818) C>S Alert 2 4.8733 (0.0000) C>S TCP FIN 6 5 9.3710 (9.2912) C>S Handshake ClientKeyExchange 6 6 9.3710 (0.0000) C>S ChangeCipherSpec 6 7 9.3710 (0.0000) C>S Handshake 6 8 9.3757 (0.0046) S>C ChangeCipherSpec 6 9 9.3757 (0.0000) S>C Handshake 6 10 9.4564 (0.0806) C>S application_data 6 11 9.9540 (0.4976) S>C application_data 6 12 9.9546 (0.0005) S>C application_data 6 13 10.0347 (0.0801) S>C application_data 1 10.0793 (9.9991) S>C TCP RST 3 10.0793 (9.9993) S>C TCP RST 5 10.0799 (10.0001) S>C TCP RST 4 10.0800 (10.0000) S>C TCP RST
- afedden_1985
Your last update didn't get added to the conversation? Are the resets happening all the time or are some flows working? Is it possible the servers SSL ciphers were updated? With all the SSL issues lately a lot in our shop has been changing to mitigate SSL issues. here are a few commands to check ssl cipher compatibility, you can try this on each of the servers in the pool to see if one of them causing a problem. This curl will test a vip or server to see if it allows sslv3 just use the correct address curl -k -v -3 https://IP.addr/test to test for tls support replace the -3 in the above command with –tls1.2 ex: * curl -k -v -3 https://10.10.34.141/test * and each of the tls versions
* curl -k -v -tlsv1.0 https://10.50.2.48/test * curl -k -v -tlsv1.1 https://10.50.2.48/test * curl -k -v -tlsv1.2 https://10.50.2.48/test- kash_49328We are doing SSL offloading at F5 LB. Both client and F5 use TLSv1 for SSL and communication between F5 and actual server is http.
Mike_Sullivan_2You mentioned ssl offloading. That implies you are not re-encrypting on the backend (If you are encrypting the backend connection, you'll need a server side ssl profile). For offloading, your pool members ought to be setup for port 80 and the VIP needs to have Address Translation and Port Translation enabled.
If the pool members are not using the LTM as the default gateway, enable Source Address Translation Auto Map. Get a dump on the backend to see what's happening there.
Cheers, Mike
- kash_49328You are right. No backend encryption, client profile is applied. Address translation, port translation and Automap is already enabled. I am pretty sure its F5 not the server because communication starts at 1700 GMT and first RST i see in the F5 logs ( after enabling the rstcause.log/rstcause.pkt ) is from VIP to client. I do see server's private IP sending RSTs but first its after VIP RSTs. thanks for your reply
- kash_49328Also this VIP works well with other clients. This is only for particular one client.
- Mike_Sullivan_2
What message did the rstcause.log have? SOL13223 has the breakdown on the messages. The fact that it works for all but one client makes my think about local firewalls on the members or even on the client.
Cheers, Mike