Forum Discussion
CirrusRandom TCP Resets from F5
Good day all,
I am researching an intermittent and random issue where our F5 WAFs respond to customers with the following:
"An error occurred while sending the request.::Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..::An existing connection was forcibly closed by the remote host."
I collected PCAPs and examined the traffic and it leads back to the WAFs sending the reset. I've searched this issue with A.I. assistance and it suggested adjusting/increasing the client ssl profile "handshake timeout" value from 10 seconds (default) to possibly 20 and 30 seconds, depending on traffic load.
Is this a legitimate suggestion and potential resolution? Has anyone modified their "handshake timeout" setting from default?
I appreciate your time and energy and look forward to your thoughts and suggestions. Thanks!
6 Replies
- Daniel_Wolf
MVP
Hi pcourtois,
I am afraid AI won't take you far in this scenario...
The error message you posted looks like a generic .NET error message. The F5 by itself would not send such error message.
The BIG-IP sending a reset package is also not unusual, sometimes that's just how TCP works.
For example: if the TCP session is timed out on the F5 and client sends data on this timed-out session, the F5 would reply with a RST, ACK.In which direction is the F5 sending the reset, to the client or to the server?
If the F5 sends a RST,ACK to the client and then, as a result, the client shows the above error message, you should adjust the tcp timeout settings in the client side TCP profile.Hope this helps,
Daniel- pcourtois
Hi Daniel, thank you for the reply. The response is coming from the F5 so I will work on adjusted handshake timeout settings in my "dev" environment and update when i have some data.
- Paul
- Juergen_Mang
Enabling Reset cause logging can give you a hint: https://my.f5.com/manage/s/article/K13223
tmsh modify sys db tm.rstcause.log value enableTest and review /var/log/ltm
- Nikoolayy1
Also pcourtois you can see Juergen_Mang article
community.f5.com/kb/communityarticles/the-state-of-http2-full-proxy-with-f5-ltm/344157 as if it is http/2 traffic WAF can send random reset.
- pcourtois
Thanks, Juergen. I'll try this and update as soon as possible.
- Paul
- pcourtois
Daniel_Wolf and Juergen_Mang,
Here are the logs from my dev Big-IP for the "(tmos)# show /net rst-cause". These two views of the logs were taken 2 minutes apart. You can see the increase in "handshake timeout." In your experience, would increasing "handshake timeout" in our client ssl profile help?
-------------------------------------------------
TCP/IP Reset Cause
RST Cause: Count
-------------------------------------------------
Flow expired (sweeper) 75008
Flow expired (sweeper: pool member down) 992
HTTP header size exceeded by client 15
ICMP unreachable received 4
Incomplete chunked response 47
Malformed HTTP header error 128
No flow found for ACK 158313
No pool member available 154381
No server selected 120
RST from BIG-IP internal Linux host 14233
SSL error 12
SSL handshake timeout exceeded 11014
SSL proxy shutdown 50
TCP RST from remote system 161318
TCP bad flags 6
TCP closed 18
TCP keep-alive timeout 232
TCP retransmit timeout 13849
TCP zero window timeout 6
handshake timeout 5020190
iRule execution error 33
-------------------------------------------------
TCP/IP Reset Cause
RST Cause: Count
-------------------------------------------------
Flow expired (sweeper) 75013
Flow expired (sweeper: pool member down) 992
HTTP header size exceeded by client 15
ICMP unreachable received 4
Incomplete chunked response 47
Malformed HTTP header error 128
No flow found for ACK 158315
No pool member available 154404
No server selected 120
RST from BIG-IP internal Linux host 14233
SSL error 12
SSL handshake timeout exceeded 11015
SSL proxy shutdown 50
TCP RST from remote system 161318
TCP bad flags 6
TCP closed 18
TCP keep-alive timeout 232
TCP retransmit timeout 13849
TCP zero window timeout 6
handshake timeout 5020388
iRule execution error 33
I've also enabled SSL Debug logs and have multiple logs of the following that don't really provide much detail:
warning tmm1[11842]: 01260013:4: SSL Handshake failed for TCP clientIP:port -> bigip:port
