Forum Discussion
TCP Profile - extremely pushy F5 (RST after 300 us)
Hello Team,
I do have VIP with the Pool of two servers and standard profile protocol: TCP.
What i have noticed that after TCP SYN packet is routed by F5 to real server F5 is extremely pushy and expects the answer (SYN ACK) in around 300 us. If not getting that answer in that time it is sending TCP RST to both real server and client. This is what is do see on the real server:
20:23:57.147148 IP 172.16.33.1.50197 > 172.16.34.101.80: Flags [S], seq 3228024889
20:23:57.147409 IP 172.16.33.1.50197 > 172.16.34.101.80: Flags [R], seq 3228024890
20:23:57.147416 IP 172.16.34.101.80 > 172.16.33.1.50197: Flags [S.], seq 2676604517
As you can see in my scenario real server is always a bit late (it's not that fast).
This is the standard TCP profile i am using:
profile tcp tcp {
reset on timeout enable
time wait recycle enable
delayed acks enable
selective acks enable
proxy max segment disable
proxy options disable
deferred accept disable
ecn disable
limited transmit enable
nagle disable
timestamps enable
slow start enable
ack on push disable
idle timeout 300
time wait 2000
fin wait 5
close wait 5
send buffer 32768
recv window 32768
keep alive interval 1800
max retrans syn 3
max retrans 8
congestion control highspeed
zero window timeout 20000
}
I have tried to tune it changing multiple options but without success.
Which option should i choose to prevent F5 sending that RST packet so quickly ?
Thanks, Michal
18 Replies
- Vik_K_236702Historic F5 Account
What version are you on ?
- Carl_Brothers
Employee
You need a TCP profile for clients and one for the slower servers. Check out https://devcentral.f5.com/articles/stop-using-the-base-tcp-profile
- teknet7_237497
Nimbostratus
I am using BIG-IP Virtual Edition version 11.6.0 (Build 0.0.401).
Yes - i am using the same profile for clients and servers:
Protocol Profile (Client): tcp-slow
Protocol Profile (Server): Use client profile
All those sessions are accepted and then marked as abandoned
root@(f5)(cfg-sync Standalone)(Active)(/Common)(tmos) show ltm profile tcp ..... ------------------------------------- Ltm::TCP Profile: tcp-slow ------------------------------------- Virtual Server Name N/A Connections Open 0 Current in CLOSE-WAIT/LAST-ACK 0 Current in FIN-WAIT/CLOSING 0 Current in TIME-WAIT 0 Accepted 33 Not Accepted 0 Established 0 Failed 0 Expired 0 Abandoned 36I have noticed that F5 is sending TCP RST even after 110 us after sending SYN to real server.
I am also a bit surprised because when i have tried to use ipother profile TCP checksum was not recalculated (even after i have set TM.TcpUdpTxChecksum software-only) by F5 and my Linux servers were dropping such TCP SYN packets - but since it's not L4 profile it's expected ?
How to troubleshoot my problem ? Is there any way to check why that RST packet is being sent that quickly ?
Thanks, Michal
- JG
Cumulonimbus
Which version of BIG-IP are you running?
- teknet7_237497
Nimbostratus
I have tested multiple servers/services/ports - always the same issue.
I have also checked Vmware settings, could you confirm that the following card should be used:
ethernet1.virtualDev = "vmxnet3"(not E1000) ?
Any idea how to troubleshoot the issue ?
Thanks, Michal
- Brad_Parker_139
Nacreous
your capture only shows one side of the conversation. If the LTM is resetting the connection you would see syn, syn, syn, before a reset is sent with the profile you provided. Try and capture the conversation on both ingress and egress vlans.
- Brad_Parker_139
Nacreous
It is definitely not normal for the LTM to send on SYN followed by a RST less than 200ms later. Using tcpdump on the LTM enable max vebose using "tcpdump -s0 -ni (server vlan):nnn host 172.16.34.101 and host 172.16.33.1 -w /tmp/reset.pcap" where (server vlan) is the name of the vlan used to connect to the pool member. The :nnn should give us a RST cause in the RST packet. You can view this in wireshark using the F5 plugin for wireshark. https://devcentral.f5.com/s/articles/getting-started-with-the-f5-wireshark-plugin-on-windows - Brad_Parker_139
Nacreous
Also, have you checked the LTM logs for anything that may pop out as a related error?
- Brad_Parker
Cirrus
your capture only shows one side of the conversation. If the LTM is resetting the connection you would see syn, syn, syn, before a reset is sent with the profile you provided. Try and capture the conversation on both ingress and egress vlans.
- Brad_Parker
Cirrus
It is definitely not normal for the LTM to send on SYN followed by a RST less than 200ms later. Using tcpdump on the LTM enable max vebose using "tcpdump -s0 -ni (server vlan):nnn host 172.16.34.101 and host 172.16.33.1 -w /tmp/reset.pcap" where (server vlan) is the name of the vlan used to connect to the pool member. The :nnn should give us a RST cause in the RST packet. You can view this in wireshark using the F5 plugin for wireshark. https://devcentral.f5.com/articles/getting-started-with-the-f5-wireshark-plugin-on-windows - Brad_Parker
Cirrus
Also, have you checked the LTM logs for anything that may pop out as a related error?
- teknet7_237497
Nimbostratus
Hi Brad,
OK. The packets from the client (172.16.33.1) to VIP (172.16.33.103) captured on the client:
21:33:01.361864 IP 172.16.33.1.59917 > 172.16.33.103.80: Flags [S] 21:33:01.363374 IP 172.16.33.103.80 > 172.16.33.1.59917: Flags [S.] 21:33:01.363407 IP 172.16.33.1.59917 > 172.16.33.103.80: Flags [.] 21:33:01.363856 IP 172.16.33.1.59917 > 172.16.33.103.80: Flags [P.] len 391 (GET) 21:33:01.364272 IP 172.16.33.103.80 > 172.16.33.1.59917: Flags [.] len 0The packets from VIP to real server (172.16.34.101) captured on real server. I can see F5 tried 4 times to establish that connection):
21:32:09.770673 IP 172.16.33.1.59917 > 172.16.34.101.80: Flags [S] 21:32:09.770813 IP 172.16.33.1.59917 > 172.16.34.101.80: Flags [R] 21:32:09.770814 IP 172.16.34.101.80 > 172.16.33.1.59917: Flags [S.] 21:32:11.769808 IP 172.16.33.1.59917 > 172.16.34.101.80: Flags [S] 21:32:11.769934 IP 172.16.33.1.59917 > 172.16.34.101.80: Flags [R] 21:32:11.769937 IP 172.16.34.101.80 > 172.16.33.1.59917: Flags [S.] 21:32:13.769422 IP 172.16.33.1.59917 > 172.16.34.101.80: Flags [S] 21:32:13.769667 IP 172.16.33.1.59917 > 172.16.34.101.80: Flags [R] 21:32:13.769673 IP 172.16.34.101.80 > 172.16.33.1.59917: Flags [S.] 21:32:15.769861 IP 172.16.33.1.59917 > 172.16.34.101.80: Flags [S] 21:32:15.770155 IP 172.16.33.1.59917 > 172.16.34.101.80: Flags [R] 21:32:15.770162 IP 172.16.34.101.80 > 172.16.33.1.59917: Flags [S.]I had to remove additional options presented by tcpdump because of antispam filters of devcentral. We can see that from the client side i was able to send HTTP GET request, but got just empty ACK (no HTTP response with payload). On the server side we do see F5 prematurely sending RST packet and because of that connection is never established.
I have around 60s time difference on client/server machine.
Does that looks like any known issue ? Or maybe my misconfiguration (i do have experience with loadbalancers but not with F5 so i am newbie).
Thanks, Michal
- Carl_Brothers
Employee
So on the topic of TCP Reset behavior, there are many reasons. Check out SOL9812
Have you used ihealth yet? It can let you know a wide number of issues as well like common config errors.
As this is a VE, what resources do you have allocated to it, cpu and memory specifically.
If this is on a system under a valid support contract, Opening a support case might be your quickest path to resolution.
As you are new to F5, we do have a good deal of free online training available here.
- teknet7_237497
Nimbostratus
Hi Carl,
Thank you for the link. Please notice that almost all the reasons are resulting in TCP RST send on the client side - not server side like in my scenario. I have increased memory to 8GB and CPU to 4 on my VM (had 4GB + 2CPU which still should be fine for basic deployment) but the issue is still there.
Also if that would be any configuration limits i would not see counters increasing for Abandoned sessions but rather Not Accepted or Failed i guess.
I will try ihealth first, then case.
Thanks, Michal
Help guide the future of your DevCentral Community!
What tools do you use to collaborate? (1min - anonymous)Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
