Forum Discussion
er_sandy_27437
Nimbostratus
NimbostratusTCP connection resets on LTM
virtual citrixappsremote{
snat automap
pool pool_citrixappsremote
destination 192.168.16.68:https
ip protocol tcp
profiles {
citrixapsremote {
clientside
}
http {}
tcp {}
}
}
pool pool_citrixappsremote {
lb method member least conn
monitor all http
members {
192.168.16.25:http {}
192.168.16.100:http {}
}
}
Executing ''curl" is successful and i can see complete SSL handshake taking place on the device. But the external user is not able to access the site.
Here is the output from TCP dump
06:07:45.514632 IP plkfintjmp01v.corporate. .53553 > citrixappsremote. .http: S 3182411697:3182411697(0) win 64512
06:07:45.514644 IP citrixappsremote. .http > plkfintjmp01v.corporate. .53553: R 0:0(0) ack 3182411698 win 0
06:07:46.012328 IP plkfintjmp01v.corporate. .53553 > citrixappsremote. .http: S 3046774539:3046774539(0) win 64512
06:07:46.012339 IP citrixappsremote. .http > plkfintjmp01v.corporate. .53553: R 0:0(0) ack 4159330139 win 0
06:07:46.414747 IP plkfintjmp01v.corporate. .53553 > citrixappsremote. .http: S 191358774:191358774(0) win 64512
06:07:46.414758 IP citrixappsremote. .http > plkfintjmp01v.corporate. .53553: R 0:0(0) ack 1303914374 win 0
06:07:50.809118 IP plkfintjmp01v.corporate. .53567 > citrixappsremote. .http: S 2623997703:2623997703(0) win 64512
06:07:50.809130 IP citrixappsremote. .http > plkfintjmp01v.corporate. .53567: R 0:0(0) ack 2623997704 win 0
06:07:51.242950 IP plkfintjmp01v.corporate. .53567 > citrixappsremote. .http: S 3133539186:3133539186(0) win 64512
06:07:51.242975 IP citrixappsremote. .http > plkfintjmp01v.corporate. .53567: R 0:0(0) ack 509541484 win 0
hooleylistCirrostratus
Hi,
What's the client that is failing? Is it a browser or something else?
Is your tcpdump showing serverside connections to a pool member? If so, it looks like the server is resetting the connection attempts from LTM. If that's showing the clientside traffic, why is the client connecting to LTM via HTTP? Your virtual server is defined on 443 for HTTPS.
Can you try running a tcpdump looking for both the client and serverside connections?
tcpdump -nni 0.0 host 192.168.16.68 or host 192.168.16.25 or host 192.168.16.100
Aaron
smp_86112hoolio asks good, valid questions. If it were me, I would also try and telnet from the LTM to the pool members on port 80, to make sure they are indeed accepting connections, and that there's not something like a firewall blocking the connection. If you can't make a connection from the LTM to the pool member, then obviously the client connection will fail.
er_sandy_27437Hello Hoolio,
Actually its the F5 box thats sending the RST packets. We created another VIP to check if there's any Certificate issue. But in both the cases issue is the same.
Curl -Gv https://192.168.16.68 --insecure this script shows perfect output from the VIP. I dont know why VIP is not responding to the internal request. Right now we have Bypassed LTM and traffic is accessible from the SERVER 192.168.16.100.- smp_86112I am a bit confused by the evidence you are providing. One one hand, you claim you are using curl to connect to an HTTPS VIP. On the other hand, the trace you provided shows a connection to port 80 - not port 443. That's why hoolio asked which side of the ltm was the trace you provided taken on - the external (i.e. client) side, or the internal (i.e. server) side? If the trace was taken on the client side, then of course the LTM sent a RST because the connection from the client was HTTP (port 80), not HTTPS (port 443), and you probably don't have an HTTP (port 80) VIP. But if the trace was taken on the server side, then it was the Pool Member who is sending a TCP RST - not the LTM. That was why I advised you to attempt to telnet to the pool member on port 80 from the LTM - it will validate whether or not a TCP connection with the Pool Member is being acknowledged.
So something isn't adding up. Which side of the LTM was the trace taken on - the client side or the pool member side? - er_sandy_27437IT was taken from the client side. I am uploading another set of capture taken for the https, still the issue was the same.
05:54:00.689292 IP 172.29.26.104.35506 > 192.168.16.68.https: S 2924856544:2924856544(0) win 64512 in slot1/tmm1 lis= flowtype=0 flowid=0 peerid=0 conflags=0 inslot=0 inport=512 haunit=0 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:00.689304 IP 192.168.16.68.https > 172.29.26.104.35506: R 0:0(0) ack 2924856545 win 0 out slot1/tmm1 lis= flowtype=134 flowid=B77FB40 peerid=0 conflags=20 inslot=0 inport=0 haunit=1 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:01.154293 IP 172.29.26.104.35506 > 192.168.16.68.https: S 1901533113:1901533113(0) win 64512 in slot1/tmm1 lis= flowtype=0 flowid=0 peerid=0 conflags=0 inslot=0 inport=512 haunit=0 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:01.154310 IP 192.168.16.68.https > 172.29.26.104.35506: R 0:0(0) ack 3271643866 win 0 out slot1/tmm1 lis= flowtype=134 flowid=BBB9980 peerid=0 conflags=20 inslot=0 inport=0 haunit=1 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:01.657424 IP 172.29.26.104.35506 > 192.168.16.68.https: S 3631880207:3631880207(0) win 64512 in slot1/tmm1 lis= flowtype=0 flowid=0 peerid=0 conflags=0 inslot=0 inport=512 haunit=0 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:01.657439 IP 192.168.16.68.https > 172.29.26.104.35506: R 0:0(0) ack 707023664 win 0 out slot1/tmm1 lis= flowtype=134 flowid=BBFCB40 peerid=0 conflags=20 inslot=0 inport=0 haunit=1 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:18.857582 IP 172.29.26.104.35554 > 192.168.16.68.https: S 2688980484:2688980484(0) win 64512 in slot1/tmm1 lis= flowtype=0 flowid=0 peerid=0 conflags=0 inslot=0 inport=512 haunit=0 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:18.857596 IP 192.168.16.68.https > 172.29.26.104.35554: R 0:0(0) ack 2688980485 win 0 out slot1/tmm1 lis= flowtype=134 flowid=BD29140 peerid=0 conflags=20 inslot=0 inport=0 haunit=1 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:19.361010 IP 172.29.26.104.35554 > 192.168.16.68.https: S 3986519246:3986519246(0) win 64512 in slot1/tmm1 lis= flowtype=0 flowid=0 peerid=0 conflags=0 inslot=0 inport=512 haunit=0 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:19.361023 IP 192.168.16.68.https > 172.29.26.104.35554: R 0:0(0) ack 1297538763 win 0 out slot1/tmm1 lis= flowtype=134 flowid=BC22540 peerid=0 conflags=20 inslot=0 inport=0 haunit=1 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:19.864088 IP 172.29.26.104.35554 > 192.168.16.68.https: S 2532989569:2532989569(0) win 64512 in slot1/tmm1 lis= flowtype=0 flowid=0 peerid=0 conflags=0 inslot=0 inport=512 haunit=0 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0
05:54:19.864103 IP 192.168.16.68.https > 172.29.26.104.35554: R 0:0(0) ack 4138976382 win 0 out slot1/tmm1 lis= flowtype=134 flowid=BC4E980 peerid=0 conflags=20 inslot=0 inport=0 haunit=1 peerremote=00000000:00000000:00000000:00000000 peerlocal=00000000:00000000:00000000:00000000 remoteport=0 localport=0 proto=0 vlan=0 - smp_86112
OK, now that makes sense. So the LTM is sending a RST to the client because the Pool Member is sending a RST to the LTM. Think of what happens on both sides of the LTM.
1) Client initiates a TCP connection with the LTM on port 443 on the external side.
2) LTM makes a TCP connection with the Pool Member on port 80 on the internal side.
3) Pool Member sends a TCP RST to the LTM on the internal side.
4) LTM sends a TCP RST to the client on the external side, because the Pool Member refused a connection.
If you take a trace of both sides simultaneously and compare the timestamps closely, this is exactly what you should find.Again, I advise you to try and make a TCP connection with your Pool Member from the LTM with telnet (telnet 80). I bet you get a TCP RST.
- er_sandy_27437Strangely this problem resolved by changing the IP of the VIP. While they were in same subnett it was not working but putting them in seperate routed subnet solved the issue. Can somebody have an explaination to that?