TCP KeepAlive - Avoid long client connection without data

Any idea why the client is sending the keepalives, it's not typical behavior.

No idea why I have client that use TCP keepalive... It can be a system setting (linux allow to define this at system level), or firewall setting (like cisco "dead peer detection" which send TCP keepalive)...

 

I've written a really small script which reproduce http request and then send tcp keepalive, it can keep tcp session opened for many days...

 

Commonly, public website usually send a "TCP RST" between 1 minute to 30 minutes after the last HTTP response....

 

Have you used that script with a public website to prove that they are somehow doing something different? Can you list your VS and TCP profile settings here (suitably anonymised?)

Yes I've used my script the other public website, and for all I've tested, send a RST after a timeout even if tcp keepalive are sent.

Here is the very simple configuration:

ltm profile tcp /Common/tcptest {
  app-service none
  close-wait-timeout 5
  defaults-from /Common/tcp
  fin-wait-timeout 5
  idle-timeout 360
  keep-alive-interval 4294967295
  max-retrans 8
  receive-window-size 65535
  send-buffer-size 65535
  syn-max-retrans 3
  time-wait-timeout 2000
  zero-window-timeout 20000
}

ltm profile http /Common/http-conf1 {
  app-service none
  defaults-from /Common/http
  insert-xforwarded-for disabled
  max-requests 200
}

ltm virtual /Common/VS {
  destination /Common/X.X.X.X:80
  ip-protocol tcp
  mask 255.255.255.255
  pool /Common/POOL1
  profiles {
    /Common/http-conf1 { }
    /Common/oneconnect { }
    /Common/tcptest { }
  }
  snat automap
  translate-address enabled
  translate-port enabled
  vlans-disabled
}

ltm virtual-address /Common/X.X.X.X {
  address X.X.X.X
  mask 255.255.255.255
  traffic-group none
}

I can't test unfortunately but as you are using OneConnect (and perhaps even if not) surely all the client-side connections are not duplicated server-side? What TCP profile is used server-side? Does the keepalive keep the server-side connection open also? I doubt it. If you could test/check that would be great. If it doesn't we could use the SERVER_CLOSED iRule event to terminate the client-side connection but possibly we'll have to disable OneConnect to use if effectively.

@Erlo

 

 

You'd have to use FastHTTP for idle timeouts... Not that I've tested this explicitly, but the docs say time with no traffic... So that SHOULD mean TCP level keepalives don't count..

 

 

H

@Steeve:

 

 

- The TCP profile on server side is the same

 

- Without oneconnect, when server wants to close connection (sending TCP FIN), the BigIP transmit the FIN to the client but agin never care about the "TIME_WAIT" status, but I want to use oneconnect feature...

 

 

Hamish:

 

 

I agree, the doc is talking about "no traffic", but actually the problem is exactly the same. On every tcp keepalive, the idle value is reset (tmsh show sys connection all-properties)

 

 

Here is a small code I use to check the keepalive behaviour. Can you please check on your side with a VS, and tell me if the connection can stay indefinitly ?

 

 

!/usr/bin/env python

import socket
import sys
import time

HOST = 'X.X.X.X'
GET = '/toto'
PORT = 80
 
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
 
if 1:
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
    sock.setsockopt(socket.SOL_TCP, socket.TCP_KEEPIDLE, 2) 
    sock.setsockopt(socket.SOL_TCP, socket.TCP_KEEPINTVL, 5)
    
sock.connect((HOST, PORT))
 
sock.send("GET %s HTTP/1.1\r\nHost: %s\r\nConnection: Keep-Alive\r\n\r\n" % (GET, HOST))
 
while 1:
    try:
        data = sock.recv(1024)
        if len(data):
            print data
        else:
            break
    except: 
        break
        
print "Socket end: Possibly TCP FIN received - TCP state should be TIME_WAIT"

while 1:
    time.sleep(10)

The Idle Timeout in the FastHTTP profile is for TCP, not HTTP unfortunately.

 

 

I'm out of ideas now. The real issue here is that a) there are a number of poorly configured clients out there and b) not honouring the keepalives would violate the TCP/IP standard really.

Check me out Hamish, standing up for standards! =]

I'm not convinced it's actually poorly configured clients...

 

 

I suspect what he's seeing is poorly written servers. It's not uncommon for 'lazy' developers to have a very long running backend procedure that they expect a client to sit & spin for. I've seen 30 minutes quoted from developers as being required... Most servers (Apache, IIS etc) will actually close the connection on the HTTP idle timeout (Default is usually 5 seconds). However the HTTP idle timeout doesn't apply for a client waiting on a reply from the server... It's typically only for the CLIENT sitting idle and the SERVER waiting for a request (I must check the RFC on that one for the wording... It's the way I remember it anyway :)

 

 

I'm not sure a client is expected to do the idle timeout. or if it is, it's only a SHOULD... Not a MUST. The server on the other hand SHOULD, because otherwise it's opening itself up for a pretty easy DOS attack... Long running replies just make that a bit easier...

 

 

H