Forum Discussion

Ryan_34424's avatar
Ryan_34424
Icon for Altostratus rankAltostratus
Apr 12, 2018

LTM :: Zero Window Server Side :: TCP Profiles

We have a virtual server setup for our receiving mail system, and it has been configured as-is for quite some time (measured in years). Never before has an issue arisen, but recently a particular client has been having problems sending attachments to us (and as far as we are aware, ONLY that client). What they claim to see is that the connection is terminated. Normal email works fine. Small file attachments work fine. However when they send us attachments that are Mb in size, the connection will not be successful.

On our side, we see the window size slowly creep down until it hits zero. The BIG-IP probes the mail system, the mail system acks the probe, but keeps the window size at zero. It does this until the zero window timeout is reached on the BIG-IP and the connection is terminated by the BIG-IP (TCP RST).

This is what the window decrease looks like on the client side (tcp.stream eq 3 and ip.src eq [the mail system]):

This is what the window decrease looks like on the server side (tcp.stream eq 2 and ip.src eq [the VIP]):

Client side end of the connection:

Server side end of the connection:

My impression initially was that this is not a BIG-IP problem... but when we remove the BIG-IP from the path, the connection works fine regardless of attachment size. Again, works fine for everyone else as far as we know regardless of if the BIG-IP is in the path... which is perplexing.

Things I've tried:

* Switching-out TCP profiles (lan optimized, wan optimized, client and server matching and different in combinations of the above). Now on mptcp-mobile-optimized with defaults.

* Moving TLS off of the F5

* Resetting TLS profile to defaults

* Different mail systems (of same type/configuration)

Current configuration:

* VIP on port 25

* TCP profile with mptcp-mobile-optimized w/defaults

* SSL Profile (defaults w/cert, optional SSL, allowed cipher suites)

* SMTPS Profile (allows TLS)

* Pool w/single mail system

* iRule w/VIP bounceback

* Source IP Persistence

VIP bounceback iRule: 
when LB_SELECTED {  
    if {[IP::addr "[IP::client_addr]/24" equals "[LB::server addr]/24"]} {  
        snat automap 
    } else {
        snat none
    }
}

Any ideas/thoughts/suggestions all welcome.

Thanks for taking the time.

application delivery
BIG-IP
flow control
LTM
profiles
tcp
tcp zero window timeout
zero window
  • Ryan77777's avatar
    Ryan77777
    Icon for Altocumulus rankAltocumulus
    Apr 12, 2018

    Reply from Barracuda:

     

    Looks like this is pretty much ubiquitous on all devices that are under reasonably large load in the real world. All devices that I've checked that are within 60% or so of their max load show hundreds to thousands of these zero window packets being sent from the ESG to other mail servers. This is giving the SMTP service sufficient time to process the data lines as they are sent. This behavior goes all the way back to 6x firmware and is not limited to virtual boxes but applies to physical devices as well. Primary issue for this customer then was the low timeout setting on his load balancer for zerowindow issues. With that timeout set higher, we are now seeing very few to no connections dropped for this reason.

     

    Solution:

     

     

    I did the above on the server side, and left the client side default (20s).

     

    Hopefully this helps somebody else!