Define: inet port exhaustion?

Sorry the first post didn't appear to have taken...

 

 

During various load tests within our environment, I'm seeing the following errors in our logs:

 

 

bigip1 local/tmm crit tmm[5406]: 01010201:2: Inet port exhaustion on 199.83.xx.xx to 10.2.1.140:7304 (proto 6)

 

 

I'm having little luck actually getting a definition of what it means - I assumed it meant number of connections are maxed out on a port...

 

 

Could someone please help with:

 

* General definition of port exhaustion

 

* Is there a known limit? Or, depends on hardware?

 

* Is the limit, per Port/per VIP? Can one port on one VIP being exhausted cause issues with other VIPs or the LTM in general?

 

 

Thanks so much,

 

-Funkdaddy

There's a SOL on AskF5 about this:

 

http://support.f5.com/kb/en-us/solutions/public/7000/800/sol7820.html?sr=15280517exhaustion

 

 

The simple fix is to add more addresses to the SNAT pool and/or create a unique SNAT pool per LB pool. If you need to ensure the same client is SNAT'd to the same SNAT address over connections you can use an iRule:

 

 

http://devcentral.f5.com/wiki/default.aspx/iRules/snat_pool_persistence.html

 

 

Aaron

FYI this is not for a SNAT, just a VIP with a pool of servers that were being load tested.

 

 

Here's the response to a my inquiry to F5 Support about being concerned that this exhaustion might impact our production VIPs:

 

 

"If you are not seeing this error in conjunction with a SNAT, you won't see an impact on the production traffic. You are correct, the number of ports per node is exhausted and shouldn't affect the other nodes, given that your test and production nodes are not the same."

Funkdaddy: In the event it helps (I know I'm coming in late here...), here's some extra information on the setup here. Connections are defined by the source IP:port and destination IP:port combination. The most common situation is that port exhaustion happens on the source side, when they're headed for a common port. So it doesn't surprise me at all that you saw this during your load tests - you probably had a single or small number of devices hammering as many connections as possible. Once you go above 65k connections you'll run out of ports...

 

 

One way to help offset this and get better test data fidelity is to reduce TIME_WAIT or enable time wait recycle on the source side of the conversation. This will recycle sockets in TIME_WAIT, which is fine to do if you're doing load testing in a controlled environment. The basic idea here is to free up those source ports asap so testing can continue.

 

 

Tech note: from a TCP perspective, whoever sends the first FIN of a teardown will go into TIME_WAIT. Less frequently you'll see a 'simulataneous close' where both sides send a FIN at the same time, and both go into TIME_WAIT. It's common that this interval is too long and it can be lower than the defaults (2*max. segment lifetime). So what happens here is that the source port will be sitting there doing nothing for the TIME_WAIT interval, even though it really could be re-used for your load testing.

 

 

-- Matt

Matt, just looked at my old post and noticed your response. Thank you very much, that's helpful!

 

 

 

 

pls what is the mean of port exhausion in term of DNS poisioning

 

pls what is the mean of port exhausion in term of DNS poisioning

 

pls what is the mean of port exhausion in term of DNS poisioning

 

pls what is the mean of port exhausion in term of DNS poisioning

 

pls what is the mean of port exhausion in term of DNS poisioning