Define: inet port exhaustion?

Funkdaddy: In the event it helps (I know I'm coming in late here...), here's some extra information on the setup here. Connections are defined by the source IP:port and destination IP:port combination. The most common situation is that port exhaustion happens on the source side, when they're headed for a common port. So it doesn't surprise me at all that you saw this during your load tests - you probably had a single or small number of devices hammering as many connections as possible. Once you go above 65k connections you'll run out of ports...

 

 

One way to help offset this and get better test data fidelity is to reduce TIME_WAIT or enable time wait recycle on the source side of the conversation. This will recycle sockets in TIME_WAIT, which is fine to do if you're doing load testing in a controlled environment. The basic idea here is to free up those source ports asap so testing can continue.

 

 

Tech note: from a TCP perspective, whoever sends the first FIN of a teardown will go into TIME_WAIT. Less frequently you'll see a 'simulataneous close' where both sides send a FIN at the same time, and both go into TIME_WAIT. It's common that this interval is too long and it can be lower than the defaults (2*max. segment lifetime). So what happens here is that the source port will be sitting there doing nothing for the TIME_WAIT interval, even though it really could be re-used for your load testing.

 

 

-- Matt