Hello,
I’m trying to monitor the embryonic/half-open connections for a particular VS on my F5 load balancer using SNMP.
I’ve read several articles, including ‘K00560557: Using SNMP information related to SynCookie Protection for monitoring potential Syn Flood attack or Unexpected Traffic spikes’ and ‘K25162232: How to get the Virtual Server state using SNMP polling.’ but I did not find how to do it.
Could you please guide me on how to retrieve this information?
Regards,
Hi Fluzocapacitor,
Keeping server side connection open even after the client side connection already been closed is a KNOWN BUG which was fixed in 11.6.x
I dont think there are SNMP MIBs for monitoring half open connections, but you can check in Pools there is a TCP Half Open monitor that can be applied.
The f5networks.f5_modules.bigip_monitor_tcp_half_open module can be used to manage F5 BIG-IP LTM TCP half-open monitors. This module is part of the f5networks.f5_modules collection version 1.27.1 and requires BIG-IP software version 12 or higher. Some parameters of the module include:
You can use the following commands to view and delete the current or active connections
Examples
tmsh show /sys connection
TMSH command to list all the options & properties with the show /sys connection command
tmsh show /sys connection ?
Options:
all-properties Display all properties for the specified items
default Units are determined based on current values
exa All values are displayed in exa-units
gig All values are displayed in giga-units
kil All values are displayed in kilo-units
meg All values are displayed in mega-units
peta All values are displayed in peta-units
raw No conversions are made to any values
save-to-file Output from the command is saved to the specified file. -
This file is placed in /shared. This allows to write a file larger than 2GB.
tera All values are displayed in tera-units
yotta All values are displayed in yotta-units
zetta All values are displayed in zetta-units
| Route command output to a filter
Properties:
“{” Optional delimiter
age Specifies the age, in seconds, of a connection
cs-client-addr Specifies the clientside remote address of the active connections
cs-client-port Specifies the clientside remote port of the active connections
cs-server-addr Specifies the clientside local address of the active connections
cs-server-port Specifies the clientside local port of the active connections
protocol Specifies the protocol used for specified connections (for example: tcp, udp)
ss-client-addr Specifies the serverside local address of the active connections
ss-client-port Specifies the serverside local port of the active connections
ss-server-addr Specifies the serverside remote address of the active connections
ss-server-port Specifies the serverside remote port of the active connections
Using my example output line above we will label each part of the output:
Client Side
172.10.50.20:30322 10.10.10.1:443
cs-client-addr:cs-client-port cs-server-addr:cs-server-port
Server Side
192.168.1.1:30322 192.40.200.20:30322
ss-client-addr:ss-client-port ss-server-addr:ss-server-port
The protocol used, the age of the connection, and which traffic management microkernel was used
tcp 14 (tmm: 6) none
protocol age tmm used
How to delete a connection from the F5 BIG-IP connection table?
To delete active connections in the BIG-IP connection table you can key on any of the property value(s). For example, if I wanted to delete all connections a user was initiating to the BIG-IP you would specify the users IP as the client side client address like this:
TMSH command to delete all connections initiated from a specific client IP
tmsh delete /sys connection cs-client-addr 172.10.50.20
TMSH command to delete ALL connections in a F5 BIG-IP - Careful, you’re going to kill EVERY session.
tmsh delete /sys connection
Make sure you check out the “all-properties” option, it has a load of great information, including how many bits the connection has pushed, and its’ age & idle-timeout. Lets take a look at the output:
TMSH command to show all details about a particular conneciton, including how much traffic, bits in and out, a connection has used
tmsh show /sys connection cs-client-addr 172-10.50.20 all-properties
172.10.50.20:30322 - 10.10.10.1:443 - 172.10.50.20:30322 - 192.40.200.20:443
----------------------------------------------------------------------------
TMM 6
Type self
Acceleration none
Protocol tcp
Idle Time 9
Idle Timeout 300
Unit ID 0
Lasthop /Common/internal 00:0d:3a:3a:25:20
Virtual Path 10.10.10.1:443
Conn Id 0
ClientSide ServerSide
Client Addr 172.10.50.20:30322 172.10.50.20:30322
Server Addr 10.10.10.1:443 192.40.200.20:443
Bits In 3.0K 1.3K
Bits Out 1.3K 3.0K
Packets In 4 3
Packets Out 3 4
HTH
Hi,
Thank you for the detailed information and the helpful links.
It’s good to know about the known bug and its fix in version 11.6.x. However, I am not affected by bug K15973 as I am running version 15.1.10.
I appreciate the suggestion to use the TCP Half Open monitor. I understand that this monitor would help in monitoring the connections between the virtual server and the pool nodes. However, my main interest is in monitoring the connections between the clients and the virtual server. A SYN flood attack would cause issues on the load balancer itself, not on the backend nodes.
I also appreciate the commands to view and delete active connections. I have used the following command to check the number of connections:
show /sys connection cs-server-addr 10.1.50.200%1
Sys::Connections
10.1.34.120%1:51080 10.1.50.200%1:80 10.1.34.120%1:51080 10.2.75.150%1:80 tcp 16 (tmm: 0) server-side(none) none
10.1.58.99%1:61766 10.1.50.200%1:80 10.1.58.99%1:61766 10.2.92.180%1:80 tcp 28 (tmm: 0) none none
10.2.85.43%1:51172 10.1.50.200%1:80 10.2.85.43%1:51172 10.2.92.180%1:80 tcp 39 (tmm: 0) both-sides(none;none) none
...
10.1.123.45%1:59943 10.1.50.200%1:80 10.1.123.45%1:59943 10.2.75.150%1:80 tcp 2 (tmm: 1) none none
10.1.87.101%1:55831 10.1.50.200%1:80 10.1.87.101%1:55831 10.2.92.180%1:80 tcp 6 (tmm: 1) none none
Total records returned: 1000
Although there are 1000 established connections, my main interest is in half-open connections, as these are the ones that trigger the SYN cookie mechanism. According to the documentation, these half-open connections appear as “Current SYN Cache” in the output of the show ltm virtual <virtual> command.
Just to clarify, is there a specific OID or SNMP method to directly monitor the number of half-open connections between the clients and the virtual server, or would the TCP Half Open monitor still be the best approach to track this metric?
Thank you again for your assistance!
Best regards,
Hi Fluzocapacitor,
Please see the specific OID for your half-open connections
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.13 | ltmVirtualServStatEphemeralPktsIn | 0 | 0 | The number of packets received ephemeral by the specified virtual server. |
|---|---|---|---|---|
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.14 | ltmVirtualServStatEphemeralBytesIn | 0 | 0 | The number of bytes received ephemeral by the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.15 | ltmVirtualServStatEphemeralPktsOut | 0 | 0 | The number of packets transmitted ephemeral from the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.16 | ltmVirtualServStatEphemeralBytesOut | 0 | 0 | The number of bytes transmitted ephemeral from the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.17 | ltmVirtualServStatEphemeralMaxConns | 0 | 0 | The maximum ephemeral connections to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.18 | ltmVirtualServStatEphemeralTotConns | 0 | 0 | The total ephemeral connections to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.19 | ltmVirtualServStatEphemeralCurConns | 0 | 0 | The current ephemeral connections to the specified virtual server. |
Check a reference for VIP OID 1.3.6.1.4.1.3375.2.2.10.2.3.1.1
http://oidref.com/1.3.6.1.4.1.3375.2.2.10.2.3.1.1
| OID | Name | Sub children | Sub Nodes Total | Description |
|---|---|---|---|---|
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.2 | ltmVirtualServStatCsMinConnDur | 0 | 0 | The minimum connection duration in milliseconds among all connections through the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.3 | ltmVirtualServStatCsMaxConnDur | 0 | 0 | The maximum connection duration in milliseconds among all connections through the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.4 | ltmVirtualServStatCsMeanConnDur | 0 | 0 | The average connection duration in milliseconds for all connections through the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.5 | ltmVirtualServStatNoNodesErrors | 0 | 0 | The number of no-node errors. If active server node is zero, this counter is incremented. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.6 | ltmVirtualServStatClientPktsIn | 0 | 0 | The number of packets received by the specified virtual server from client-side. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.7 | ltmVirtualServStatClientBytesIn | 0 | 0 | The number of bytes received by the specified virtual server from client-side. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.8 | ltmVirtualServStatClientPktsOut | 0 | 0 | The number of packets sent to client-side from the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.9 | ltmVirtualServStatClientBytesOut | 0 | 0 | The number of bytes sent to client-side from the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.10 | ltmVirtualServStatClientMaxConns | 0 | 0 | The maximum connections from client-side to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.11 | ltmVirtualServStatClientTotConns | 0 | 0 | The total connections from client-side to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.12 | ltmVirtualServStatClientCurConns | 0 | 0 | The current connections from client-side to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.13 | ltmVirtualServStatEphemeralPktsIn | 0 | 0 | The number of packets received ephemeral by the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.14 | ltmVirtualServStatEphemeralBytesIn | 0 | 0 | The number of bytes received ephemeral by the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.15 | ltmVirtualServStatEphemeralPktsOut | 0 | 0 | The number of packets transmitted ephemeral from the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.16 | ltmVirtualServStatEphemeralBytesOut | 0 | 0 | The number of bytes transmitted ephemeral from the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.17 | ltmVirtualServStatEphemeralMaxConns | 0 | 0 | The maximum ephemeral connections to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.18 | ltmVirtualServStatEphemeralTotConns | 0 | 0 | The total ephemeral connections to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.19 | ltmVirtualServStatEphemeralCurConns | 0 | 0 | The current ephemeral connections to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.20 | ltmVirtualServStatPvaPktsIn | 0 | 0 | The total number of hardware accelerated packets received by the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.21 | ltmVirtualServStatPvaBytesIn | 0 | 0 | The total number of hardware accelerated bytes received by the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.22 | ltmVirtualServStatPvaPktsOut | 0 | 0 | The total number of hardware accelerated packets sent out from the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.23 | ltmVirtualServStatPvaBytesOut | 0 | 0 | The total number of hardware accelerated bytes sent out from the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.24 | ltmVirtualServStatPvaMaxConns | 0 | 0 | The maximum hardware accelerated connections to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.25 | ltmVirtualServStatPvaTotConns | 0 | 0 | The total hardware accelerated connections to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.26 | ltmVirtualServStatPvaCurConns | 0 | 0 | The current hardware accelerated connections to the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.27 | ltmVirtualServStatTotRequests | 0 | 0 | The total number of requests going through the specified virtual server. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.28 | ltmVirtualServStatTotPvaAssistConn | 0 | 0 | The total number of connections to the specified virtual server, which are partially hardware accelerated. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.29 | ltmVirtualServStatCurrPvaAssistConn | 0 | 0 | The current number of connections to the specified virtual server, which are partially hardware accelerated. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.30 | ltmVirtualServStatCycleCount | 0 | 0 | CPU cycle count since last system restart. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.31 | ltmVirtualServStatVsUsageRatio5s | 0 | 0 | The percentage of time Virtual Server was busy over the last 5 seconds. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.32 | ltmVirtualServStatVsUsageRatio1m | 0 | 0 | The percentage of time Virtual Server was busy over the last 1 minute. |
| 1.3.6.1.4.1.3375.2.2.10.2.3.1.33 | ltmVirtualServStatVsUsageRatio5m | 0 | 0 | The percentage of time Virtual Server was busy over the last 5 minutes. |
Please mark it as SOLUTION if your query is answered as it helps others to find a solution as well for similar query.
Please let me know for any further discussion , i would be glad to assist you on F5 related topics.
HTH
Hi,
I do apologise but I do not know what Ephemeral conections are and whether they have something to do with half-open ones.
‘Ephemeral’ in SNMP MIB descriptions | DevCentral
Regards,
@fluzocapacitor - did you ever finally resolve this issue? If you found a solution it would be great to reply with that (and mark it as a solution). If not…did you find a workaround? or engage with F5 Support?
cheers.