Forum Discussion
bchick2_8645
Nov 21, 2011Nimbostratus
Limit Client Connections with Table
We have been using (verbatim) the iRule given on the Wiki at Click Here to successfully limit the number of concurrent connections from a single source IP address. However, we recently learned the use of global variables in the rule was demoting our LTM 1600 out of CMP mode. Some research seemed to indicate that replacing the global array with the new (at least new to me) table command was the right approach. I took a first shot at that and it seemed to be working fine at first but recently we started receiving log messages as follows:
Unable to resume pending rule event CLIENT_CLOSED on closed flow XXX.XXX.XXX.XXX:XXXX->YYY.YYY.YYY.YYY:443
Where the X's are the client source IP address / port and the Y's are the server IP. This wasn't happening at first but started over time. Didn't notice it before the upgrade from 10.2.2 HF3 to 10.2.3 but won't swear that it didn't start until then. Searching devcentral and the support site I can't find any hits on that particular log entry. Can anybody tell me what I might be doing wrong? I'm pasting the text of the iRule below, any help confirming the validity of the iRule or pointers for improving it would be greatly appreciated.
Thanks in advance.
when RULE_INIT {
The maximum number of TCP connections to the virtual server from a single client IP address
set static::max_connections_per_ip 1000
Table name (will later be appended with VS name)
set static::tbl "vsratelimit"
}
when CLIENT_ACCEPTED {
If client IP is exempted skip the rest of the rule and return
if {[class match [IP::client_addr] equals concurrent_sessions_exemptions]} {
return
}
Set the key to the client IP
set key "[IP::client_addr]"
Append the VS name to the end of the table name
set tbl ${static::tbl}_[virtual name]
If the IP is already in the table increment the count
if {[table keys -subtable $tbl] contains $key} {
set value [table lookup -subtable $tbl $key]
Check if over the max connection limit and if so reject
if {$value >= $static::max_connections_per_ip} {
log local0. "Max connections exceeded for IP: $key, rejecting connection"
reject
} else {
Increment the connection count
table incr -subtable $tbl $key
Only for debug:
log local0. "Count for $key incremented to $value"
}
} else {
IP is not already in the table so add it with count 1
table set -subtable $tbl $key 1 indef
log local0. "Key added to table for $key"
}
}
when CLIENT_CLOSED {
If client IP is exempted skip the rest of the rule and return
if {[class match [IP::client_addr] equals concurrent_sessions_exemptions]} {
return
}
Set the key to the client IP
set key "[IP::client_addr]"
Append the VS name to the end of the table name
set tbl ${static::tbl}_[virtual name]
Check if the client has a count in the table
if {[table keys -subtable $tbl] contains $key} {
Decrement the count by 1
table incr -subtable $tbl $key -1
Check if the count is 0 or negative
set value [table lookup -subtable $tbl $key]
log local0. "Count for $key decremented to $value"
if {$value <= 0} {
Clear the table entry
table delete -subtable $tbl $key
log local0. "Table entry deleted for $key"
}
}
}
- nitassEmployeeare you able to get packet trace while log is happening?
- bchick2_8645NimbostratusI'll work on getting that. The problems are easiest to replicate when we're under heavy load from simulated load testing which makes catching just the relevant traffic kind of tricky since there's so much. I'll see what I can do.
- hooleylistCirrostratusI thought you'd always have the option of doing clean up in the CLIENT_CLOSED event. But this error seems to indicate that sometimes the context for the iRule or table access from it is gone when the event fires. If you don't get a reply from a developer on this shortly, I'd try opening a case to get clarification on why it's happening. If you do open a case, can you post the case number so we can check in on it?
- bchick2_8645NimbostratusI opened case C998162 on this. Will still try to troubleshoot a little on my own as time permits but we have a lot going on today so I just went ahead
- spark_86682Historic F5 AccountThere have been some issues with suspending commands (like the table command) and connections that are in the process of being removed. That error message was added to 10.2.3 to note when that problem was hit (instead of silently failing like before). You should continue your support case to get a resolution for your issue, as it should eventually get to a developer with more information than I have, and it would be great if you could post that here.
- HamishCirrocumulusAh... yeah.. Lightbulb moment... Head rush... Room is spinning... That strikes a chord... IIRC with 20.20 hindsight I had a similar case open last year with checkpoint support when I was working on the LDAP performance measurement iRule and software. Sorry,. I'm at a different client now, so I don't have access to my email train with the support call.
- bchick2_8645NimbostratusOkay, I guess that makes sense. If that's the case then the iRule from the Wiki entry that I linked in my original post would have the same problem since it also relies on the CLIENT_CLOSED event to decrement the count. So based on the example in the link posted by spark it looks like I would have to modify my iRule to have a separate table for each client IP address and use the client port as the key and then look at how many keys are in the table to determine the number of clients from that IP. Then rather than decrementing a count I just remove the key when the CLIENT_CLOSED event fires or otherwise rely on the timer to clear it.
- bchick2_8645NimbostratusHere is my rewrite based on the approach I mentioned above. I have not had a chance to test it yet so I apologize if there is something obvious wrong with it. I will test it later today but based on the examples in the link posted by spark I think this is a better approach. After I have a chance to run it I will let you know whether the log entries go away and whether it appears to otherwise work okay.
when RULE_INIT { The maximum number of TCP connections to the virtual server from a single client IP address set static::max_connections_per_ip 1000 Table name (will later be appended with VS name) set static::tbl "vsratelimit" } when CLIENT_ACCEPTED { If client IP is exempted skip the rest of the rule and return if {[class match [IP::client_addr] equals concurrent_sessions_exemptions]} { return } Set the key to the client TCP port set key "[TCP::client_port]" Append the VS name and client IP to the end of the table name set tbl ${static::tbl}_[virtual name]_[IP::client_addr] Check if over the max connection limit and if so reject if {[table keys -subtable $tbl -count] >= $static::max_connections_per_ip} { Reject the connection event CLIENT_CLOSED disable reject log local0. "Connection rejected for [IP::client_addr], max connections exceeded" } else { Connection accepted, add key for client port table set -subtable $tbl $key 1 180 log local0. "Entry added to $tbl for $key" Set timer to keep key alive, if connection dies without CLIENT_CLOSED the timer will die and the timeout on the key will cause it to be removed set timer [after 60000 -periodic { table lookup -subtable $tbl $key }] } } when CLIENT_CLOSED { If client IP is exempted skip the rest of the rule and return if {[class match [IP::client_addr] equals concurrent_sessions_exemptions]} { return } Set the key to the client IP set key "[TCP::client_port]" Append the VS name to the end of the table name set tbl ${static::tbl}_[virtual name]_[IP::client_addr] Remove the timer after cancel $timer Clear the table entry table delete -subtable $tbl $key log local0. "Table $tbl entry deleted for $key" }
- spark_86682Historic F5 AccountThat is a much better approach. Nice work! Couple improvements could be made, mostly because we haven't updated the example iRule in that article (sorry). First, the key and tbl local variables should carry over from CLIENT_ACCEPTED to CLIENT_CLOSED, so there's no need to compute them again there. Second, to help protect against the issue you originally encountered, I'd delete the table entry in CLIENT_CLOSED before canceling the timer. Finally, if you have a critical need to never exceed the limit even by a little, then you should do the table set before getting the key count and then delete the entry you just added if you are over limit and the reject the connection.
- bchick2_8645NimbostratusThanks, I've made those changes and have started running with it on a dev box. At first glance it appears to be working fine so I'll just let it run and see how it behaves over time.
Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects