HTTP Request Throttle by IP and UserAgent
Problem this snippet solves:
This is a modification of Kirk Bauer's iRule found here: https://devcentral.f5.com/codeshare/http-request-throttle
The modification is for rate limiting individual IP addresses and UserAgents, not to specific URIs but to all virtual servers covered by the rule, testing for x-forwarded-for, etc..
The ultimate goal is to be able to slow down individual IP addresses or bots that we do not necessarily want to block but do not want them to continue to eat up resources at their current rate.
Note: I am strongly considering rewriting this to use rateclass instead of using a table to dictate how many requests can be made during a period of time. I feel like using rateclass will not only clean up the code a little, but we will be using built in rate limiting functionality instead of writing our own.
Code :
when RULE_INIT { # This is a variation of https://devcentral.f5.com/s/articles/http-request-throttle by by Kirk Bauer # This defines the maximum requests to be served within the timing interval defined by the static::timeout variable below. #log local0. "RateLimit Rule Init" set static::maxReqs 4; # Timer Interval in seconds within which only static::maxReqs Requests are allowed. # (i.e: 10 req per 2 sec == 5 req per sec) # If this timer expires, it means that the limit was not reached for this interval and the request # counting starts over. # Making this timeout large increases memory usage. Making it too small negatively affects performance. set static::timeout 8; } when HTTP_REQUEST { if { [HTTP::header exists X-forwarded-for] } { set client_IP_addr [getfield [lindex [HTTP::header values X-Forwarded-For] 0] "," 1] } else { set client_IP_addr [IP::client_addr] } set client_UserAgent [string tolower [HTTP::header "User-Agent"]] #log local0. "$client_UserAgent" if { not ([class match $client_IP_addr equals sec_whitelist ] )} { if {[class match $client_IP_addr equals sec_RateLimitedIPs]} { set getIPcount [table lookup -notouch $client_IP_addr] if { $getIPcount equals "" } { table set $client_IP_addr "1" $static::timeout $static::timeout } else { if { $getIPcount < $static::maxReqs } { table incr -notouch $client_IP_addr #log local0. "$client_IP_addr IP Count: $getIPcount, uri: [HTTP::uri]" } else { reject #log local0. "$client_IP_addr Rejected: $getIPcount" } } } set client_UserAgent [class match -name $client_UserAgent contains sec_RateLimitedUserAgents] #log local0. "$client_UserAgent$client_IP_addr" #Rate limit will be based on UserAgent in the context of Source IP so the two are combined for the tracked value if { [string length $client_UserAgent] > 0 } { set getUserAgentCount [table lookup -notouch $client_UserAgent$client_IP_addr] #log local0. "[class match $client_UserAgent contains sec_RateLimitedUserAgents]" if { $getUserAgentCount equals "" } { table set $client_UserAgent$client_IP_addr "1" $static::timeout $static::timeout } else { if { $getUserAgentCount < $static::maxReqs } { table incr -notouch $client_UserAgent$client_IP_addr #log local0. "$client_UserAgent UserAgent Count: $getUserAgentCount" } else { reject #log local0. "$client_UserAgent Rejected: $getUserAgentCount - [HTTP::host]" } } } } }