Forum Discussion
amacks_3981
Nimbostratus
NimbostratusTraffic Throttling?
Is it possible to use an iRule to slow incoming traffic from a specific useragent? I know how to recognize the agent, I'm just not sure if there is a way to say(pseudocode):
on connect{
if...
Kirk_Bauer_1018
Altostratus
AltostratusI have this iRule that I wrote and tested but never deployed. You should be able to adapt this pretty easily although I know it is pretty complex...
HTTP Request Limit iRule
Kirk Bauer
Need a statistics class associated with each VS using this rule as follows:
profile stats Request_Limit_Stats {
defaults from stats
field1 throttled_dropped_requests
field2 blacklisted_dropped_requests
field3 throttled_clients
field4 unthrottled_clients
}
Also need to define two host/network data groups named:
RequestLimit_Whitelist
RequestLimit_Blacklist
when RULE_INIT {
Expiration for tracking IPs not yet throttled (seconds)
set ::standard_expiration_time 300
Expiration for tracking throttled clients
set ::throttled_expiration_time 3600
Throttle clients that exceed this number of requests/min
set ::throttle_activate_rate 30
Once throttled, limit clients to this number of requests/min
set ::throttle_limit_rate 5
Limit blacklisted clients to this number of requests/min
set ::blacklist_limit_rate 2
Message to be displayed when requests are dropped
set ::drop_msg "Too many requests, please try again later."
}
when HTTP_REQUEST {
HTTP::header insert "X-RequestLimit-Class" "normal"
set client_ip [IP::remote_addr]
if {[matchclass [IP::remote_addr] equals $::RequestLimit_Whitelist]} {
HTTP::header replace "X-RequestLimit-Class" "whitelisted"
} else {
set curr_time [clock seconds]
set blacklisted 0
set throttle_timekey throttle_starttime_$client_ip
set throttle_reqkey throttle_reqcount_$client_ip
set throttle_request_count [session lookup uie $throttle_reqkey]
if {[matchclass [IP::remote_addr] equals $::RequestLimit_Blacklist]} {
set blacklisted 1
if {$throttle_request_count <= 0} {
session add uie $throttle_timekey [expr {$curr_time - 2}] [expr {$::throttled_expiration_time + 2}]
set throttle_request_count 1
}
}
if {$throttle_request_count > 0} {
set throttle_start_time [session lookup uie $throttle_timekey]
incr throttle_request_count
session add uie $throttle_reqkey $throttle_request_count $::throttled_expiration_time
set elapsed_time [expr {$curr_time - $throttle_start_time}]
if {$elapsed_time < 60} {
set elapsed_time 60
}
set curr_rate [expr {$throttle_request_count / ($elapsed_time/60)} ]
if {$blacklisted} {
HTTP::header replace "X-RequestLimit-Class" "blacklisted"
set limit_rate $::blacklist_limit_rate
} else {
HTTP::header replace "X-RequestLimit-Class" "throttled"
set limit_rate $::throttle_limit_rate
}
if {$curr_rate > $limit_rate} {
Throttle this request
HTTP::respond 500 content $::drop_msg
}
} else {
if {([HTTP::uri] contains "RemoteAdvancedSearchView") or ([HTTP::uri] contains "CatalogSearchResultView")} {
set timekey starttime_$client_ip
set reqkey reqcount_$client_ip
set request_count [session lookup uie $reqkey]
if {$request_count > 0} {
set start_time [session lookup uie $timekey]
incr request_count
session add uie $reqkey $request_count $::standard_expiration_time
set elapsed_time [expr {$curr_time - $start_time}]
if {$elapsed_time < 60} {
set elapsed_time 60
}
set curr_rate [expr {$request_count / ($elapsed_time / 60)}]
if {$curr_rate >= $::throttle_activate_rate} {
Throttle this client after this request (add them to throttle list)
session add uie $throttle_timekey $start_time [expr {$::throttled_expiration_time + 2}]
session add uie $throttle_reqkey $request_count $::throttled_expiration_time
log local0. "RequestLimit: !THROTTLED! $client_ip: rate=$curr_rate, request_count=$request_count, elapsed_time=$elapsed_time"
}
} else {
Add tracking sessions for new client IP
session add uie $timekey $curr_time [expr {$::standard_expiration_time + 2}]
session add uie $reqkey 1 $::standard_expiration_time
}
}
}
}
}
Also it is important to note that by watching for HTTP_REQUEST you are effectively request-rate-limiting at layer 7 as opposed to simply connection-rate-limiting at layer 4 as you can do with other products (because multiple requests can come in one connection). Also note that this is set up for limiting to a certain number of requests per minute but it can be changed to requests per-sec pretty easily.
