Forum Discussion
mrintzler
Nimbostratus
Feb 04, 2008iRule causing ever-increasing TMM CPU utilization
I've adapted an iRule written by Deb Allen to rate limit (in count, not in bps) PDF downloads from one of our web applications. It basically counts the number of PDF's downloaded in the past 10 seconds, and gives an error response if they are exceeding the threshold. The rule works as expected. However, as the rule is in place, TMM CPU utilization slowly increases, until performance is affected after about a week. If I re-initialize the iRule (make a small modification and save, or simply remove and reassign to the virtual) the TMM utilization drops back down to normal levels. I recently upgraded from 9.2.3 to 9.4.4 in hopes that a couple memory leak issues were the culprit, but the issue remains. iRule timing shows that the avg CPU time for the iRule steadily increases as time goes on. I'm thinking the issue is due to the array growing in size, despite the lines clearing the older entries out. Is there a way to automate periodically re-initializing the array? Below is the iRule:
rule pdfapp-abuse-detection-rule
Limits the rate of PDF downloads for a particular IP client
Adapted from 'RateLimit_HTTP' on devcentral
Adapted by Michael Rintzler
August 20th, 2007
Original concept by Deb Allen, F5 Networks
April 2006
when RULE_INIT {
set ::pdfappmaxRate 10 ;set later per user from class
set ::pdfappwindowSecs 10 ;global
init array if non-existent
array set ::pdfHistory { }
wipe array if already existent
array unset ::pdfHistory
}
when HTTP_REQUEST timing on {
if { [HTTP::method] eq "GET" } {
if { ([HTTP::uri] contains ".pdf") and not ([HTTP::header exists Range ]) and ([HTTP::cookie exists ERIGHTS])} {
log local0. "pdfapp PDF Initial download detected."
Extract clients IP address
set client_ip [IP::remote_addr]
set pdfapp_session [HTTP::cookie ERIGHTS]
set currentTime [clock seconds]
we need to count requests in last $windowSecs seconds, so mark the cutoff time
set pdfappwindowStart [expr {$currentTime - $::pdfappwindowSecs}]
find GETs for this sessionID
set pdfCount 1
set mypdfappMaxRate $::pdfappmaxRate
count GETs within the window, delete those that are older
foreach { requestID requestTime } [array get ::pdfHistory ${pdfapp_session}*] {
count pdf downloadss with start time > $pdfappwindowStart, delete the rest
if { $requestTime > $pdfappwindowStart } {
incr pdfCount 1
} else {
unset ::pdfHistory($requestID)
}
}
if { $pdfCount < $mypdfappMaxRate } {
Allow request and add new record to array w/myUserID.rand + currentTime
set pdfapprequestID to a random number
set requestID "${pdfapp_session}.[expr { int(10000000 * rand()) }]"
set ::pdfHistory($requestID) $currentTime
log local0. "PDF Abuse - download from $client_ip, SESSION $pdfapp_session - Current rate: $pdfCount"
} else {
Reject request with 200 response
log local0. "PDF Abuse - User $client_ip, Session $pdfapp_session"
return
}
}
else { log "Partial PDF download detected." }
}
}
- spark_86682Historic F5 AccountIt sounds like you've diagnosed the problem correctly in that your array just grows over time. If a client just downloads a PDF and leaves, I don't see any way that your iRule ever cleans that entry out of the array, so under normal use, the array would just grow and grow.
- Deb_Allen_18Historic F5 AccountTo answer your question, I've used the clock to trigger periodic housekeeping in other circumstances, something like this:
when RULE_INIT { set ::pdfappmaxRate 10 ;set later per user from class set ::pdfappwindowSecs 10 ;global init array if non-existent array set ::pdfHistory { } wipe array if already existent array unset ::pdfHistory set ::housekeeping_clock [clock seconds] set ::housekeeping_interval 900 } when HTTP_REQUEST { if { [clock seconds] > [expr {$::housekeeping_clock + $::housekeeping_interval}] }{ wipe array if already existent array unset ::pdfHistory } ... }
Not sure why it isn't doing so. Before I added another periodic housekeeping function, esp one that's indiscriminate like that above, I'd try to figure out why this isn't working. I'd start by adding some logging here to see if anything is matching the delete condition.count GETs within the window, delete those that are older foreach { requestID requestTime } [array get ::pdfHistory ${pdfapp_session}*] { count pdf downloadss with start time > $pdfappwindowStart, delete the rest if { $requestTime > $pdfappwindowStart } { incr pdfCount 1 } else { unset ::pdfHistory($requestID) } }
- mrintzler
Nimbostratus
You're exactly right. That was my problem. When I first implemented a version of this iRule, it was keyed off of the end-users IP address. Most of our customers will have the same IP address indefinitely, or at least for a long time. When I modified the iRule for a different application, I keyed the users by their session cookie, which changes each time they log in. As you pointed out, I only wipe their history from the array when they come back to download, so if a user comes back an hour later with a different session ID, their old entries in the array will be stuck their forever.if { $::totalPDF > $::refreshint } { array unset ::pdfHistory log local0. "$::refreshint downloads reached. Resetting array" set ::totalPDF 0 }
Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects