Forum Discussion
Russell_McGinni
Nimbostratus
NimbostratusiRule to replace http response data causes havoc
We have a live and beta VIP setup in F5 and in order to keep users in the environment they started in, we would like to use an iRule - we already use an iRule to inject a javascript include on most html pages and thought we could piggy back on that.
However when we included code to replace "www" with "beta" in our urls our F5's started failing over - not instantly - but more frequently when traffic peeked, less frequently during low traffic usage.
Any ideas what we are doing wrong ?
Many thanks in advance
Russell
when RULE_INIT {
Text to inject at the end of the stream
set ::InjectionText ""
set ::InjectText 0
set ::uri ""
}
when HTTP_REQUEST {
Save the URI so we can check it in the response
set ::uri [string tolower [HTTP::uri]]
set ::InjectText 0
}
when HTTP_RESPONSE {
if { [HTTP::status] eq "200" } {
set replace 0
set ctype [HTTP::header Content-Type]
First check to see if we are dealing with html or js content
switch -glob $ctype {
"*html*" -
"*javascript*" -
"*xml*" {
set replace 1
}
}
Incase there is no content type set on an htm(l) or js file
switch -glob $::uri {
"*.htm*" -
"*.js" {
set replace 1
}
}
If we have html content, make sure we add the Injection text later
if { $ctype starts_with "text/html" } {
set ::InjectText 1
}
But dont add the Injection text if we are dealing with certain files
That have a content type of text/html
switch -glob $::uri {
"*.ashx*" -
"*.asmx*" -
"*.axd*" -
"*.js*" -
"/members/login/*" -
"/members/uploader*" {
set ::InjectText 0
}
}
Trigger the event for HTTP_RESPONSE_DATA if needed
if { $replace == 1 } {
log local0. "Replace value: $replace"
if { [HTTP::header exists "Content-Length"] } {
set content_length [HTTP::header "Content-Length"]
} else {
set content_length 2048000
}
if { $content_length > 0 && $content_length < 2048001} {
HTTP::collect $content_length
} else {
log local0. "Http content greater than 2mb (actual: $content_length) for: $::uri"
}
}
}
}
when HTTP_RESPONSE_DATA {
First check to see if we have any text to inject
if { $::InjectText == 1 } {
set idx [string last "" [HTTP::payload]]
if { -1 == $idx } {
set idx [string last "" [HTTP::payload]]
}
if { -1 == $idx } {
set offset [HTTP::payload length]
log local0.debug "Got here but NOT injecting text into $::uri"
} else {
set offset $idx
HTTP::payload replace $offset 0 $::InjectionText
log local0.debug "Injecting text into $::uri"
}
}
set clen [string length [HTTP::payload]]
set old_name "www.mydomain.com"
set new_name "beta.mydomain.com"
set old_name_len [string length $old_name]
set new_name_len [string length $new_name]
set last [string last $old_name [HTTP::payload $clen]]
for {set old_index 0} {$old_index < $last} {incr old_index $new_name_len}{
set old_index [string first $old_name [HTTP::payload $clen] $old_index]
HTTP::payload replace $old_index $old_name_len $new_name
}
set clen [string length [HTTP::payload]]
set old_name "www.myotherdomain.org"
set new_name "beta.myotherdomain.org"
set old_name_len [string length $old_name]
set new_name_len [string length $new_name]
set last [string last $old_name [HTTP::payload $clen]]
for {set old_index 0} {$old_index < $last} {incr old_index $new_name_len} {
set old_index [string first $old_name [HTTP::payload $clen] $old_index]
HTTP::payload replace $old_index $old_name_len $new_name
}
HTTP::release
}
hoolioCirrostratus
Hi,
There are a few issues. The first one that jumps out is you're using global variables where you should be using locals:
set ::InjectText 0
set ::uri ""
These should both be set as locals and removed from RULE_INIT:
set InjectText 0
set uri ""
Another potential issue is if the server response contains a Content-Length header set to a value greater than 4Mb TMM will crash.
SOL6578: TMM will crash if an iRule collects more than 4MB of data
https://support.f5.com/kb/en-us/solutions/public/6000/500/sol6578.html
You should consider logic like this to limit the payload collection size:Trigger collection for up to 1MB of data if {([HTTP::header exists "Content-Length"]) && ([HTTP::header "Content-Length"] <= 1024000)}{ set content_length [HTTP::header "Content-Length"] } else { set content_length 1024000 } if { [info exists content_length] } { HTTP::collect $content_length if {$::rule_debug}{log local0. "[IP::client_addr]:[TCP::client_port]: Collecting $content_length"} }
I didn't look too closely at the replacement logic. Does it work after you make the above changes?
In terms of optimisation, you could eliminate a lot of the intermediate variables like 'set ctype [HTTP::header Content-Type]' and just reference [HTTP::header Content-Type]. The value for most HTTP:: commands is cached in the same event so there isn't a performance hit for re-referencing the command. There is extra memory used when saving the values to a new variable though.
Aaron
Russell_McGinniThanks Aaron for the prompt and helpful reply...
I am using global variables initialize in RULE_INIT as i thought that was the only way they would be visible across all event handlers - this was a bad assumption?
Additionally I thought I was being careful of the 4MB limit, also taking into account Unicode by limiting the payload collection size to 2Mb - however as I am adding content to the payload I can see how that would break through the limits I set.
I'll make the changes and test again.
Thanks again Aaron- Russell_McGinniThanks Aaron, is my assumption correct, that if I do not "collect" for payloads greater than a limit (2Mb in my example), the HTTP_RESPONSE_DATA event isnt triggered - and therefore no payload replacement takes place ?
Also, "I'd just suggest setting a max and collecting at least that for responses that are bigger than the limit, as you can do partial payload collection." - I'm not really sure I understand what you mean by this.
Russell - hoolioHi Russell,
That's correct. If you don't call HTTP::collect, no payload will be collected and no replacements can be made to that response. You can collect the start (the first x bytes) of the payload and avoid the 4Mb limit even if the payload is larger than x bytes. You could then perform the replacement(s) in that collected portion. If there are strings you wanted to replace past the collected amount, they wouldn't be modified as that portion of the payload wasn't buffered.
I have a faint memory of someone discussing the possibility of collecting x bytes, releasing it and then re-collecting more payload, but I can't find the post and haven't ever seen it implemented. Does anyone have more info on this possibility?
Aaron