Forum Discussion
HTTP::payload replace
I have 2 needs...1 to replace/hide the server in the http redirects that are being sent by the node. 2) to replace/hide the server in the actual content.
when HTTP_RESPONSE {
if { [LB::server addr] eq "10.1.1.1" } {
HTTP::header replace Location [string map { } [HTTP::header Location]] }
elseif { [LB::server addr] eq "10.1.1.2" } {
HTTP::header replace Location [string map { } } [HTTP::header Location]] }
}
the irule above works great for the redirects but not for the content / payload. When I read up on the HTTP::payload command though it's specifying offset or begin point and I won't know that because the servername will appear different on different pages.
Any suggestions ? I'd love to do a HTTP::payload replace with a findstr if possible but they way I read the wiki for HTTP::payload replace I can't.
Oh, and by the way, using stream profile isn't working. I have a separate support case open for that. I would have preferred to use stream but for some reason it ain't working whereas the irules are.
thanks!!!
17 Replies
- Here's the irule now..with the HTTP::collect in the first event, I get page can't be displayed so that now the 301 redirect isn't even working. If I move the HTTP::collect to the 2nd event, the 301 redirect logic works properly but nothing in the 2nd event works.
when HTTP_RESPONSE {
if { [HTTP::status] starts_with "301" } {
if { [LB::server addr] eq "10.0.19.50" } {
HTTP::header replace Location [string map {nwte1jas7:8882 E1NP.crocs.com} [HTTP::header Location]]
log "replacing nwte1jas7:8882 in header with E1NP" }
elseif { [LB::server addr] eq "10.0.19.42" } {
HTTP::header replace Location [string map {nwte1jas6:8882 E1NP.crocs.com} [HTTP::header Location]]
log "replacing nwte1jas7:8882 in header with E1NP" }
}
}
when HTTP_RESPONSE_DATA {
HTTP::collect
log "HTTP payload length is [HTTP::payload length]"
if { [LB::server addr] eq "10.0.19.50" } {
regsub -all "nwte1jas7:8882" [HTTP::payload] "bob" newdata
log "Replacing nwtejas7:8882 with bob in the payload."
HTTP::payload replace 0 [HTTP::payload length] [string map {nwte1jas7:8882 E1NP.crocs.com} [HTTP::payload]]
HTTP::release }
elseif { [LB::server addr] eq "10.0.19.42" } {
regsub -all "nwte1jas6:8882" [HTTP::payload] "bob" newdata
log "Replacing nwtejas6:8882 with bob in the payload."
HTTP::payload replace 0 [HTTP::payload length] [string map {nwte1jas6:8882 E1NP.crocs.com} [HTTP::payload]]
HTTP::release }
} - OK..update...after checking this in HTTPwatch the Content-length is in the HTTP header but I'm not getting it in the irule..Every time this irule runs, the content_length variable gets set to 1000000
when HTTP_RESPONSE {
if {([HTTP::header exists "Content-Length"]) && ([HTTP::header "Content-Length"] <= 1000000)}{
set content_length [HTTP::header "Content-Length"] }
else {
set content_length 1000000 } - current irule looks like below...issues, the http::collect is killing performance.
when HTTP_RESPONSE {
grab the response
if { [HTTP::header exists "Content-Length"] } {
set content_length [HTTP::header "Content-Length"]
} else {
set content_length 20000
}
if { $content_length > 0 } {
HTTP::collect $content_length
}
}
when HTTP_RESPONSE_DATA {
if { [HTTP::status] starts_with "30" } {
if { [LB::server addr] eq "10.0.19.50" } {
HTTP::header replace Location [string map {nwte1jas7:8882 e1np.crocs.com} [HTTP::header Location]]
log "replacing nwte1jas7:8882 in header with E1NP"
} elseif { [LB::server addr] eq "10.0.19.42" } {
HTTP::header replace Location [string map {nwte1jas6:8882 e1np.crocs.com} [HTTP::header Location]]
log "replacing nwte1jas7:8882 in header with E1NP"
}
}
log "HTTP payload length is [HTTP::payload length]"
if { [LB::server addr] eq "10.0.19.50" } {
regsub -all "nwte1jas7:8882" [HTTP::payload] "bob" newdata
log "Replacing nwtejas7:8882 with bob in the payload."
HTTP::payload replace 0 [HTTP::payload length] [string map {nwte1jas7:8882 e1np.crocs.com} [HTTP::payload]]
HTTP::release
} elseif { [LB::server addr] eq "10.0.19.42" } {
regsub -all "nwte1jas6:8882" [HTTP::payload] "bob" newdata
log "Replacing nwtejas6:8882 with bob in the payload."
HTTP::payload replace 0 [HTTP::payload length] [string map {nwte1jas6:8882 e1np.crocs.com} [HTTP::payload]]
HTTP::release
}
} 
hoolioCirrostratus
Are you testing a single connection and seeing the response times go very high, or are you doing load testing with multiple clients. If the former, I'd try capturing a tcpdump on interface 0.0 which includes the TCP traffic and UDP514 traffic (syslog). You can then compare the packets in the connection with the log statements from the rule to see what's happening. If you're doing load testing, I would expect some additional latency even if the rule was working correctly as LTM is having to buffer the response content.
Aaron- I wanted to update / close this out. We were never able to get the irule to rewrite all of the HTTP_RESPONSE_DATA being passed back. well, correction. We were able to rewrite data up to the amount being collected w/ the HTTP::collect. When we set a length to the HTTP::collect of 1 MB we had issues w/ the pages not even displaying plus the latency introduced was not acceptable to the customer. They made changes to their application so that a simple http redirect irule and a simple source <=> target substitution stream profile were able to address the problem.
The final irule that this post was about is below. This seemed to provide the best mix of least delay and getting "most" of the data rewritten. The reason the app was changed is that none of the gifs would display properly to the customer since the applicaiton server in question wanted it's server name in the the headers it was seeing and we never were able to completely rewrite the http_request data properly.
when HTTP_REQUEST {
switch [HTTP::header Host] {
nwte1jas7:8882 {
HTTP::header replace Host "nwte1jas7:8882"
}
nwte1jas7:8882 {
HTTP::header replace Host "nwte1jas6:8882"
}
}
}
when HTTP_RESPONSE {
if { [HTTP::header exists "Content-Length"] } {
set content_length [HTTP::header value "Content-Length"]
} else {
set content_length 5000
}
if { $content_length > 0 } {
HTTP::collect $content_length
}
if { [HTTP::status] starts_with "3" } {
switch [LB::server addr] {
10.0.19.50 {
HTTP::header replace Location [string map {nwte1jas7:8882 e1np.crocs.com} [HTTP::header Location]]
log local0.info "Replacing nwte1jas7:8882 with e1np.crocs.com in the header."
}
10.0.19.42 {
HTTP::header replace Location [string map {nwte1jas6:8882 e1np.crocs.com} [HTTP::header Location]]
log local0.info "Replacing nwte1jas6:8882 with e1np.crocs.com in the header."
}
}
}
}
when HTTP_RESPONSE_DATA {
log "running http_reponse_data event"
grab the response
switch [LB::server addr] {
10.0.19.50 {
regsub -all "nwte1jas7:8882" [HTTP::payload] "e1np.crocs.com" newpayload
log "Replacing nwtejas7:8882 with e1np.crocs.com in the payload."
HTTP::payload replace 0 [HTTP::payload length] $newpayload
HTTP::release
}
10.0.19.42 {
regsub -all "nwte1jas6:8882" [HTTP::payload] "e1np.crocs.com" newpayload
log "Replacing nwtejas6:8882 with e1np.crocs.com in the payload."
HTTP::payload replace 0 [HTTP::payload length] $newpayload
HTTP::release
}
}
} - hoolioThanks for posting the additional info. What was the issue/fix for using a stream profile?
Also, for anyone:
Does anyone else have ideas on how best to collect response data when there isn't a content length header? If you specify a smaller number of bytes than what is sent, does the HTTP::payload command only return what you've collect? If so, don't you miss the change to perform the replacement in any subsequent response data that wasn't collected?
Thanks,
Aaron - initially each app server was putting in it's name and non-standart port - ie appserver1:8882, appserver2:8882. the stream profile for a multi-string search / replace was to put @appserver1:8882@dnsname.vip.com@ @appserver2:8882@dnsname.vip.com@ . That multi-search didn't work. They changed their appservers to put in dnsname.vip.com but couldn't stop the non-standard port. So either app server would had all the content / headers dnsname.vip.com:8882. I wrote an irule to handle the redirects and then used a simple stream profile like source "dnsname.vip.com:8882" target "dnsname.vip.com" .
I would also like to know how to limit the HTTP::response. What I was seeing w/ their content is that some of the HTTP headers had the Content-Length field and some didn't. I can also accurately say that when I was using the irule above, w/ the default content_length set to 5k we were repeatedly seeing a 16 second delay in hitting the initial long page. Normally, browsing directly to the page they would see 3-4 seconds to bring it up. With the original HTTP::response irule they would take 16-18 seconds to display the page.
