Forum Discussion
qqdixf5_74186
Nimbostratus
Dec 17, 2007http::retry question
I am having some problems with a rule which uses http::retry command. What I am trying to do here is authentiting the request against an external service. When a request comes in, an auth request is sent to the auth service and based on the response, the original request is either rejected or sent to the backend. I have the iRule partially working, which means, it works for some requests, but not all. By adding extra logging and tcpdump at the LTM, I see some requests failed with "Bad Request(Invalid Hostname)" error.
When I first wrote the iRule, I was following an example here and using a simple flag to indicate if the request need authentication. I thought after the authentication passed the request would be sent to the selected pool directly. But somehow, it turned out that the rule was applied to the request again(Is this the way how BigIP handle this?). So I had to add a custom header in the request to avoid getting into an infinite authenticating loop. That's the only change I made to the request. I am not sure why some requests are failing. Hoping I can get some help here. Thank you!
Here is the rule:
when CLIENT_ACCEPTED {
set the flag to control lookup
set lookup 0
}
when HTTP_REQUEST {
set the lookup flag. Value 0 means the request needs to be
looked up
set lookup 0
if ([HTTP::header exists "Authenticated"]) {
set lookup [HTTP::header "Authenticated"]
}
if {$lookup == 0} {
save original request
set original_request [HTTP::request]
get the content length and replace the request payload with empty string
set payload_size [HTTP::payload length]
set original_payload [HTTP::payload]
HTTP::payload replace 0 $payload_size " "
inject lookup URI
HTTP::uri "/AuthService/ValidateToken.jsp?token=sometokenidfromtheoriginalrequest"
remove extra headers from the auth request.
HTTP::header sanitize "Host"
send the auth request to auth service pool
pool AUTH_SERVICE
} else {
pool SERVICE_POOLA
}
}
when HTTP_RESPONSE {
if {$lookup == 0 } {
collect first response (from lookup server) only
HTTP::collect 1
}
}
when HTTP_RESPONSE_DATA {
Reads the auth response and look for "Accepted"
if {$lookup == 0} {
check if the payload contains "Accepted".
If so, replay the original request.
if {$payload contains "Accepted"} {
log local0. "Request authenticated"
use pool SERVICE_POOLA
insert a custom http header to the original request and send it
set new_request [string map {"Host" "Authenticated: 1 Host"} $original_request]
HTTP::retry $new_request
} else {
reject the request
reject
}
}
}
10 Replies
- Patrick_Chang_7Historic F5 AccountAre you persisting these requests to the same back end server?
- qqdixf5_74186
Nimbostratus
No, I am not persisting the requests. But I am testing this with just one server in the back end pool. - qqdixf5_74186
Nimbostratus
Anybody has any idea of what might be wrong about this rule? Thank you! - qqdixf5_74186
Nimbostratus
I did some more tests to test http::retry using the following rule. I tested the same request that failed on my original rule. From the logging, I see the first request always went through but the retry request always get a "Bad Request" response. I looked at the http::retry wiki page again and saw this - "Resends a request to a server. The request header must be well-formed and complete." So, I think that the original request's headers are likely the cause of the failure, although they looked fine to me. I just wonder why this command is strict on the request headers if all it supposed to do is resending a request.
when CLIENT_ACCEPTED {
set the flag to control lookup
set lookup 0
}
when HTTP_REQUEST {
save the original request for retry later
if { $lookup == 0 } {
set original_request [HTTP::request]
set original_payload [HTTP::payload]
log local0. "Original Request"
log local0. "Original Request = $original_request"
log local0. "Original Payload = $original_payload"
} else {
log local0. "Retry Request"
set retry_request [HTTP::request]
set retry_payload [HTTP::payload]
log local0. "Retry request = $retry_request"
log local0. "Retry payload = $retry_payload"
}
pool NEXTGEN_POOLA
}
when HTTP_RESPONSE {
collect first response (from lookup server) only
if { $lookup == 0} {
log local0. "Received First Response"
} else {
log local0. "Received Retry Response"
}
HTTP::collect 1
}
when HTTP_RESPONSE_DATA {
set payload [HTTP::payload]
if { $lookup == 0 } {
log local0. "First Response Payload = $payload"
retry the request again.
set lookup 1
pool NEXTGEN_POOLA
HTTP::retry $original_request
} else {
log local0. "Retry Response Payload = $payload"
}
} - hoolio
Cirrostratus
Have you already captured a tcpdump of a retried request which the web server responds to with a 400 - bad hostname? If so, are the headers valid? Can you compare the request the BIG-IP sends to the web server with the original client request?
It looks like you're using string map to replace Host with Authenticated 1 Host. Without inserting a carriage return and line feed this would result in the host header being corrupted:
Sample original request:
GET /index.html HTTP/1.1
Host: test.example.com
Connection: close
After modification, the host header would be gone:
GET /index.html HTTP/1.1
Authenticated: 1 Host: test.example.com
Connection: close
The web server would parse the Authenticated header as having a value of "1 Host: test.example.com".
You could try to figure out how to insert a carriage return and line feed (0x0d and 0x0a in hex) between the 1 and Host. Else, it would probably be easier to use 'HTTP::header insert Authenticated 1' and then save the HTTP::request value. It looks like this second method would work:when HTTP_REQUEST { log local0. "original \[HTTP::request\]: [HTTP::request]" HTTP::header insert new_header some_value log local0. "modified \[HTTP::request\]: [HTTP::request]" }
Request:
curl http_vip
Log output:
: original [HTTP::request]: GET / HTTP/1.1 User-Agent: curl/7.15.4 (i686-pc-cygwin) libcurl/7.15.4 OpenSSL/0.9.8d zlib/1.2.3 Host: http_vip Accept: */*
: modified [HTTP::request]: GET / HTTP/1.1 User-Agent: curl/7.15.4 (i686-pc-cygwin) libcurl/7.15.4 OpenSSL/0.9.8d zlib/1.2.3 Host: http_vip Accept: */* new_header: some_value
Aaron - qqdixf5_74186
Nimbostratus
Thank you for your reply!
Yes, I thought about the header I added in my original rule might be the problem so I ran tests and captured the tcpdump. It does look like what you descripted.
GET /website.aspx HTTP/1.0
Authenticated: 1 Host: test.example.com
However, weird thing is it works for some requests. I would expect all request fail if that causes the problem.
Also, since the purpose of the rule is authenticate before sending request to the backend server, I can't insert a header before knowing the request is authenticated by checking the authentication response.
I tried to retry the original request without modification. My test rule (in my later post) just sends the request and retries it without any change. It fails too. I also tried sanitize all headers before the request was resent. It doesn't work either. Just not sure what "the request headers must be well-formed and complete" means now. If someone knows more about this, please help. Thank you! - hoolio
Cirrostratus
Actually, this is a bit trickier, because you're not deciding whether to insert the header into the request until the HTTP_RESPONSE_DATA event. So you can't use HTTP::request to get the original client request and then use HTTP::header insert to insert your custom header. If you wanted to stick with inserting a header in the request, you could either insert the authenticated header and then save every request in the HTTP_REQUEST event, or you could work out how to insert a carriage return and line feed in the payload.
The original example for this (Click here), uses a local variable to track whether the request has already been authenticated. You said that you were seeing the requests looping to the authentication pool when you used a variable. Perhaps it would be better to go back to that version of the rule and troubleshoot why requests were constantly being sent to the auth pool.
Aaron - hoolio
Cirrostratus
Can you post your current rule so we can take a look?
Thanks,
Aaron - qqdixf5_74186
Nimbostratus
Here is the current rule which is working. Thanks for taking a look!when CLIENT_ACCEPTED { set the flag to control lookup set lookup 0 } when HTTP_REQUEST { check lookup flag. Value 0 means the request needs to be authenticated. if { $lookup == 0 } { log local0. "Start Authentication Process..." save the original request and original payload and payload length set original_request [HTTP::request] set original_payload [HTTP::payload] set original_payload_length [HTTP::payload length] log local0.debug "Original Request = $original_request" log local0.debug "Original Payload = $original_payload" authenticate against auth service set token_id "i0adc5d150456711529677be25a1b52e6" clean the auth request payload HTTP::payload replace 0 $original_payload_length " " inject lookup URI in place of original request HTTP::uri "/AuthService/ValidateToken.jsp?token=$token_id" remove extra headers from the auth request HTTP::header sanitize "Host" log local0.debug "Sending Auth Request to AUTH_SERVICE pool = [HTTP::request]" pool AUTH_SERVICE } else { log local0.info "Request is authenticated, sending it to service pool" correct the request payload and content length set retry_payload_length [HTTP::payload length] HTTP::payload replace 0 $retry_payload_length $original_payload if { [HTTP::header exists "Content-Length"] } { HTTP::header replace "Content-Length" $original_payload_length } log local0.debug "Request = [HTTP::request]" log local0.debug "Payload = [HTTP::payload]" pool SERVICE_POOLA } } when HTTP_RESPONSE { collect first response (from lookup server) only if { $lookup == 0} { log local0.debug "Received Auth Response from Sso service" } else { log local0.debug "Received Response for original request" } HTTP::collect 1 } when HTTP_RESPONSE_DATA { set payload [HTTP::payload] if { $lookup == 0 } { check if the auth payload contains "Accepted". If so, replay the origianl request. if { $payload contains "Accepted" } { log local0.info "Request authenticated. Now send the original request to the backend service pool = $original_request" set lookup 1 pool SERVICE_POOLA HTTP::retry $original_request } else { log local0.info "Request Denied. No request will be sent to backend service pool" reject } } else { log local0.debug "Original Response Payload = $payload" } } - hoolio
Cirrostratus
Thanks for posting the working version. I haven't worked with HTTP::request / HTTP::retry before, so I was unsure about whether the payload, if present, would be captured using HTTP::request. After some testing, it looks like it isn't and can't be.
The HTTP::request command is only available in the HTTP_REQUEST event. That event is triggered when the headers have been parsed--the data if present hasn't been parsed yet. So I think your method of collecting the payload is the best way to handle this.
This would be good to confirm and clarify in the HTTP::request wiki page.
Aaron
Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects
