Howdy All
Is there anyway to fetch content from an external source, to determine what to do with it?
ie go get http://182.168.0.1/page.aspx?ip=1.1.1.1
and then act on the response?
Thanks
Adrian
Howdy All
Is there anyway to fetch content from an external source, to determine what to do with it?
ie go get http://182.168.0.1/page.aspx?ip=1.1.1.1
and then act on the response?
Thanks
Adrian
anyone know if you can use standard tcl geturl?
eg
http::geturl $url
I don’t think so… I get an undefined procedure on that one when checking syntax. The url can be constructed, however, with HTTP::host and HTTP::uri.
The iRules are event driven, so when a request/response arrives, do this… HTTP::host and HTTP::uri will give you the information from within a request, but I’m not sure what you are looking to do is possible within the context of iRules.
Could you give a little more detail into what you are trying to accomplish?
You could build a class up front for each country and put as grand a summarization of those spaces as possible, or you could sub-optimally route the first request from each range, log any new range request, which could kick off an event to an iControl app that can build the class for you for future requests from that range.
the tables for a country would be huge!!
is there anyway to forward a request to a specific url, get a http 200 or 404 from that url, for 200’s let the request continue to the original destination and reject anything else?
thanks for your help btw
Yes. In fact, if you sent the initial request to server X, you could have it check the country and forward it back in the response, then you could resend the request to the appropriate pool.
Framework:
when client is accepted, set a variable to 1 to establish as new connection
when first http_request event triggers, forward to server X to process country of origin
when first http_response event triggers, pull out country information from response and set the variable to 0.
when subsequent http_request events trigger, match country to appropriate pool
Write back if you get stuck with the context, or if my assumption that your server X will process the country is way off base.
Hi
server x can indeed process the country lookup.
that will then force the user to be sent a re-direct and then follow that to generate a second call back to the original URL which would then be able to lookup which country they are in.
is there anyway to do it in a single fetch? ie get the response back from server x and then re-submit the request back to the correct pool?
is there any way to add custom TCL commands? ie I could put an app on the F5 itself that does the geo lookup if I can call it from the TCL? Or opening a tcp socket connection to server x, etc?
I can think of so many ways that it would be good to be able to use GeoIP based lookups in IANA rules, wonder if it would ever become a feature!
Thanks for your help, appreciate it, think I’m nearly there…
Hi Adrian --
The “HTTP::retry” command offers a useful trigger for scenarios like this, where there is a requirement for an out-of-band socket connection to a 3rd host. Something like this might be a good start to what you’re after:
when HTTP_REQUEST {
save original request
set req [HTTP::request]
flag as new request needing lookup
set lookup 1
inject lookup URI in place of original request
HTTP::uri "/page.aspx?ip=[IP::client_addr]"
set pool to lookup server pool
pool lookup_server
}
when HTTP_RESPONSE {
if {$lookup == 1 }{
collect first response (from lookup server) only
HTTP::collect
}
}
when HTTP_RESPONSE_DATA {
Get poolname from server response
Response would ideally be in the form of a pool name only.
Otherwise parse or derive the poolname here
set myPool [HTTP::payload]
verify pool exists and has members
if { ![catch [pool $myPool]] }{
HTTP::retry $req
} else {
insert dead/non-existent pool logic here
}
re-set flag so that subsequent response to re-tried
request from real server is not processed as a lookup
set lookup 0
}
HTH
/deb
I would like to get this example to work, but all my requests go
to a virtual server with a full 0.0.0.0 destination.
vs_main_http Common 0.0.0.0 80 (HTTP) Standard
As such I don’t really use a pool for the HTTP requests, so the
“set myPool [HTTP::payload]” does not seem to make sense in my
use case as I don’t think I have a pool.
I think I would have to somehow undo and remove the “pool lookup_server”
statement in my use case to get back to the default routing behavior.
/jon
Hi Jon,
Can you explain what are you trying to do by making an out of band request to a pool? Are you determining if the client request is valid? Or something else?
Thanks,
Aaron
Aron,
Can you explain what are you trying to do by making an out of band request to a pool? Are you determining if the client request is valid? Or something else?
Actually I want to do a lookup from an external HTTP (RESTful) server to get a Mobile Phone Number (MDN) from the IP of the requesting client e.g. “/[IP::client_addr]” these mobile phones route through the F5 to the internet at large.
Yes it is an out-of-band socket connection to a 3rd host - but in the event the 3rd host is down or it fails to provide a valid MDN then the out-of-band operation does nothing and the out-of-band logic should be ignored.
Future Work: If I couldn’t look up the MDN I might re-try the out-of-band operation again once each 30 seconds as long as the connection is up.
Future Work: Once I have an MDN (if and only if I have one) then I might do some HTML injection on the response or a redirect.
Here is some proposed code - hopefully it will work any advice?
when CLIENT_ACCEPTED {
flag as new request needing lookup
set lookup 1
set flag to indicate if we had a valid lookup.
set keyfound 0
}
when HTTP_REQUEST {
if {$lookup == 1} {
we are doing a 3rd host out-of-band lookup
save original request
set req [HTTP::request]
inject a lookup URI in place of original request
HTTP::uri "/[IP::client_addr]"
set pool to lookup server pool called "mdn_lookup"
hopefully the pelow "pool mdn_lookup" is not sticky
such that the later HTTP::retry $req done in the
HTTP_RESPONSE_DATA will not go back to this pool, I
can not set it to another "normal" pool after this
out-of-band processing is done as the requests are
going to the internet at large (not servers under
my control).
pool mdn_lookup
}
}
when HTTP_RESPONSE {
if {$lookup == 1 }{
collect first response (from lookup server) only
HTTP::collect
if {[HTTP::status] == 200} {
200 OK we got an IP lookup from the RESTful server
we have to have a valid MDN or MobileNumber, all other
responses imply no lookup.
collect first response (from lookup server) only
HTTP::collect $clength
valid lookup we have a MDN and can do something with it
set keyfound 1
}
}
normal path for NOT out-of-band responses
}
when HTTP_RESPONSE_DATA {
Once the lookup is done there is NO pool to go back to
as the clients are mobile phones going out to the internet
if {$keyfound == 1} {
set mdn [HTTP::payload]
log local0. "mdb_lookup gives <<$mdn>>"
we want to actually do something with it later but
this iRULE is a proof of concept. Typical we would
alter the true response based on the mdn we just set.
If we never get a valid MDN we the request will
proceed as if the out-of-band lookup never happened.
} else {
log local0. "mdb_lookup gives NULL"
}
HTTP::retry $req
NO ERROR CHECKING YET ....
if mdn_lookup is dead/non-existant I just want to
process things as if there was no out-of_band code
re-set flag so that subsequent response to re-tried
request from real server is not processed as an MDN
lookup - eventually I might put in retry every 15
second logic until we get a valid lookup
set lookup 0
}
Fetching content from external source, seems to be a bit buggy, at leas in my use case
where I have a pair of load balanced servers that share the same nodes. For some reason
selecting a pool doesn’t seem to force a particular port.
Question 1, Why do I have to define virtual servers (it will not work at all if I do not),
from the examples I though I only needed to reference pools ?
Question 2, Why does it only work http://10.0.185.150:8002/prox_evdo/auth.gif?rnd12345,
or why doesn’t the right OOB port, 8002, get selected on the fetch from
http://10.0.185.150/prox_evdo/auth.gif?rnd12345?
Based on iRULEs examples ....
SEE: http://devcentral.f5.com/Default.aspx?tabid=63&articleType=ArticleView&articleId=105
SEE: http://devcentral.f5.com/Default.aspx?tabid=53&view=topic&forumid=5&postid=14001
SEE: http://devcentral.f5.com/Wiki/default.aspx/iRules/OpenSSO_authentication.html
I have two virtual servers (that share the same LB nodes) with this same rule attached
A) mdn_track 10.0.185.150:80 nodes 10.0.180.3 and 10.0.180.5
B) mdn_lookup 10.0.185.150:8002 nodes 10.0.180.3 and 10.0.180.5
*) Note, mdn_track is a web server and /prox_evdo/auth.gif returns a 1x1 gif
*) Note, mdn_track always returns "808111222 "
*) Note, on the F5 we have self IPs for 10.0.180.10&11 and 10.0.185.10&11
THIS REQ WORKS (mdn_lookup goes to port 8002, mdn_track goes to port 80)
http://10.0.185.150:8002/prox_evdo/auth.gif?rnd12345
: OOB mdn_lookup host <10.0.185.150:8002> mod-uri LB::server
: OOB mdn_lookup: response STAT <200> lbsvr stat up conlen <10>
: OOB mdn_lookup: STAT <200> VALID payload ... <8081112222 >, retry orig
: new req
: mdn_track host <10.0.185.150:8002> orig-uri LB::server
: mdn_track: response STAT <200> lbsvr stat up conlen <44>
THIS REQ FAILS (mdn_lookup goes to port 80, mdn_track never called) why doesn't "pool mdn_lookup" select the 8002 port
http://10.0.185.150/prox_evdo/auth.gif?rnd12345
: OOB mdn_lookup host <10.0.185.150> mod-uri LB::server
: OOB mdn_lookup: response STAT <404> lbsvr stat up conlen <271>
: OOB mdn_lookup: STAT <404> REJECT payload ... <
Question 1, Why do I have to define virtual servers (it will not work at all if I do not),
from the examples I though I only needed to reference pools ?
Question 2, Why does it only work http://10.0.185.150:8002/prox_evdo/auth.gif?rnd12345,
or why doesn't the right OOB port, 8002, get selected on the fetch from
http://10.0.185.150/prox_evdo/auth.gif?rnd12345?
when CLIENT_ACCEPTED {
set lookup 1
set valid_string "808"
}
when HTTP_REQUEST {
if { [HTTP::version] eq "1.1" } {
if { [HTTP::header is_keepalive] } {
HTTP::header replace "Connection" "Keep-Alive"
}
HTTP::version "1.0"
}
if {$lookup == 1} {
set def_server [LB::server]
set LB_request [HTTP::request]
inject lookup URI in place of original request;
HTTP::uri "/[IP::client_addr]"
the below never seems to update the port only works if initial port is 8002, it fails
to update the port to 8002 if the original request is 80
pool mdn_lookup
set oob_server [LB::server]
log local0. "OOB mdn_lookup host <[HTTP::host]> mod-uri <[HTTP::uri]> LB::server <$oob_server>"
} else {
log local0. "mdn_track host <[HTTP::host]> orig-uri <[HTTP::uri]> LB::server <$def_server>"
pool mdn_track
}
}
when HTTP_RESPONSE {
if {$lookup == 1} {
log local0. "OOB mdn_lookup: response STAT <[HTTP::status]> lbsvr <[LB::server]> stat [LB::status] conlen <[HTTP::header Content-Length]>"
if {[HTTP::header exists Content-Length] && \
([HTTP::header Content-Length] < 1048576)}{
set clength [HTTP::header Content-Length]
} else {
set clength 1048576
}
HTTP::collect $clength
} else {
log local0. "mdn_track: response STAT <[HTTP::status]> lbsvr <[LB::server]> stat [LB::status] conlen <[HTTP::header Content-Length]>"
}
}
when LB_FAILED {
log local0.debug "LB failed for: [LB::server]"
log local0.debug "LB status: [LB::status]"
}
when HTTP_RESPONSE_DATA {
if {[HTTP::payload] contains $valid_string}{
log local0. "OOB mdn_lookup: STAT <[HTTP::status]> VALID payload ... <[HTTP::payload]>, retry orig"
the below always seems to update the port to 80 as expected, evenif the original port was 8002
pool mdn_track
log local0. "new req <$LB_request>"
HTTP::retry $LB_request
} else {
log local0. "OOB mdn_lookup: STAT <[HTTP::status]> REJECT payload ... <[HTTP::payload]>"
reject
}
set lookup 0
}
Hi Jon,
How did you have the VIP defined when were you trying to add the iRule to a 0.0.0.0 VIP with a pool? Was it a Performance / FastL4 VIP? If so, can you try with a standard IP 0.0.0.0 VIP instead?
Also, can you move this line from HTTP_REQUEST to CLIENT_ACCEPTED and retest?
set def_server [LB::server]
LB::server will return the currently selected pool--not necessarily the VIP’s default pool. The value is only reset to the default pool value on a new TCP connection.
Thanks,
Aaron
Hi Jon,
> How did you have the VIP defined when were you trying to add the iRule to a 0.0.0.0 VIP with a
> pool? Was it a Performance / FastL4 VIP? If so, can you try with a standard IP 0.0.0.0 VIP instead?
1. I have a main virtual server “vs_main” that gets all non-http traffic it is (performance layer-4)
no profiles but a single iRULE showing debug if needed on just ‘connections’
2. I have a focused virtual server “vs_main_http” that gets port 80 traffic (but in the above posting
NOT 8002 traffic) it is (type: Standard - with a HTTP/httpclass profile
-- Perhaps when I select the virtual server for :8080 it goes to vs_main (and there is NO
HTTP/httpclass profile) this might be my entire issue.
It also might explain why “I needed to create virtual servers with HTTP/httpclass profile’s”
on top of my LB node pairs to get anything to work at all.
> Also, can you move this line from HTTP_REQUEST to CLIENT_ACCEPTED and retest?
> set def_server [LB::server]
I retested (moving the def_server line) no luck - BUT the key was to enable
"port translation" on the VIP - thanks to your pointer (different post).
Obviously I don't fully grasp all the concepts of the F5 there but I am learning
> LB::server will return the currently selected pool--not necessarily the VIP’s default pool.
> The value is only reset to the default pool value on a new TCP connection.
Interesting, I still have one question how does one “force” the VIP’s default pool
Thanks in Advance
Jon
Hi Jon,
Let me know how the testing goes with the LB::server command moved to the CLIENT_ACCEPTED event. That is the way to force the use of the VIP’s default pool--save the name of the default pool before it’s been changed on the TCP connection using the LB::server command in the CLIENT_ACCEPTED event and select it using ‘pool $def_pool’ in the rest of the iRule.
Aaron
We route requests to destination servers based upon a value they have in an LDAP server. In order to do this with F5, we created a lookup script which will return the pertinent values in a header. The iRule then parses this header and will route based upon which application the web service is intended for. Upon testing it, we figured out that the default HTTP:retry was not sending the original payload. In order to fix this, we had to append the original payload to the original headers. I have pasted the iRule below, has anyone else seen this issue or know of another way to get around it?
when CLIENT_ACCEPTED {
set flag to control logical flow. 1 means lookup is pending.
set lookup 1
}
when HTTP_REQUEST {
HTTP::version “1.0”
set param “ID”
set search_string “Portal”
set default_pod “p_webservice_default”
if {$lookup eq 1} {
set OID [HTTP::header $ID]
if {OID ne “” } {
log local0. “LookUp Log: ID found: $OID”
save the payload, trigger collection for up to 1MB of data
if {[HTTP::header exists “Content-Length”] && [HTTP::header “Content-Length”] <= 1048576}{
set content_length [HTTP::header “Content-Length”]
} else {
decide what to do here, throw soap fault maybe?
drop
}
Check if $content_length has been set and is not set to 0
if { [info exists content_length] && $content_length > 0} {
HTTP::collect $content_length
}
save original request headers
set origRequest [HTTP::request]
log local0. “LookUp Log: Original Request is: $origRequest”
HTTP::header replace “ID” $OID
inject lookup URI in place of original request
HTTP::uri “/lookup.php”
Route to the lookup service
node x.x.x.x 80
}
} else {
try to route to appropriate pod
if {![catch {pool p_webservice_$resolved_pod}]} {
log local0. “Web Service Log: Routing finally to p_webservice_$resolved_pod”
} else {
let the request go to default pod
log local0. “Web Service Log: Routing to default pod (reason was $resolved_pod)”
pool $default_pod
}
}
}
when HTTP_REQUEST_DATA {
if {($lookup eq 1) and ($OID ne “”)}{
save the payload
set originalPayload [HTTP::payload]
HTTP::release
}
}
when HTTP_RESPONSE {
HTTP::version “1.0”
if {($lookup eq 1) and ($OID ne “”) }{
set lookup 0
log local0. “LookUp Log: Response from lookup received, LBGroup: [HTTP::header LBGroup], OID: [HTTP::header OID_Sent]”
set full_header [HTTP::header LBGroup]
switch -glob $full_header {
“ERROR” {
ERROR CONDITIONS AND RESPONSES
}
default {
set resolved_pod [findstr $full_header $search_string 11 “;”]
log local0. “LookUp Log: Routing to resolved pod: $resolved_pod, and length is [string length $resolved_pod]”
Make sure this OrgOID has a POD for this product ($search string)
if {[string length $resolved_pod] > 0} {
verify pool exists and has members
if {![catch [pool p_webservice_$resolved_pod]]}{
Retry with the original request
log local0. “LookUp before retry Log: Original Payload is: $originalPayload”
append origRequest $originalPayload
log local0. “LookUp before retry Log: Request + Payload is: $origRequest”
HTTP::retry $origRequest
} else {
POD is invalid or was not defined in F5 pools
log local0. “Lookup Log Critical Error PODNOTFOUNDINF5: OrgID is $ADP_ORGOID has a valid RG $resolved_pod with no F5 pool defined”
set soapFaultPodNotFound “Inv…de>Invalid Resource GroupLDAP Search failedWeb Tier F5 Gateway”
HTTP::header replace “Content-Type” “text/xml”
HTTP::payload replace 0 [HTTP::header value Content-Length] $soapFaultPodNotFound
}
} else {
this is odd ball case where lookup would not return a pod or know error code
review if we need it or we can reject request here
set resolved_pod “INVALIDCODE”
append origRequest $originalPayload
HTTP::retry $origRequest
}
}
}
}
}
when HTTP_RESPONSE_DATA {
set lookup 0
}
when LB_SELECTED {
log local0. “*** Web Service Log: Selected [LB::server name]:[LB::server port]”
}
HTTP::request only contains the HTTP headers in a request. If you want to retry a request with the original payload you would need to collect the payload and append it to the request you’re retrying.
So what you’re seeing is expected.
Aaron