Forum Discussion
Krzysztof_Kozlo
Nimbostratus
NimbostratusFetching multiple headers
We are attempting to use Universal Persistence to work around source IP addresses being obscured by intervening infrastructure (i.e. proxy servers). The proxy servers in our environment insert an X-Forwarded-For header, as does the BigIP. Normally, the value of the header we wish to base universal persistence on is that of the first X-Forwarded-For.
However, the BigIP does not appear to allow you to fetch the first header using HTTP::header if a subsequent header appears. (Please don't tell me that the HTTP RFC does not permit multiple headers with the same field-name, the BigIP itself will create a new header if "Insert X-Forwarded-For" is enabled instead of appending to the existing header.)
I can't even fetch headers by index, which would allow me to iterate the header-names, increment a variable, and retrieve the header -- HTTP::header at n returns only the header name, not the header value.
The iRules quick reference seems to suggest that HTTP::request and HTTP::collect will somehow return the entire HTTP request, which would also do the job, but I can't get these working.
HTTP::collect
set x [HTTP::payload]
this returns null
HTTP::request doesn't even compile.
The only way I've found to get what I want so far is to TCP::collect when CLIENT_ACCEPTED and then parse the TCP::payload. There has to be a better way.
Incidentally I notice several others asked about how to get at the nth Set-Cookie or Accept header, and when the initial responses proved unhelpful or incomplete, the questions seemed to die on the vine. As I am often encouraged to search through or post my questions in DevCentral, I harbor some hope these are the exception rather than the norm.
9 Replies
- Yes, we are aware of that limitation and no, we are not going to saying anything like the RFC doesn't allow that (because it quite specifically does). The problem is more of an implementation detail. We hope to eventually get it fixed. It might help to make a call into support so a Case file gets created and tracked. This will help raise the priority of such a requested feature enhancement. Thanks.
Krzysztof_KozloThanks for the fast response. I suppose HTTP::request and HTTP:collect on HTTP_REQUEST events are not implemented in 9.1.1 which we run here.
I find myself wishing there was an LTM guide for 9.1 available as there are definitely some features in 9.1 that don't exist in 9.0 and yet reading the 9.2 guide one has to simply experiment to see what works and what doesn't.
Is there any way to specify a character class or a set of characters to terminate the findstr command? Even the TCP::collect kluge I proposed above won't work properly unless I can specify the findstr term character being either a comma or a space/CR, as I need to retrieve the first IP following the first XFF header and that can be one-per-line or a set of comma-separated values following a single field-name.- I can't speak for the ltm guides, but the wiki documentation for the iRule commands contain an "Introduced" field that lists the product version they were first available. In the case of HTTP::request, it states BIGIP-9.2.0.
-Joe - Ok, we will give the people what they want. We are considering adding the follwing new rule.
HTTP::header values
This rule will return all header values as a TCL list (same order they appear in the header). This will most probably make it out on the next release of BigIP V9.4. - Krzysztof_KozloThat would be great, as the current HTTP::header functions are simply inadequate for application in a high-volume heterogenous production environment. It's hard enough to persuade owners of client applications to adhere to RFCs without having to ask them to further limit their implementation to i.e. unique field-names.
The irony here is that the BigIP steps on its own tail when you ask for the X-Forwarded-For header -- it supplies you with the one it just inserted! - Then why don't you remove the X-Forwarded-for option from the http profile and insert it in the rule after you retreive the upstream header value. That might allow you to get around this for the time being.
- Krzysztof_KozloGood idea, but I can't guarantee that there's not an upstream BigIP (i.e. in front of the web proxies) or another device that already added an XFF header, obscuring the original one to the BigIP in question.
hoolioCirrostratus
I ran into this issue of getting the values for headers with identical names (mainly worrying about Set-Cookie). It looks like the 'HTTP::header values ' command Bl0ndie mentioned has been added in 9.4:when HTTP_RESPONSE { insert some test headers with the same name HTTP::header insert header_1 value_1 HTTP::header insert header_1 value_2 HTTP::header insert header_1 value_3 test a few options for parsing the headers log local0. "\[HTTP::header header_1\]: [HTTP::header header_1]" log local0. "\[HTTP::header value header_1\]: [HTTP::header value header_1]" log local0. "\[HTTP::header values header_1\]: [HTTP::header values header_1]" }
Log output:[HTTP::header header_1]: value_3 [HTTP::header value header_1]: value_3 [HTTP::header values header_1]: value_1 value_2 value_3
But I'm wondering if multiple headers with the same name could be handled better.
With the 'values' command, I can actually get the values of headers that exist multiple times (which is a good improvement over past versions). But I have to check to see if a header name exists more than once and then parse the list of values.
Could a method for getting the values of the headers by their index be added? Before I saw this post, I assumed that the 'HTTP::header at $index' command would return the value of a header at $index. I'm not sure what use there is in getting the header name at a specific index. Maybe 'HTTP::header at $index' could be changed to return the value of the header? Else, maybe a new command could be added, like 'HTTP::header value at $index'.
I'll submit an RFE on this, but I figured I'd check here first to find out more on the logic for the existing functionality.
Thanks,
Aaron- Krzysztof_KozloAgreed that this would be a useful addition, particularly if you could get the size of the index for a given header and calculate that to grab the nth one.
