Forum Discussion

Evan_25555's avatar
Evan_25555
Historic F5 Account
Oct 19, 2011

Problem with iRule that logs excessive HTTP header lengths

Background:

 

 

We had been seeing occasional errors in /var/log/ltm that look like this:

 

 

HTTP head (33182) exceeded maximum allowed size of 32768

 

 

Before increasing MAX HTTP header size across the board, we thought we'd increase the MAX Header size to 64k (doubling it), on one of our VIPs and create an iRule to log HTTP Headers when the total length of them exceeds 32768.

 

 

When the threshold is set much higher than 15000, we see TMM errors associated with the iRule( http-header-size) that look like this:

 

 

Oct 19 09:16:31 local/tmm3 err tmm3[6715]: 01220001:3: TCL error: http-header-size - Not found (line 1) invoked from within "HTTP::header names"

 

 

When the threshold is set low, the iRule works correctly and we see log entries that look like this:

 

 

Oct 19 12:29:17 local/tmm5 info tmm5[6717]: Rule http-header-size : Header length = 1311 Oct 19 12:29:17 local/tmm5 info tmm5[6717]: Rule http-header-size : Accept (164): image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-powerpoint, application/vnd.ms-excel, application/msword, */* Oct 19 12:29:17 local/tmm5 info tmm5[6717]: Rule http-header-size : Referer (33):

 

 

and so forth

 

 

Can anyone offer any insights as to why the iRule fails when the logging threshold is set above 10000 or so?

 

 

 

Here's the iRule - please forgive the formatting

 

 

when HTTP_REQUEST {

 

Loop through each header by name

 

set count 0

 

foreach aHeader [HTTP::header names] {

 

incr count [string length [HTTP::header value $aHeader]]

 

}

 

if {$count > 32768} {

 

Log details for the request

 

log local0. "Header length = $count"

 

foreach aHeader [HTTP::header names] { log local0. "$aHeader ([string length [HTTP::header value $aHeader]]): [HTTP::header value $aHeader]"

 

}

 

}

 

}

 

 

 

application delivery
dev
devops
iRules
  • Michael_Yates's avatar
    Michael_Yates
    Icon for Nimbostratus rankNimbostratus
    Oct 19, 2011
    Hi Evan,

     

     

    You may want to take a look at this solution:

     

    SOL8482: Error Message: HTTP header exceeded maximum allowed size of

     

    http://support.f5.com/kb/en-us/solutions/public/8000/400/sol8482.html?sr=17162762

     

     

     

    Note: F5 recommends that you allocate just enough bytes required to handle the HTTP headers. Since buffer space is allocated to each request handled by the profile, an excessive value may result in memory exhaustion.

     

  • Evan_25555's avatar
    Evan_25555
    Historic F5 Account
    Oct 19, 2011
    Thanks, Michael. I should have mentioned that the objective was to increase MAX HTTP Header size temporarily so we could capture some of the excessively long headers and have our development team determine if they were valid. If they're valid, we would increase MAX HTTP Header size across the board.
  • Patrick_Chang_7's avatar
    Patrick_Chang_7
    Historic F5 Account
    Oct 20, 2011
    In F5 land, the HTTP headers also include the initial request line (method and URI). You could do the following

     

    when HTTP_REQUEST {

     

    set hlength [string length [HTTP::query]]

     

    if { $hlength > 32768 } {

     

    log local0.info "Header length = $hlength, [HTTP::query]"

     

    }

     

    }
  • nitass's avatar
    nitass
    Icon for Employee rankEmployee
    Oct 20, 2011
    i think HTTP::query does not include method and full uri. do i misunderstand?

     

     

    HTTP::query wiki

     

    http://devcentral.f5.com/wiki/iRules.HTTP__query.ashx
  • Evan_25555's avatar
    Evan_25555
    Historic F5 Account
    Oct 20, 2011
    Follow-up

     

     

    Turns out my iRule did exactly what it was supposed to-namely log data regarding HTTP requests with excessively long HTTP headers. Not sure what those TMM errors were all about. In any case I have the data I need and appreciate everyone's help!
  • Evan_25555's avatar
    Evan_25555
    Historic F5 Account
    Oct 20, 2011
    Follow-up

     

     

    Turns out my iRule did exactly what it was supposed to-namely log data regarding HTTP requests with excessively long HTTP headers. Not sure what those TMM errors were all about. In any case I have the data I need and appreciate everyone's help!
  • PinkieCom's avatar
    PinkieCom
    Icon for Nimbostratus rankNimbostratus
    Dec 07, 2015

    A lot late to the party, but I needed to do the same.

    I noticed one thing, the total header size calculation is slightly wrong. It only counts the total for the header values, but doesn't include the header names themselves.

    For instance, the header “Connection: keep-alive”, it was only counting the string length of "keep-alive" it was counting that as 10 bytes and not the correct 22 bytes for that actual line of header, if you have a lot of header with long names and short values it can throw your count by a reasonable amount.

    You need to add together the string length of the header name, the string length of the header value, plus 2 (this account for the ": " between the header name and value.

    In our case we have upped the max header value to 64k, and we're using this to work out what the actual maximum is.

    So to remedy:

    when RULE_INIT {
    set static::Header_Alert_Size 32768
    }

when HTTP_REQUEST {
        set header_total 0
        foreach header [HTTP::header names] {
            incr header_total [string length $header]
            incr header_total 2
            incr header_total [string length [HTTP::header value $header]]
        }
        if { ($header_total > $static::Header_Alert_Size) } {
                set LogString "Client [IP::client_addr]:[TCP::client_port] -> [HTTP::host][HTTP::uri]"
                log local0. "$LogString - Header Total $header_total bytes"
        }
    }