Forum Discussion

chiewming_15294's avatar
chiewming_15294
Icon for Nimbostratus rankNimbostratus
Jan 18, 2012

HTTP Profile "Maximum Header Size"

Hi All,

 

 

 

Our platform is intended to act as a proxy or transparent proxy to serve ONLY http traffic from handset or laptop. (Non http traffic bypassed at F5 level)

 

 

 

After we putting live production traffic to platform, there are lots of HTTP header size error logged and causing the F5 stop sending traffic to one of the pool member.

 

We tried to increase the size to 64KB and it's error still keep logging in /var/log/ltm.

 

To avoid downtime, we had the "Maximum Header Size" disabled in HTTP profile.

 

Please see below for the error log.

 

 

History log:

 

Jan 4 23:58:53 local/tmm err tmm[5238]: 011f0005:3: HTTP header (33304) exceeded maximum allowed size of 32768 (Client side: vip=ext_HTTP_VS_80 profile=http pool=Pool_3128)

 

Jan 4 23:58:54 local/tmm1 err tmm1[5239]: 011f0005:3: HTTP header (33304) exceeded maximum allowed size of 32768 (Client side: vip=ext_HTTP_VS_80 profile=http pool=Pool_3128)

 

Jan 4 23:58:55 local/tmm3 err tmm3[5241]: 011f0005:3: HTTP header (33304) exceeded maximum allowed size of 32768 (Client side: vip=ext_HTTP_VS_80 profile=http pool=Pool_3128)

 

Jan 4 23:58:55 local/tmm err tmm[5238]: 011f0005:3: HTTP header (33304) exceeded maximum allowed size of 32768 (Client side: vip=ext_HTTP_VS_80 profile=http pool=Pool_3128)

 

 

I need advice on inquiry below:

 

1.Is this a critical messages?

 

2.If particular VS hitting too much of this error, what is the expected F5 behavior? stop sending traffic to one of the pool member?

 

3.Need input from your security point of view, what value is best approach?

 

4.If we disable “Maximum Header Size” checking, will it cause security issue? Like DOS attack?

 

Thanks,

 

chiewming

 

 

 

 

 

 

  • Hi Chiewming,

    See below for more info:

    1.Is this a critical messages?

    When this error occurs, TMM resets the client connection.

    2.If particular VS hitting too much of this error, what is the expected F5 behavior? stop sending traffic to one of the pool member?

    See above. There is no impact on the pool members. TMM simply resets the client connection. Increasing the max headers size in the HTTP profile will force TMM to allocate more memory to store the HTTP headers though.

    3.Need input from your security point of view, what value is best approach?

    I'd increase the max size in the HTTP profile and use an iRule to log the request details. You can then determine whether the request is malicous, from a poorly designed client or incorrect web app behavior.

    4.If we disable “Maximum Header Size” checking, will it cause security issue? Like DOS attack?

    This setting can't be disabled, but you could set it to a very large value. I wouldn't recommend this though as you'll eat up more TMM memory.

    Here's an iRule you can use to log long header values. Note that the HTTP profile setting for Max HTTP headers size must be greater than the actual request's headers in order to avoid being reset and trigger the HTTP_REQUEST event. This version checks the total headers size before looping through each header.

    
    when HTTP_REQUEST {
    
     Check the total HTTP headers size
    if {[string length [HTTP::request]] > 10000 }{
    
     Check if the URI is very long
    if {[string length [HTTP::uri]] > 1000}{
    
    log local0. "Uri is long. Length [string length [HTTP::uri]], URI: [HTTP::uri]"
    
     Exit this event from this iRule
    return
    }
    
     Loop through the headers by name
    foreach header {[HTTP::header names]} { 
    
     Check for a long header value
    if {[string length [HTTP::header value $header]] > 1000 } { 
    log local0. "Header is long. Header Name: $header,\
    Length: [string length [HTTP::header value $header]], Value: [HTTP::header value $header]" 
     Exit this event from this iRule
    return
    }
    }
    
     If we are still here the request was large, but not the URI or any single header.
     Log the first 1k bytes of the full request
    log local0. "Request is long: [HTTP::request]"
    }
    }
    

    Aaron
  • Hi Aaron,

     

     

    Thanks for your prompt reply.

     

     

    I need some advice on question below :

     

    1)When you say TMM reset the connection, it's mean client need resend http request again?

     

    2)Is "Maximum Header Size" applicable to only client http request? other http activity?

     

    3)TMM check on the packet's "Header length" for the size or TMM does calculate on the header size?

     

    4)Regarding your reply on question (4), it thought it can be disabled by uncheck the check box in Local Traffic -> Profiles -> Services -> HTTP -> *my_http_profile* ?

     

    5)What could be the cause of larger http header size other than web application/URI behavior?

     

    6) I read on some F5 article saying that DOS attack could make use of this to create spamming if value set too high.

     

     

    Thanks in advance.

     

     

    Regards,

     

    chiewming

     

     

  • 1) When you say TMM reset the connection, it's mean client need resend http request again?

     

    TMM sends the client a TCP reset. This would result in a failure of the current HTTP request and the client would need to hit refresh to resubmit the request. If they send the same request with a large HTTP header set, they would receive another reset.

     

     

    2) Is "Maximum Header Size" applicable to only client http request? other http activity?

     

    The max header size is checked for the HTTP request headers only as far as I'm aware.

     

     

    3) TMM check on the packet's "Header length" for the size or TMM does calculate on the header size?

     

    TMM calculates it based on the start of the HTTP headers up until the first \r\n\r\n sequence which ends the headers.

     

     

    4) Regarding your reply on question (4), it thought it can be disabled by uncheck the check box in Local Traffic -> Profiles -> Services -> HTTP -> *my_http_profile* ?

     

    No. That would just use the default HTTP profile's value for max header size.

     

     

    5) What could be the cause of larger http header size other than web application/URI behavior?

     

    It could be a malicious client, a poorly designed client or incorrect web app behavior.

     

     

    6) I read on some F5 article saying that DOS attack could make use of this to create spamming if value set too high.

     

    Yes, but I don't think it's much of a risk if you temporarily raise the limit even to 64KB in order to log details of the request and fix the issue.

     

     

    Aaron
  • Great. Thanks Aaron...

     

     

    Another question :(

     

     

    Currently,we have a few iRule associate with the virtual server and average is about 25% CPU load.

     

    Will this iRule cause additional check for each and every request and increase te CPU load?

     

     

    Any pros or cons you can think off other then the resource loading?

     

     

    Thanks,

     

    chiewming
  • Hi Aaron,

     

     

    I did some modification on the iRule provided earlier to check only on header size is larger than 65536 bytes else do nothing.

     

    The reason is if http request length more than 10000 doesn't mean it's more than 32 or 64KB. Please correct me if I'm wrong.

     

     

    Please advice on iRule below:

     

    when HTTP_REQUEST {

     

    set hlength [string length [HTTP::query]]

     

    if { $hlength > 32768 } {

     

    Check the total HTTP headers size

     

    if {[string length [HTTP::request]] > 10000 }{

     

     

    Check if the URI is very long

     

    if {[string length [HTTP::uri]] > 1000}{

     

     

    log local0. "Uri is long. Length [string length [HTTP::uri]], URI: [HTTP::uri]"

     

     

    Exit this event from this iRule

     

    return

     

    }

     

     

    Loop through the headers by name

     

    foreach header {[HTTP::header names]} {

     

     

    Check for a long header value

     

    if {[string length [HTTP::header value $header]] > 1000 } {

     

    log local0. "Header is long. Header Name: $header,\

     

    Length: [string length [HTTP::header value $header]], Value: [HTTP::header value $header]"

     

    Exit this event from this iRule

     

    return

     

    }

     

    }

     

     

    If we are still here the request was large, but not the URI or any single header.

     

    Log the first 1k bytes of the full request

     

    log local0. "Request is long: [HTTP::request]"

     

    }

     

    }

     

    }

     

     

    Thanks,

     

    chiewming
  • Currently,we have a few iRule associate with the virtual server and average is about 25% CPU load.

    Will this iRule cause additional check for each and every request and increase te CPU load?

    I don't think this iRule will add that much load. But to be safe, you could only use it temporarily during a low traffic period to log details on large requests. Once you get some data logged, you can disable the iRule.

    Any pros or cons you can think off other then the resource loading?

    I'm not sure what you mean. Can you rephrase this question?

    As for your edit: If you want to only check if the total headers are over 32k, you can use this:

    
    when HTTP_REQUEST {
         Check the total HTTP headers size
        if {[string length [HTTP::request]] > 32768}{
             Check if the URI is very long
            if {[string length [HTTP::uri]] > 1000}{
                log local0. "Uri is long. Length [string length [HTTP::uri]], URI: [HTTP::uri]"
                 Exit this event from this iRule
                return
            }
             Loop through the headers by name
            foreach header {[HTTP::header names]} { 
                 Check for a long header value
                if {[string length [HTTP::header value $header]] > 1000 } { 
                    log local0. "Header is long. Header Name: $header,\
                        Length: [string length [HTTP::header value $header]], Value: [HTTP::header value $header]" 
                     Exit this event from this iRule
                    return
                }
            }
             If we are still here the request was large, but not the URI or any single header.
             Log the first 1k bytes of the full request
            log local0. "Request is long: [HTTP::request]"
        }
    }

    Aaron
  • Rephrase my second question:

     

    Any disadvantage of this iRule?

     

     

    I have talk to an engineer regarding to implement this iRule, I need you advice as he mentioned:

     

    1)When thought about this logically you shouldnt be able to analyze a http request exeeding the maximum header size allowed using an iRule, when maximum http header size is exeeded F5 simply drop that request and send a RST to the client.

     

    2)Not able to analyze such requests using an irule. because if F5 has the capablity of analyzing HTTP requests exeeding the maximum header size, whats the point of defining a maximum header size in the first place.

     

     

    Would like to check with you on the traffic flow in F5,

     

    1)when traffic reach F5, it will checked by iRule.

     

    2) iRule will do necessary iRule check. reject/forwarding if specified in iRule.

     

    3)Traffic reaches Virtual Server which associate with HTTP profile.

     

    4) HTTP profile will then check on the HTTP header size and it will decide allow or send RST to client.

     

     

    Am I correct? :)

     

     

    Thanks in advance.

     

    chiewming

     

     

     

    Thanks,

     

    chiewming
  • 1)When thought about this logically you shouldnt be able to analyze a http request exeeding the maximum header size allowed using an iRule, when maximum http header size is exeeded F5 simply drop that request and send a RST to the client.

     

     

    That's correct. You need to temporarily increase the HTTP profile setting for max headers size so that TMM will accept the HTTP request and trigger the HTTP_REQUEST event. The setting for triggering the iRule needs to be lower than the max headers size in the HTTP profile.

     

     

    2)Not able to analyze such requests using an irule. because if F5 has the capablity of analyzing HTTP requests exeeding the maximum header size, whats the point of defining a maximum header size in the first place.

     

     

    See above.

     

     

    Would like to check with you on the traffic flow in F5,

     

    1)when traffic reach F5, it will checked by iRule.

     

    2) iRule will do necessary iRule check. reject/forwarding if specified in iRule.

     

    3)Traffic reaches Virtual Server which associate with HTTP profile.

     

    4) HTTP profile will then check on the HTTP header size and it will decide allow or send RST to client.

     

     

    That's not completely accurate. The order is:

     

     

    TMM receives an HTTP request

     

    TMM determines the size of the HTTP headers

     

    If the size of the headers is less than the max headers size from the HTTP profile, the HTTP_REQUEST event is triggered

     

    The iRule logic from HTTP_REQUEST is triggered.

     

     

    If the size of the headers is greater than the max headers size from the HTTP profile, the HTTP_REQUEST event is not triggered and TMM resets the client's TCP connection.

     

     

    Aaron