http_request payload logging is not full

Hi Ravi,

The HTTP_RESPOND event is triggered once TMM has collected enough packets to get the last HTTP header. If you call [HTTP::payload] in HTTP_RESPONSE, it will only return the HTTP payload available in the packet(s) required to parse all of the HTTP headers. If you want to collect the full HTTP payload, you'll need to use HTTP::collect:

 http://devcentral.f5.com/wiki/default.aspx/iRules/http__collect
when HTTP_RESPONSE {

   Trigger collection for up to 1MB of data
  if {[HTTP::header exists "Content-Length"] && [HTTP::header "Content-Length"] <= 1048576}{
    set content_length [HTTP::header "Content-Length"]
  } else {
      set content_length 1048576
  }
   Check if $content_length has been set and is not set to 0
  if { $content_length > 0} {
    HTTP::collect $content_length
  }
}
when HTTP_RESPONSE_DATA {
   do stuff with the payload
  set payload [HTTP::payload]
}

Aaron

Hi Aaron,

 

 

Thanks for your response. I tried executing your above code but it do not do the required.

 

 

Still the entire request is not being logged. I read the link on HTTP::collect. It saya that if we use it with specifying any content_length arguement to it it will print the entire payload. but that seems to hang my client.

 

 

 

 

what i observere is the entire payload is chunked and sent. the above code prints the first chunk and exits of.

 

 

 

thanks,

 

Ravi

 

 

 

 

Hi Ravi,

You can set the request HTTP version to 1.0 to prevent the server from sending chunked responses:

when HTTP_REQUEST {

    Prevent the server from sending a compressed response
    remove the compression offerings from the client
   HTTP::header remove "Accept-Encoding"

    Don't allow response data to be chunked
   if { [HTTP::version] eq "1.1" } {

       Force downgrade to HTTP 1.0, but still allow keep-alive connections.
       Since HTTP 1.1 is keep-alive by default, and 1.0 isn't,
       we need make sure the headers reflect the keep-alive status.

       Check if this is a keep alive connection
      if { [HTTP::header is_keepalive] } {

          Replace the connection header value with "Keep-Alive"
         HTTP::header replace "Connection" "Keep-Alive"
      }

       Set server side request version to 1.0
       This forces the server to respond without chunking
      HTTP::version "1.0"
   }
}

Aaron

Hi Aaron,

 

 

Below is the complete irule i have with your suggestions:

 

 

when HTTP_REQUEST {

 

if {[HTTP::header value Content-Type] contains "application/vnd.syncml+xml"}{

 

 

Prevent the server from sending a compressed response

 

remove the compression offerings from the client

 

HTTP::header remove "Accept-Encoding"

 

 

Don't allow response data to be chunked

 

if { [HTTP::version] eq "1.1" } {

 

 

 

Force downgrade to HTTP 1.0, but still allow keep-alive connections.

 

Since HTTP 1.1 is keep-alive by default, and 1.0 isn't,

 

we need make sure the headers reflect the keep-alive status.

 

 

Check if this is a keep alive connection

 

if { [HTTP::header is_keepalive] } {

 

 

Replace the connection header value with "Keep-Alive"

 

HTTP::header replace "Connection" "Keep-Alive"

 

}

 

 

Set server side request version to 1.0

 

This forces the server to respond without chunking

 

HTTP::version "1.0"

 

}

 

 

Trigger collection for up to 1MB of data

 

if {[HTTP::header exists "Content-Length"] && [HTTP::header "Content-Length"] <= 1048576}{

 

set content_length [HTTP::header "Content-Length"]

 

}

 

 

Check if $content_length has been set and is not set to 0

 

if { [info exists content_length] && $content_length > 0} {

 

HTTP::collect $content_length

 

log local0. " content_length: $content_length"

 

}

 

}

 

}

 

 

 

 

when HTTP_REQUEST_DATA {

 

do stuff with the payload set payload [HTTP::payload]

 

log local0. " HTTP::payload: [HTTP::payload]"

 

}

 

 

 

 

 

But still i don't get the complete payload logged. please advice.

 

 

Thanks,

 

Ravi

 

 

 

 

 

 

 

Hi Ravi,

 

 

Sorry, I completely missed that you're talking about the request payload. I don't know of a way to collect a chunked request payload. If you have control over the client, you could disable chunking.

 

 

Anyone else have ideas? If not, I suggest you open a case with F5 Support and ask them to consider supporting a method to unchunk request payloads.

 

 

Aaron

Hi,

 

 

dont forget Syslog. If you try to log more than 1022 bytes, syslog will cut your message.

 

Try to split and log your payload or configure your syslog to log more than 1014 bytes.

 

 

Regards,

 

 

Sören