LTM HTTP Profile Option: Response Chunking

The HTTP profile contains an option to control HTTP response chunking, and it's not exactly obvious why you'd choose one over the other.

This article gives a high level explanation of HTTP chunking, and (with lots of input from unRuleY) attempts to dispel the mystery and capture the subtle nuances of each response chunking mode.

What is HTTP Response Chunking?

Chunking is a message transfer encoding scheme for HTTP that enables dynamic content length management to help reduce latency.

The introduction of chunked transfer encoding in HTTP 1.1 allows the server to begin sending data to the client before the content length is known, and in some cases before the entire response has been generated by the application.

Chunked Transfer-Encoding

Chunked transfer encoding modifies the body of a message in order to transfer it as a series of chunks.

In chunked responses, the Content-Length header is replaced by a Transfer-Encoding header with a value of "chunked", and chunks of data are sent to the client as they become available. (You can identify a chunked response because it contains the "Transfer-Encoding: chunked" header.)

The chunked response is a normal HTTP response beginning with HTTP headers, followed by 2 consecutive CR/LF's, then the message body. A complete chunked message body contains a series of data chunks ending with a 0 length chunk. Footers may follow the message body, and the chunked message is terminated by 2 consecutive CR/LF's. Once the end of the chunked transaction is seen, the connection is available for re-use.

Each data chunk consists of 4 parts:

• a header indicating the size of the chunk data in hex (and possibly other non-standard parameters you can safely ignore)
• <CR><LF>
• a chunk of the payload data
• <CR><LF>

Here's an unchunked server response:

 HTTP/1.1 200 OK
 Date: Fri, 31 Dec 1999 23:59:59 GMT 
 Content-Type: text/plain
 Content-Length: 42<CR><LF>
 <CR><LF>
 abcdefghijklmnopqrstuvwxyz1234567890ABCDEF 

And the same response using chunked transfer encoding:

 HTTP/1.1 200 OK
 Date: Fri, 31 Dec 1999 23:59:59 GMT
 Content-Type: text/plain
 Transfer-Encoding: chunked<CR><LF>
 <CR><LF>
 1a; non-std-params-here<CR><LF>
 abcdefghijklmnopqrstuvwxyz<CR><LF>
 10<CR><LF>
 1234567890ABCDEF<CR><LF>
 0<CR><LF>
 <CR><LF> 

The response data may be split into any number of chunks of any size. The chunks can contain any binary data, and may be much larger than the examples here.

Chunking Options in the LTM HTTP Profile

The HTTP profile contains a "Response Chunking" option that can be set to 4 different modes depending on how server responses should be encoded. The effect of each mode depends on whether the server response is sent using chunked encoding. This section should help you choose the best response chunking setting for your application.

Summary of the 4 response chunking modes:

Each mode has a different functional effect on chunked and unchunked server responses. Here are a few more details about how each mode affects each type of response.

Unchunk

Unchunk mode will send all content unchunked regardless of whether the server sent it chunked or unchunked.

For content that is chunked, Unchunk buffers the response, removing the HTTP transfer encoding headers and chunk headers, processes the HTTP content, and then sends the unchunked response to the client. The client connection will be closed once the response is completed.

For content that is already unchunked, Unchunk handles the response as a normal HTTP transfer: It buffers the response, processes the HTTP content and sends the response unchunked.

Unchunk mode is only recommended when you need to force unchunked responses for all clients accessing a virtual server. Since the process of unchunking a response requires LTM to buffer the entire response before sending it, using Unchunk is not optimal for normal webserver/browser traffic. nchunk should not be used if LTM will be modifying the payload length.

Rechunk

Rechunk mode will re-chunk all content regardless of whether the server sent it chunked or unchunked.

For content that is already chunked, Rechunk unchunks the HTTP content, then as it processes the data, creates and sends new chunks to the client. For optimization, this mode attempts to preserve the original chunks, but new chunks may be different sizes than the original chunks, especially if the original response was modified.

For content that is unchunked, Rechunk buffers the response and processes the HTTP content, adds the transfer encoding and chunk headers to the response, and then sends a newly chunked response.

Rechunk mode (or Selective) is recommended if you are using any other feature that modifies the length of the original server response (such as HTTP compression, or iRules / stream profile modifying HTTP payload). While your server may only chunk some responses, chunking all responses will help optimizes data transfer through multiple proxy hops.

Selective

Selective mode will re-chunk chunked content, and leave unchunked content alone.

For content that is chunked, Selective is the same as Rechunk: It unchunks the HTTP content, then as it processes the data, creates and sends new chunks to the client, preserving the original chunks if optimal to do so.

For content that is unchunked, Selective mode is the same as Unchunk: It processes the HTTP content and sends the response unchunked.

Selective mode is recommended for most applications. Selective mode (or Rechunk) is recommended if you are using any other feature that modifies the length of the original server response (such as HTTP compression, or iRules / stream profile modifying HTTP payload).

Preserve

Preserve mode will preserve the original transfer encoding, changing nothing at all regardless of whether the server sends chunked or unchunked data.

For content that is chunked, Preserve processes the HTTP content and as it is processed, sends the response to the client with the same chunk boundaries.

For content that is unchunked, Preserve buffers the response, processes the HTTP content and sends the response unchunked.

Preserve mode is recommended for troubleshooting or when chunk size errors can't be resolved, and should not be used if LTM will be modifying the payload length.