The State Of HTTP/2 With F5 LTM
In this article, I will attempt to summarize the known challenges of an HTTP/2 full proxy setup, point out possible solutions, and document known bugs and incompatibilities.
Most major browsers had added HTTP/2 support by the end of 2015. However, I hardly ever see F5 LTM setups with HTTP/2 full proxy configured.
The time has come to configure LTM with HTTP/2 as well.
I would of course welcome any additions or comments. I will keep this article up-to-date.
Why HTTP/2?
HTTP/2 is a major improvement over HTTP/1. HTTP/2 decrease latency to improve page loading speed in web browsers by considering header compression and multiplexing requests over a single tcp connection. Accelerate applications is one point that we all want to achieve with an ADC.
Speed is important, but security is more important. HTTP/2 offers us more security out of the box by making HTTP Request Smuggling almost impossible. That is the second point that we all want to achieve.
Read more about the Desync Endgame: HTTP/1.1 Must Die
How to configure a HTTP/2 Full Proxy Setup
Activating HTTP/2 Full Proxy is simple:
- Configure a ClientSSL and a ServerSSL Profile with disabled renegotiation. SSL Renegotiation is not supported by HTTP/2 and also not for TLS1.3.
- Configure a client-side and a server-side HTTP/2 Profile, ensure that "Enforce TLS Requirements" is enabled. It is very likely that you must increase the "Number of streams per connection" and "Header table" size settings.
- Configure the Virtual Server with client-side and server-side SSL profile and HTTP/2 profile and active the "HTTP MRF Router" option.
- Use a HTTP/2 Monitor for the pools: K29224049: Overview of the BIG-IP HTTP/2 monitor
With this setup the Virtual Server speaks HTTP/1 and HTTP/2 for client-side and server-side connections.
F5 References:
Some common errors
- K09644974: Configuration checks for HTTP/2 COMPRESSION_ERROR
- K000141193: Application fails with ERR_HTTP2_PROTOCOL_ERROR when HTTP2 is enabled on F5 BIG-IP
- K19447103: BIG-IP system occasionally actively close HTTP2 connection
- K34767034: Slow uploding/POST performance when HTTP/2 profile is applied
Discovered issues with this setup
If everything were working as usual, I wouldn't have written this article. I tried HTTP/2 the first time with 15.1 and I quickly stopped implementing this for various reasons. I was hoping that a few years later, things would have improved somewhat.
F5 itself has a support document with some gotchas: K95032535: Detecting the use of and troubleshooting HTTP/2 traffic on the BIG-IP
Notable ones for my setups are:
- The HTTP/2 protocol is not compatible with NTLM protocols
- OneConnect, Hash and SSL session persistence types, and Connection mirroring in High Availability (HA) configurations are not compatible with HTTP/2 in a full-proxy deployment.
No support for APM Portal Access
LTM+APM is anyway the preferred mode, because of occasionally occurring rewriting problems in Portal Access mode.
Websocket Profile not supported
You can not attach a Websocket Profile to a HTTP/2 enabled Virtual Server. HTTP/2 and Websocket support are exclusive options. I do not know why this can not be configured, but a websocket profile is required as soon as you attach an ASM policy.
It would be useful to have a single Virtual Server that can handle HTTP/2 connections and Websockets over HTTP/1.
F5 References:
- K81313105: HTTP2 and Websocket not working
- K84421441: WebSocket traffic does not work on a virtual server with transparent security policy without wss profile
No possibility to enforce server-side HTTP/2
I have no clue how to enforce server-side HTTP/2 to mitigate the security issues with HTTP/1. However, HTTP/2 is automatically preferred.
Bot Defense blocks
Bot defense blocks traffic with anomaly Invalid HTTP Headers Presence or Order. The root cause is that the headers are all lower case in HTTP/2.
Missing iRule Event
The "HTTP_REQUEST_SEND" event is not triggering in a HTTP/2 full proxy setup and there is no full replacement event for it. You must rewrite your iRules to not use this event. I have not found any documentation about this, but F5 support confirmed this.
ASM unblocking not working
A simple "ASM::unblock" throws an error and the connection is reset. F5 support confirmed that this is a bug.
iRule Variable Scope issues
Kai has discovered this issue some time ago: Dealing with iRule $variables for HTTP2 workload while HTTP MRF Router is enabled
F5 References:
I opened a support call to clarify the observations and point out the desired behavior and there is a bug or not. The outcome was that a variable set in the HTTP_REQUEST event should never been seen in the server side events like SERVER_CONNECTED. That this is working if you not set a variable in the CLIENT_ACCEPTED event is the bug.
We have the same variable scope for client-side and server-side of the tcp connection. Events like CLIENT_ACCEPTED, CLIENTSSL_CLIENTHELO and SERVER_CONNECTED are sharing the scope.
We have then for each HTTP/2 stream another variable scope that is a carbon copy of the parent tcp connection scope. Events like HTTP_REQUEST and HTTP_RESPONSE_RELEASE are sharing this scope.
At time of writing I found no workaround for this problem and F5 engineering is working on this case.
1 Comment
Excellent summary of where we are at with HTTP/2 the moment.