NimbostratusCloudflare + F5 w/ Cookie session affinity
We're trying to setup cloudflare with our F5 using cookie based session affinity. This appears to be a known complication for which both cloudflare and F5 have some guidance.
"Session cookies
If using HTTP cookies to track and bind user sessions to a specific application server at the load balancer, it is best is to configure the load balancer to parse HTTP requests by cookie headers and directing each request to the correct application server even if HTTP requests share the same TCP connection due to keep-alive.
For example: F5 BIG-IP load balancers will set a session cookie (if none exists) at the beginning of a TCP connection and then ignore all cookies passed on subsequent HTTP requests made on the same TCP socket. This tends to break session affinity because Cloudflare will send multiple different HTTP sessions on the same TCP connection. (HTTP cookie-based session affinity)."
https://support.f5.com/csp/article/K7964
By default, the BIG-IP system performs load balancing for each TCP connection, rather than for each HTTP request. After the initial TCP connection is load balanced, all HTTP requests seen on the same connection are sent to the same pool member. You can modify this behavior by forcing the server-side connection to detach after each HTTP request, which in turn allows a new load balancing decision according to changing persistence information in the HTTP request. You can force the server-side detachment by applying both an HTTP profile and a OneConnect profile to the virtual server, or by using an iRule to explicitly detach the server-side connection after each HTTP request.
In working with Rackspace engineers (the F5 is managed) we've applied the OneConnect profile however this profile is only meant to address the issue of sending multiple https requests over a single TCP connection. We are still facing issues with maintaining session affinity due to what appears to be cloudflare selecting a different proxy IP somewhat randomly. For instance if I sit on the web page in the browser and simply click refresh, this will often times create a new TCP connection to the F5 coming from a different cloudflare proxy IP. Rackspace engineer has informed me that due to the creation of a new TCP connection the load balancing algorithm is evaluated before the session affinity cookie (As this is done in this order for all new TCP connections) and as such we cannot maintain session affinity.
I have a hard time believing that Cloudflare simply doesn't work with F5's using cookie based session affinity, but I don't have the detailed knowledge of how the F5 functions as this is a managed device for us. Can anyone help provide some guidance here?
From Rackspace engineer:
"As per our discussion today, when I ran couple tests from my browser and did the captures, I never saw my connection fail and also the cloud-flare proxy IP remained the same for every refresh and also the back-end server remained the same.
But when I did a capture for your requests which were intermittently failing, I saw your cloud-flare proxy IPs were random and the back-end connections were hopping through different servers. When there is a change in proxy IP, the load balancer is receiving a new TCP handshake request due to which the load balancer will not adhere to the session cookie that was sent in the previous request and would try to do load balancing of the new connection. This seems to cause the issue when the connection is being end up on a different server."
From Cloudflare team:
"This is indeed the expected behavior from Cloudflare, so unfortunately we would not be adding much of any value joining a call with Rackspace. It's something that would need to work out at the origin. From the support ticket, it appears you are already familiar with best practices for working with F5 Load balancers plus it seems the oneconnect option has been implemented. From where xxx and I sit, we can confirm this has worked for other customers in the past."
Thanks in advance for any assistance.
