Forum Discussion
Nimbostratusunbalanced pool , persistence problem
Cspillane_18296Hi Theizer, here's a couple of things to look at:
1) Set the cookie persistence profile to the standard 'cookie insert' (preconfigured) one rather than any custom created ones. Or at least ensure the BigIP inserts its own cookie name (don't specify one).
2) Ensure the f5 is configured to use client/server ssl profiles (as appropriate depending on SSL terminating on the LTM or not). this is to ensure the BigIP can decrypt the traffic correctly (although from your description this all sounds correct but it is worth checking).
3) Ensure the cookie is set to be a session cookie, or even better for test purposes set it to a long lifetime (like a week or more)
4) Ensure client browser accepts cookies.
5) Any iRules that could over-ride persistence? Or remove the cookie?
6) does the entry get written in the persistence table? ( try " b persist show all")
6) If you take a pcap on the LTM can you confirm that once you disconnect then reconnect, as a client, you send the cookie with your requests?
Let me know how you get on!
Chris
theizer_91634Thank you for your response. 1) yes.using the standard cookie persistence profile 2) yes. using clientssl profiles doing ssl offload at F5 3) yes. cookie is session cookie 4) yes. browsers accept cookies 5) only irules i have are http to https redirection 6) i do see the external ip of the call center in the persistence table. it is always no more than 1 second old it seems. 7) not sure how to do a pcap on the ltm. I will research.
smp_86112Cirrostratus
I'm no expert on persistence, believe me. But I bet you have it correct. Proxies are designed to pipeline the HTTP client requests into a relatively small number of TCP connections on the web server side. What you might want to consider is applying a OneConnect profile to the VIP. You should read the doc on it, but the way I think about OneConnect, it's a way for the LTM to load-balance at an HTTP session level versus the TCP level. With OneConnect, the LTM can distinguish between HTTP clients based on the information in the HTTP requests, rather than at the TCP level. We had to enable this with a VIP used by a small group of users behind a proxy server. We found that users would intermittently get forwarded to the wrong webserver, despite cookie persistence, because the proxy server would suddenly decide to establish a new TCP session and switch HTTP requests from the same user between them. This broke the persistence intermittently. With OneConnect, the LTM load-balances each HTTP request versus TCP connection. So we have concluded that it only makes sense to load-balance multiple pool members using cookie persistence with OneConnect - without it you are at risk of breaking persistence. The LTM config guide does a better job at explaining this.
- Cspillane_18296@smp - of course, it all seems so obvious now (slaps forehead). I think this article on Ask f5 explains all: https://support.f5.com/kb/en-us/solutions/public/7000/900/sol7964.html?sr=9457601
- theizer_91634This is my theory at the moment - is this plausible?
Currently, there are two persistence methods configured. The default is cookie and the fallback is source-ip. The theory is that when a client connects and doesn’t have a cookie, the F5 looks at the client source ip (in this case, all our call center users share the same source ip) and sends the client to the pool member that ip last connected to. When the session is established, the client receives a cookie that persists the session with that web server. Thus, all our call center traffic is persisted to the same server. This would also explain why the hardest hit server changes when a code rollout is done and the servers are disabled in the pool one at a time. When a server is disabled in the pool, new connections are not allowed and so they go to a different web server and the cycle repeats.
My thought, was to remove the fall back persistence method of source-ip. - Cspillane_18296Hi Theizer,
the first bit of your theory makes sense, but if you remove the fallback method and cookie persistence isn't working, you'll be left with no persistence whatsoever. The fallback method should only be used when there's no record of the primary method of persistence.
Have you enabled the OneConnect profile on the VS to see if this allows cookie persistence to work correctly? - theizer_91634I have not enabled OneConenct yet - there is some concern regarding http pipelining and how it will behave. Also, I wanted to only change one thing at a time. After removing the source-ip fallback persistence, we are definately seeing a big difference - when looking at pool stats, my connections are MUCH more balanced between members now. I am interested in one-connect and how it could further reduce server load, but i just don't understand it enough yet to feel comfortable enabling it. The manual's explanation of it leaves me guessing. I am continuing to read up on it trying to wrap my mind around it.