F5 XC – Persistence and Resiliency pt. I (persistence)

Consistent Hash - a brief introduction

Since we're talking about distributed environments (multiple PoPs, CEs, etc.) and need to ensure consistent behavior without necessarily being able to form a cluster (e.g., PoP Frankfurt and PoP Amsterdam), we use a method that is suitable for distributed structures: Ring Hashing (Consistent Hashing).

 

Hash-Ring - TL;DR:

Consistent Hashing is based on a so-called hash ring, which can be imagined, in simplified terms, as a circle with positions from 0 to 59. Both servers (nodes) and data (e.g., clients identified by keys or IP addresses) are distributed on this ring. The three main components:

1. The Hash Ring: The ring consists of a fixed number of positions (e.g., 60 positions). This number is purely illustrative and can be much larger in real systems (e.g., 2³² positions).
2. Placement of Nodes: In our example, we have four servers (nodes). These are distributed on the ring. The position can be determined, for example, by hashing the server's IP address (so it always ends up in the same place):
   • Node 1 at position 0
   • Node 2 at position 15
   • Node 3 at position 30
   • Node 4 at position 45
3. Distribution of Keys: A key represents (in xC) the client. This can be composed of several components (in xC): source IP, headers, cookies (passive or active → source & destination ports + addresses):
   • Client 1 has key 20 → position 20
   • Client 2 has key 35 → position 35

Now, each key is assigned to the next node in a clockwise direction on the ring:

• Client 1 (position 20) goes to Node C (position 30)
• Client 2 (position 35) goes to Node D (position 45)

This is a simplified example for better illustration … in real scenarios, keys/nodes will be distributed much less evenly.

Important: A key is not assigned to the closest node counterclockwise, but always to the next one clockwise. If the key, for example, is at position 59, it will be assigned to Node A (position 0), since the ring is cyclical.

Simple example Hash-Ring with Node/Key distribution

 

 

Because the input (key) is made up of quasi-static values (at least for the duration of a session), the same input will always land at the same place on the ring:

• Client 1 (IP 1.2.3.4) hashed is always value x (here position 20)
• Client 2 (IP 1.2.1.2) hashed is always value y (here position 35)

 

In a static setup (no nodes added/removed), we establish quasi-persistence by default. Since the algorithm behaves the same on all RE/CE nodes, this holds true across all REs and CEs, even without synchronization.

 

Server Failure (Origin Server)

A server fails (Node 4 at position 45):

• Client 2 (position 35) is now reassigned to Node 1 (position 0)
• Client 1 (position 20), assigned to Node 3/position 30) remains unchanged

Re-Assignment (“loss of persistence”) takes place only for clients in the Ring-Area between 30 and 45. Other Nodes within the key ring keep their position and assigned Node. No complete re-assignment of all Nodes and all keys is required.

 

"Persistence" in xC

• Persistence is handled at the ingress (HTTP Load Balancer)
• The persistence node value is derived from a combination of the origin pool server entry and the site (RE or CE - e.g., Site-01/webserver01, Site-02/webserver01, etc.)
• The persistence key (Client Key) can be derived based on the following criteria:
  • Source IP
  • Cookie (passive and/or active)
    • XC can read an existing cookie (passive)
    • XC creates a cookie based on Source IP, Destination IP, and Ports
  • HTTP header (via hash policies)
  • A combination of all the above using hash policies

 

Since all Regional Edges and Customer Edges use the same mathematical calculation to create the ring hash, and because the client key is derived from a static value (such as the IP address), the assignment to the origin server will be consistent across all locations.

Keep in mind: the RE/CE site is part of the calculation. This means a client with the same cookie/header might be routed to a different server in the Frankfurt PoP than in the London PoP. However, within a single PoP, the client should maintain persistence as long as no recalculation takes place.

 

Test – Round Robin

Scenario: Hybrid (RE-CE) / vSite

LB Method: Round Robin

Curl-Test (respnse: Webserver Hostname)

curl -s https://echo-hybrid-central.edge.de1chk1nd.de | jq ".os.hostname"

 

Explanation: Round-Robin – no consistent hash is used. Traffic will be passed through all webservers.

 

Test – Source IP Stickiness

Scenario: Hybrid (RE-CE) / vSite

LB Method: Source IP Stickiness**

** Ring Hash with Key based on source IP

Curl-Test (respnse: Webserver Hostname)

curl -s https://echo-hybrid-central.edge.de1chk1nd.de | jq ".os.hostname"

 

Explanation: Since IP Stickiness is based on source IP, and we do not change the origin server routing/pool (so placement/ID on the ring hash is the same), clients will be sent to the same destination.