Achieving Low Latency and Resiliency with F5 Distributed Cloud CE and Arista Switch

Hey everyone! Just wanted to share how I recently put together a proof-of-concept focused on achieving low latency and high resiliency for application delivery. The core idea was integrating F5 Distributed Cloud Customer Edge (CE) with Arista switches in a smart, multi-site setup. I built this around two locations: a Primary Site in a private data center hosting the app locally, and an Overflow Site in AWS to kick in when local resources get busy.

 

Introduction

At the heart of it, the F5 Distributed Cloud Customer Edge acted as the control plane, setting up a secure network fabric connecting the F5XC backbone, my data center, and the AWS environment. Hand-in-hand with that, the Arista switch managed BGP peering and ECMP routing. I established BGP peering directly between the Arista switch and the Customer Edge nodes, leveraging ECMP to distribute traffic efficiently for resilience and scalability. I also set up an HTTP load balancer on the Customer Edge cluster, with its IP address dynamically advertised into the Arista network. This ensured that local traffic was efficiently routed to the appropriate application server within the local environment. Steering traffic over to that AWS overflow site was handled automatically based on passive health checks and performance thresholds I easily configured within the F5 Distributed Cloud console. It was a cool way to ensure the application stays available and performs well, no matter what.

 

Architecture Overview

Private Data Center Deployment

F5 Distributed Cloud CE Cluster:

• • Deploy three Customer Edge (CE) nodes in a cluster to ensure resiliency.
  • These interconnected nodes form the primary entry point into the private data center.
  • Configurations through a single-pane-of-glass interface on the F5 Distributed Cloud console

Arista Switch Integration:

• • The Arista switch connects with the CE cluster via L3 interfaces.
  • BGP peering is established between the Arista switch and the CE nodes, enabling dynamic routing.
  • ECMP (Equal-Cost Multi-Path) routing is used to distribute traffic evenly, enhancing the overall resiliency of the network.

Local Load Balancing:

• • A load balancer is configured on the CE cluster with the IP address 10.10.124.122.
  • This IP is propagated to the Arista switch, which directs local traffic to the application server in the data center.
  • By keeping the traffic local, users benefit from the lowest possible latency

AWS Overflow Site:

• Secondary Deployment:
• The same application is also deployed on an AWS site (scs-aws3-site).
• This AWS deployment is configured with a lower load balancer priority compared to the local data center.
• The unified load balancer URL (e.g., juicebox.app.com) ensures that both sites can be accessed seamlessly.

• Overflow Traffic Handling:

• When the local data center experiences congestion, the load balancer intelligently routes traffic to the AWS site.
• This overflow mechanism guarantees that users always have access to the application, even during peak load times.

 

How Overflow Works: Detailed Mechanisms

One of the most intriguing aspects of this architecture is the dynamic traffic steering between the local data center and the AWS site. Here’s how the load balancer manages congestion and overflow:

Health Monitoring and Probes

• Active Health Checks:
  The load balancer continuously sends health probes (HTTP/TCP requests) to the local application server. These probes monitor key metrics such as response time, error rate, and overall responsiveness.
• Passive Health Checks:
  In addition to active probes, the system monitors real-time traffic patterns and application performance metrics. Any degradation in service quality triggers a more in-depth analysis.

Thresholds and Policies

• Configurable Thresholds:
  Within the F5 Distributed Cloud console, you can set specific performance thresholds. When these thresholds are exceeded, indicating that the application is under stress, the load balancer marks the local site as degraded.
• Priority Settings:
  Since the local data center is configured as the primary site, it will handle the bulk of the traffic under normal conditions. The AWS site, with its lower priority, only receives traffic when local performance metrics fall below acceptable levels.

 

Seamless Traffic Steering

• Dynamic Rebalancing:
  Once congestion is detected at the local site, the load balancer begins redirecting a portion of the traffic to the AWS overflow site. This process is entirely dynamic and happens in real time without manual intervention.
• Routing Coordination:
  The BGP peering and ECMP routing handled by the Arista switch ensure that local traffic flows efficiently. Meanwhile, the load balancer’s policies ensure that the overflow mechanism is triggered only when necessary, preserving low latency for local users.

 

Benefits and Use Cases

Enhanced Resiliency

• Multiple Failover Layers:
  With both local and AWS deployments, your application is protected against hardware failures, congestion, or unexpected spikes in traffic.

Optimized Performance

• Low Latency for Local Users:
  Local traffic is served directly from the private data center, ensuring minimal latency.
• Scalable Overflow:
  The AWS site acts as a scalable overflow solution, maintaining application availability even during high demand.

Centralized Management

• Unified Configuration:
  The F5 Distributed Cloud console provides a single pane of glass for managing both local and cloud resources, simplifying operations and maintenance.

 

Configuration Walkthrough

F5 Distributed Cloud Customer Edge Configuration

Overview

The F5 Distributed Cloud CE cluster is central to our solution. Each Customer Edge node is configured with redundant connections to the Global Distributed Cloud Regional Edges. Key configurations include setting up health checks to monitor local app performance and defining routing policies that determine when to shift traffic to the overflow site.

Create a Secure Mesh 3-node control Site in the private Data Center. The nodes are connected to the Regional Edges and the Arista Switch using L3 interfaces. You can do that by navigating to the Site Management  Secure Mesh Site v2 option. Visit https://docs.cloud.f5.com/docs-v2/multi-cloud-network-connect/how-to/site-management/create-secure-mesh-site-v2 for details. Similarly, create an AWS site by referring to https://docs.cloud.f5.com/docs-v2/multi-cloud-network-connect/how-to/site-management/deploy-sms-aws-clickops

Private Data Site showing the CE node connected to the Regional Edges.

A three-node cluster is deployed in a private data center to ensure resilience.

Displaying the BGP peering details with the Arista switch in the private data center.

Below are the BGP routes learned through the Arista switch and the subnet allocated for the Juice Shop app, supporting the HTTP load balancer running on the local data center site node.

Arista Data Center site node interface IPs for internal and external traffic.

The Arista Data Center site displays routes for ECMP with the Arista switch.

The Juice Shop app's HTTP load balancer, running in the Arista private data center, displays the origin pool with details such as priority traffic for the application. Also, show the IP address for the Load balancer for the juice shop App.

Below is the IP address assigned to the HTTP Load Balancer for the Juice Shop App, along with the site used to route the traffic.

Here's a basic Arista switch configuration for setting up VLANs, interfaces, and BGP peering:

vlan 10
   name backend
!
interface Ethernet1
   description connected to backend
   speed forced 1000full
   switchport access vlan 10
interface Ethernet11
   speed forced 1000full
   no switchport
   ip address 192.168.60.222/24
!
interface Management1
   ip address 172.16.60.195/24
!
interface Vlan10
   ip address 10.70.100.1/24
!
ip routing
!
ntp server 0.us.pool.ntp.org
ntp server pool.ntp.org
!
router bgp 65001
   router-id 192.168.60.222
   neighbor 192.168.60.52 remote-as 65002
   neighbor 192.168.60.52 description node-32eae
   neighbor 192.168.60.52 ebgp-multihop 2
   neighbor 192.168.60.137 remote-as 65002
   neighbor 192.168.60.137 description node-8e5f4
   neighbor 192.168.60.137 ebgp-multihop 2
   neighbor 192.168.60.143 remote-as 65002
   neighbor 192.168.60.143 description node-1f456
   neighbor 192.168.60.143 ebgp-multihop 2
   !
   address-family ipv4
      neighbor 192.168.60.52 activate
      neighbor 192.168.60.137 activate
      neighbor 192.168.60.143 activate
      network 10.70.100.0/24
!
end

 

Arista switch BGP peering details with the F5 Distributed Cloud Customer Edge node, including the Juice Shop app's HTTP load balancer route learned by the Arista switch, along with the corresponding next-hop CE addresses

 

 

Conclusion

Integrating F5 Distributed Cloud CE with Arista switches presents a powerful solution for enterprises that require both low latency and high resiliency. By intelligently steering traffic based on real-time performance metrics and utilizing an overflow AWS site, this architecture ensures that your application remains available and responsive under all conditions.

Whether you’re a network engineer, IT architect, or simply interested in modern infrastructure solutions, this deployment model offers valuable insights into building scalable, high-performance applications.  For additional details on this solution, please visit the YouTube Link