F5 BIG-IP in Cisco ACI Multi-Site and Multi-Pod - Design Options

Jennifer_Yeung · ‎05-Jun-2023

F5 BIG-IP Design Options in Cisco ACI Multi-Site and Multi-Pod

There are options to integrate F5 BIG-IP DNS and LTM into Cisco ACI Multi-Site or Multi-Pod. Figure 1 illustrates the common BIG-IP design options for north-south traffic flows (traffic flows between servers in the data center and clients in the external network), which is also applicable to east-west traffic flows that are the traffic flows between real servers through load balancer within data centers and across data centers. :

Figure 1. Common BIG-IP design options

As shown in figure 1, it is typical to deploy F5 BIG-IP DNS behind L3Out whereas there are different options to deploy F5 BIG-IP LTM:

ACI Fabric as default gateway with SNAT: ACI Fabric as default gateway for the real servers and use of SNAT on the BIG-IP LTM.
ACI Fabric as default gateway with PBR: ACI Fabric as default gateway for the real servers and use of ACI PBR to steer to BIG-IP LTM the return traffic flows between real servers and external clients.
BIG-IP LTM as default gateway without the use of SNAT or PBR: BIG-IP LTM as default gateway for the real servers, without the use of SNAT or PBR.
ACI Fabric as default gateway without the use of SNAT or PBR: ACI Fabric as default gateway for the real servers and deployment of a “VRF Sandwich” configuration to deploy the BIG IP LTM inline to the traffic flows (no need for SNAT or PBR).

Also, BIG-IP DNS is typically deployed as a shared resource among multiple tenants whereas BIG-IP LTM is deployed for each tenant or is used as a shared resource among multiple tenants. In this article, we will provide you a quick overview of the common BIG-IP design options and each of its highlight of the key characteristics. Please keep in mind that these options are illustrated in Cisco ACI Multi-Site and are applicable to Cisco ACI Multi-Pod deployments as well, with the assumptions listed below that represent common BIG-IP DNS and LTM deployment best practices:

Independent Active/Standby HA pair of BIG-IP LTMs is deployed in routed mode (L3) in each data center.
Standalone BIG-IP DNS is deployed, which means DNS module and LTM module are deployed separately on individual BIG-IP devices.
BIG-IP DNSs are deployed in the same DNS synchronization group

Let us start with BIG-IP DNS.

BIG-IP DNS behind L3Out

Figure 2. Example of F5 BIG-IP DNS deployment

Figure 2 illustrates an example of typical F5 BIG-IP DNS deployment. As you can see, standalone BIG-IP DNSs are deployed behind L3Out and communicate with BIG-IP LTMs using iQuery via L3Out for information exchang. Also, BIG-IP DNSs are in the same synchronization group so that they work together to monitor the availability and performance of global resources (such as data centers, VIPs, etc.) and use that information to manage network traffic patterns. Next, let us look at BIG-IP LTM.

ACI Fabric as default gateway with SNAT

Figure 3. Example of a north-south BIG-IP LTM with SNAT design (with Service Graph)

Figure 3 illustrates an example of ACI fabric as the default gateway with the use of SNAT on BIG-IP LTM where service graph is used. Higlights of the key characteristics of this design are as follows:

SNAT is used on the BIG-IP LTM.
VIP and its real servers can be deployed in different data centers.
PBR is not required since SNAT is used to ensure both incoming and returning traffic are through the same BIG-IP LTM.
When a service graph is used, the service BD must be stretched across sites, which means the BIG-IP interfaces (or Self IPs) at different data centers must be in the same bridge domain. The bridge domain can contain more than one subnet if needed. One contract is used: a contract between L3Out EPG and Web EPG with a BIG-IP LTM service graph. There is no need to create EPGs for the BIG-IP LTM external and internal interfaces by users because the EPGs for the service device and required security rules (that are called zoning-rules in ACI) are created automatically as part of service graph deployment.
When a service graph is not used, the service BD does not need to be stretched across sites, which means the BIG-IP interfaces (or Self IPs) at different data centers can be in different bridge domains. Two contracts are used: one contract is between L3Out EPG and the BIG-IP LTM EPG for its external interface; and the other contract is between the other BIG-IP LTM EPG for its internal interface and the Web EPG.
BIG-IP DNSs and BIG-IP LTMs communicate to each other using iQuery via L3Out for information exchange

ACI Fabric as default gateway with PBR

Figure 4. Example of a north-south ACI Fabric as default gateway with PBR design

Figure 4 illustrates an example of ACI fabric as the default gateway with the use of ACI PBR to steer the return traffic to BIG-IP LTM. Higlights of the key characteristics of this design are as follows:

SNAT is not used on the BIG-IP LTM.
VIP and its real servers must be deployed in the same data center.
PBR is used to steer the return traffic (sourced from the real servers to the original external clients) toward the same BIG-IP LTM that handled the inbound flow (location-based PBR needs to be enabled in the case of ACI Multi-Pod).
Because a service graph is mandatory to use PBR, the service BD must be stretched, which means the BIG-IP interfaces (or Self IPs) at different data centers must be in the same bridge domain. The bridge domain can contain more than one subnet if needed.
One contract is used: a contract between L3Out EPG and Web EPG with a BIG-IP LTM service graph. There is no need to create EPGs for the BIG-IP LTM external and internal interfaces by users because the EPGs for the service device and required security rules (that is called zoning-rules in ACI) are created automatically as part of service graph deployment.
BIG-IP DNSs and BIG-IP LTMs communicate to each other using iQuery via L3Out for information exchange

BIG-IP LTM as default gateway without the use of SNAT or PBR

Figure 5. Example of a north-south BIG-IP LTM as gateway without SNAT or PBR design

Figure 5 illustrates an example of BIG-IP LTM as the default gateway without SNAT or PBR. Since the BIG-IP LTM is in the traffic path based on routing, both directions of traffic will always be forced to go through the same BIG-IP LTM and neither SNAT nor PBR is required. Higlights of the key characteristics of this design are as follows:

Neither SNAT nor PBR is required.
BIG-IP LTM is the gateway for the real servers.
VIP and its real servers’ gateway must be defined on the same BIG-IP LTM.
Service graph is not mandatory.
When a service graph is used, the service BD must be stretched across sites, which means the BIG-IP interfaces (or Self-IPs) at different data centers must be in the same bridge domain. The bridge domain can contain more than one subnet if needed. One contract is used: a contract between L3Out EPG and Web EPG with a BIG-IP LTM service graph. There is no need to create EPGs for the BIG-IP LTM external and internal interfaces by users because the EPGs for the service device and required security rules (that is called zoning-rules in ACI) are created automatically as part of service graph deployment.
When a service graph is not used, the service BD does not need to be stretched across sites, which means the BIG-IP interfaces (or Self-IPs) at different data centers can be in different bridge domains. Two contracts are used: one contract is between L3Out EPG and the BIG-IP LTM EPG for its external interface; and the other contract is between the other BIG-IP LTM EPG for its internal interface and the Web EPG.
BIG-IP DNSs and BIG-IP LTMs communicate to each other using iQuery via L3Out for information exchange

ACI Fabric as default gateway without the use of SNAT or PBR

Screen Shot 2023-05-16 at 4.33.11 PM.png

Figure 6. Example of a north-south ACI gateway without SNAT or PBR design (VRF sandwich)

Figure 6 illustrates an example of ACI fabric as the default gateway without SNAT or PBR. In this design option, the ACI Fabric is the default gateway of the real servers and the BIG-IP LTM is inserted inline in the data path by using another L3Out for the internal interface of the BIG-IP LTM, an option usually referred to as “VRF sandwich”. Since the BIG-IP LTM is in the traffic path based on routing, both directions of the traffic will flow through the same BIG-IP LTM, and neither SNAT nor PBR are required. Higlights of the key characteristics of this design are as follows:

Neither SNAT nor PBR are required.
ACI fabric is the default gateway of the real servers.
VIP and its real servers must be part of the same site.
This is the traditional VRF sandwich design with 2 VRFs: one for internal and the other for external. All inter-VRF traffic is steered through the BIG-IP LTM based on routing.
ACI fabric internal VRF must have a route to the external network via the BIG-IP LTM.
Because an L3Out cannot be stretched across sites, an L3Out for the internal interface of the BIG-IP LTM is defined in each site.
Service graph cannot be used as of NDO Release 4.1.
Because Service graph is not used, the service BD for the external interface of BIG-IP LTM does not need to be stretched across sites, which means the BIG-IP external interfaces (or Self-IPs) at different data centers can be in different bridge domains. Two contracts are used: one contract is between L3Out EPG for external network and the BIG-IP LTM EPG; and the other contract is between the internal interface of BIG-IP LTM EPG and the Web EPG
BIG-IP DNSs and BIG-IP LTMs communicate to each other using iQuery via L3Out for information exchange

To find out more information on each of the design options which includes the detail traffic flows and design recommendations etc, please refer to Cisco ACI Multi-Site/Multi-Pod and F5 BIG-IP Design Guide, a white paper jointly developed by Cisco and F5.

DevCentral