Forum Discussion
FFive
Altocumulus
AltocumulusNSX-T and F5 HA using BGP
Hi All,
I am working on a lab to get F5 LTM VE high availability pair working with NSX-T T0 router using BGP
The routing domain all works find, I am able to establish the BGP neighborship and I see the T0 routes, and the T0 sees my routes.
What I am trying to find information on, is on what the best practice is for the Active/Standby F5 HA pair to be BGP paired to the Active/active T0.
As is, the NSX-T T0 router sees routes being advertised from both F5, even the standby unit. I ran into a problem where the Standby unit was receiving traffic as it was a valid route in the table of the NSX-T0 and to resolve the issue I created a BGP Floating self ip and configured it as the next-hop ip address for the NSX-T0. This way the active F5 always processes the traffic. I am wondering if this is the intended way to do such a design or if there is a better way to do this, a standardize way to do this. Here is an ASCII representation of the design:
+-------------------------------+
| |
| CAMPUS NETWORK |
| |
+-----+---------------------+---+
| |
eBGP eBGP
| |
+-----+---------------------+---+
| Active Active |
| +-----+ +-----+ |
| |EDGE1| NSX-T |EDGE2| |
| +-+---+ T0 +---+-+ |
| |.1 .2| |
+----+----------------------+---+
| |
| |
| |
eBGP eBGP
| |
| NEXT-HOP |
| FLOAT-IP |
|.3 .5 .4|
+-+--+ +---+-+
|F5-1+------HA-------+F5-2 |
+----+ +-----+
Active Passive
Hello,
This configuration is entirely supported and used at many service provider accounts, including my own.
For the BGP peers you will want to peer with the self-IP, not with the floating self-IP. This will indeed allow for two active peers and they will receive routes from both BIP-IP's. The way you control traffic is indeed with the floating self-IP, you just need to set the outbound (or inbound on the routers) next-hop as the floating self-IP using a quick route-map. Here is a quick example config:
router bgp xxxx
bgp log-neighbor-changes
bgp graceful-restart restart-time 120
neighbor x.x.x.x remote-as xxxxx
neighbor x.x.x.x description xxxxx
neighbor x.x.x.x route-map blue-to-bgp outroute-map blue-to-bgp permit 100
set metric 100
set ip next-hop x.x.x.x primary <--Floating Self-IP
Frederick_WitteEmployee
Hello,
This configuration is entirely supported and used at many service provider accounts, including my own.
For the BGP peers you will want to peer with the self-IP, not with the floating self-IP. This will indeed allow for two active peers and they will receive routes from both BIP-IP's. The way you control traffic is indeed with the floating self-IP, you just need to set the outbound (or inbound on the routers) next-hop as the floating self-IP using a quick route-map. Here is a quick example config:
router bgp xxxx
bgp log-neighbor-changes
bgp graceful-restart restart-time 120
neighbor x.x.x.x remote-as xxxxx
neighbor x.x.x.x description xxxxx
neighbor x.x.x.x route-map blue-to-bgp outroute-map blue-to-bgp permit 100
set metric 100
set ip next-hop x.x.x.x primary <--Floating Self-IP
FFiveThis is the exact configuration I used for the lab we are running. It works fine for our purposes, my question was more of if there was a different way to not have to use this and just rely on the failover of the units.
Unfortunately, due to the nature of our design, virtual server and kernel route redistribution doesn't seem like an option at this point, or at least from my current knowledge standpoint. The routes I would need to redistribute as Kernel routes exist in the F5 as the Self IPs (Float and Non), which are the gateways to our web application servers, but regardless. This configuration is valid, and it works for our design, which we are also following and looking closer at the NSX-T and F5 document provided by Eric Chen.
I really appreciate all of your time.
- Frederick_Witte
That's great to hear you're using the configuration and it is working as intended.
I've not had any issues in the past using VIP/kernel distribution with this exact design as the next hop will always be that of the active unit from the perspective BGP peers.
Should you have any trouble please do not hesitate to reach out, always happy to jump on a call with a customer and work through these types of congiruations.
Best,
Fred
- Brandon_
F5's BGP implementation relies on kernel routes for Active/Standby configuration. You need to redistribute the kernel routes to be advertised in to BGP. This covers Virtual Addresses, SNATs, etc that you can assign to a floating traffic-group.
The kernel routes will only be available on the Active device and thereby only advertised from that device. The difference with a floating self-IP and a non floating self-IP is:
If no floating self-IP is defined, the next-hop address will be the active device's local Self-IP. This will change in the event of a failover.
If a floating self-IP is defined, the next-hop address will be that of the floating Self-IP and won't change in a failover.
If you are originating routes yourself by redistributing static routes defined on the BIG-IP those are not failover objects and both devices will advertise.
This lab we did a couple years ago walks through some of these details.
https://clouddocs.f5.com/training/community/adc/html/class5/module1/module1.html
- Eric_Chen
When you setup Advanced Routing on a BIG-IP it will tend to run indepedent of the status of the device unless you enable Route Health Injection to use the state of virtual-address to indicate whether the device should advertise out.
The following has an example of how to setup a /32 address to only advertise on the active device:
There's a couple of different ways you could approach this either by advertising specific VIPs or trying to modify your BGP config to summarize routes. Hopefully this will get you pointed in the right direction and others can chime-in. In this example I'm assuming that each device is configured to do its own BGP advertisements and not use the floating self-ip. I'm not familiar with how it would work if you used a floating IP (and not sure that would work???).
A while back I wrote an article about active/active in AWS with BGP. You could try and modify the steps to use "traffic-group-1" instead of "none" in the examples from the article to setup an active/standby setup: https://community.f5.com/t5/technical-articles/aws-transit-gateway-connect-gre-bgp/ta-p/281647
Looking around I also found the following resource that goes into exhaustive details of setting up NSX with BIG-IP with both static / dynamic / active/standby and active/active: https://www.f5.com/pdf/deployment-guides/f5-vmware-nsx-t-deployment-guide.pdf
- JRahm
Admin
I haven't done any production design/implementation of BGP on BIG-IP, so I'd defer to someone who has, but my understanding is that your neighbor relationships should be active to both active and standby devices, and only the active device should be publishing routes.
- FFive
Hi JRahm,
I appreciate your time to give a reply. The problem, or better said the behaivor, I have noticed is that the Active/Standby F5 units both advertise the BGP routes while connected to the Active/Active Edge routers of NSX-T. I don't know if this is a behavior of F5 by default when in HA pair.
In contrast, and I know these systems aren't 1 to 1, when configuring another system such as a Cisco firewall in HA, the passive firewall will suppress the advertisements until a failover occurs, and the downstream and upstream routers will not see a route being installed, in fact the BGP neighborship will remain idle until after the failover.
Is there any reason why the F5 in HA does not behave similarly?
Regards
- JRahm
I'm out on PTO the rest of the week, but if you want to email me your BGP configuration (sanitized is fine as long as the integrity of the config doesn't change) I'll mock it up in the lab and take a look early next week, and reach out to some peers who work actively with BGP on the BIG-IP
