err_http2_protocol_error

Is the F5 BIG-IP a router? Yes! No! Wait what?

Can the BIG-IP run a routing protocol? Yes. But should it be deployed as a core router? An edge router?

Stay tuned. We'll explore these questions and more through a series of common use cases using BGP on the BIG-IP... And oddly I just realized how close in typing BGP and BIG-IP are, so hopefully my editors will keep me honest. (squirrel!)

In part one we will explore the routing components on the BIG-IP and some basic configuration details to help you understand what the appliance is capable of. Please pay special attention to some of the gotchas along the way.

Can I Haz BGP?

\n\n

Ok. So your BIG-IP comes with ZebOS in order to provide routing functionality, but what happens when you turn it on? What do you need to do to get routing updates in to the BGP process? And well does my licensing cover it? Starting with the last question…

tmsh show /sys license | grep \"Routing Bundle\"

The above command will help you determine if you’re going to be able to proceed, or be stymied at the bridge like the Black Knight in the Holy Grail.

Fear not! There are many licensing options that already come with the routing bundle.

Enabling Routing

First and foremost, the routing protocol configuration is tied to the route-domain. What’s a route-domain? I’m so glad you asked!

Route-domains are separate Layer 3 route tables within the BIG-IP. There is a concept of parent and child route domains, so while they’re similar to another routing concept you may be familiar with; VRF’s, they’re not quite the same but in many ways they are. Just think of them this way for now. For this context we will just say they are. Therefore, you can enable routing protocols on the individual route-domains. Each route-domain can have it’s own set of routing protocols. Or run no routing protocols at all.

By default the BIG-IP starts with just route-domain 0. And well because most router guys live on the cli, we’ll walk through the configuration examples that way on the BIG-IP.

tmsh modify net route-domain 0 routing-protocol add { BGP }

So great! Now we’re off and running BGP. So the world know’s we’re here right? Nope.

Considering what you want to advertise.

The most common advertisements sourced from the BIG-IP are the IP addresses for virtual servers. Now why would I want to do that? I can just put the BIG-IP on a large subnet and it will respond to ARP requests and send gratuitous ARPs (GARP). So that I can reach the virtual servers just fine.

<rant>

Author's opinion here: I consider this one of the worst BIG-IP implementation methods. Why? Well for starters, what if you want to expand the number of virtual servers on the BIG-IP? Well then you need to re-IP the network interfaces of all the devices (routers, firewalls, servers) in order to expand the subnet mask. Yuck! Don't even talk to me about secondary subnets.

Second: ARP floods! Too many times I see issues where the BIG-IP has to send a flood ofGARPs; and well the infrastructure, in an attempt to protect its control plane, filters/rate limits the number of incoming requests it will accept. So engineers are left to try and troubleshoot the case of the missing GARPs

Third: Sometimes you need to migrate applications to maybe another BIG-IP appliance as it grew to big for the existing infrastructure. Having it tied to this interface just leads to confusion.

I'm sure there's some corner cases where this is the best route. But I would say it's probably in the minority.

</rant>

I can hear you all now… “So what do you propose kind sir?”

See? I can hear you...

Treat the virtual servers as loopback interfaces. Then they’re not tied to a specific interface. To move them you just need to start advertising the /32 from another spot (Yes. You could statically route it too. I hear you out there wanting to show your routing chops.) But also, the only GARPs are those from the self-ip's

This allows you to statically route of course the entire /24 to the BIG-IP’s self IP address, but also you can use one of them fancy routing protocols to announce the routes either individually or through a summarization.

Announcing Routes

Hear ye hear ye! I want the world to know about my virtual servers. *ahem*

So quick little tangent on BIG-IP nomenclature. The virtual server does not get announced in the routing protocol. “Well then what does?”

Eery mind reading isn't it? Remember from BIG-IP 101, a virtual server is an IP address and port combination and well, routing protocols don’t do well with carrying the port across our network. So what BIG-IP object is solely an IP address construct? The virtual-address!

“Wait what?” Yeah… It’s a menu item I often forget is there too. But here’s where you let the BIG-IP know you want to advertise the virtual-address associated with the virtual server. But… but… but… you can have multiple virtual servers tied to a single IP address (http/https/etc.) and that’s where the choices for when to advertise come in.

tmsh modify ltm virtual-address 10.99.99.100 route-advertisement all

There are four states a virtual address can be in: Unknown, Enabled, Disabled and Offline. When the virtual address is in Unknown or Enabled state, its route will be added to the kernel routing table. When the virtual address is in Disabled or Offline state, its route will be removed if present and will not be added if not already present. But the best part is, you can use this to only advertise the route when the virtual server and it’s associated pool members are all up and functioning. In simple terms we call this route health injection. Based on the health of the application we will conditionally announce the route in to the routing protocol.

At this point, if you’d followed me this far you’re probably asking what controls those conditions. I’ll let the K article expand on the options a bit. https://my.f5.com/manage/s/article/K15923612

“So what does BGP have to do with popcorn?”

Popcorn? Ohhhhhhhhhhh….. kernel! I see what you did there!

\n\n

I’m talking about the operating system kernel silly. So when a virtual-address is in an unknown or enabled state and it is healthy, the route gets put in the kernel routing table. But that doesn’t get it in to the BGP process. Here is how the kernel (are we getting hungry?) routes are represented in the routing table with a 'K'

\n\n

This is where the fun begins! You guessed it! Route redistribution? Route redistribution!

And well to take a step back I guess we need to get you to the ZebOS interface.

To enter the router configuration cli from the bash command line, simply type imish. In a multi-route-domain configuration you would need to supply the route-domain number but in this case since we’re just using the 0 default we’re good. It’s a very similar interface to many vendor’s router and switch configuration so many of you CCIE’s should feel right at home. It even still lets you do a write memory or wr mem without having to create an alias. Clearly dating myself here..

I’m not going to get in to the full BGP configuration at this point but the simplest way to get the kernel routes in to the BGP process is simply going under the BGP process and redisitrubting the kernel routes.

BUT WAIT! Thar be dragons in that configuration!

First landmine and a note about kernel routes. If you manually configure a static route on the BIG-IP via tmsh or the tmui those will show up also as kernel routes

\n\n

Why is that concerning? Well an example is where engineers configure a static default route on the BIG-IP via tmsh. And well, when you redistribute kernel routes and that default route is now being advertised into BGP. Congrats! AND the BIG-IP is NOT your default gateway hilarity ensues. And by hilarity I mean the type of laugh that comes out as you're updating your resume.

The lesson here is ALWAYS when doing route redistribution always use a route filter to ensure only your intended routes or IP range make it in to the routing protocol. This goes for your neighbor statements too. In both directions! You should control what routes come in and leave the device.

Another way to have some disasterous consequences with BIG-IP routing is through summarization. If you are doing summarization, keep in mind that BGP advertises based on reachability to the networks it wants to advertise. In this case, BGP is receiving it in the form of kernel routes from tmm. But those are /32 addresses and lots of them! And you want to advertise a /23 summary route. But the lone virtual-address that is configured for route advertisement; and the only one your BGP process knows about within that range has a monitor that fails.

The summary route will be withdrawn leaving all the /23 stranded. Be sure to configure all your virtual-addresses within that range for advertisement.

Next: BGP Behavior In High Availability Configurations

I seem to have a workaround, although I still need clarification on the recommended way to do this.

I believe I have 4 ways to share a route with my BGP neighbors from BIG-IP. In each case I need to run \"redistribute kernel\" and, as you point out, use a route map to filter out the 0.0.0.0/0 route and perhaps others I don't want to share, so that I don't send the BIG-IP's default route to the BGP peers and take down the network.

a) I can use the imish command \"network 192.168.100.0/24\" when configuring BGP on BIG-IP, as is done in this KB article from F5.

b) I could create a static route to share, using F5 GUI or tmsh. However, since I can't set a next hop IP address to my own self IP, I don't think this can help me when creating a VIP range to redistribute.

c) I could create a /32 VIP and use the imish command \"aggregate-address 192.168.100.0/24\", as is done this lab from F5. I think this is what you mean when you say we should summarize routes.

d) I can create a VIP where the VirtualAddress is 192.168.100.0/24 and set Route Advertisement to Enabled. It's a single VIP, and the imish command \"show ip route\" shows that it's directly connected to my Active device, and not my Standby.

I think that only option D gets me the VIP range advertised from the Active device only. Options A,B,C all seem to lead to the situation where both Active and Standby devices are advertised as equally-weighted next hops for the route.

I seem to have a workaround, although I still need clarification on the recommended way to do this.

a) I can use the imish command \"network 192.168.100.0/24\" when configuring BGP on BIG-IP, as is done in this KB article from F5.

c) I could create a /32 VIP and use the imish command \"aggregate-address 192.168.100.0/24\", as is done this lab from F5. I think this is what you mean when you say we should summarize routes.

In my case, I have a single VIP which is a /32 route. My VIP is 192.168.100.100/32, but I want to advertise a summary route, like you've stated in your article. I used the imish command \"aggregate-address 192.168.100.0/24\" which does the trick. Now, the /24 route is advertised, not the /32. Great.

My problem is that both the Active and the Standby unit are advertising this route with equal weight, so I am seeing my BGP neighbors with 2x next hops for this VIP range (the self IP of the Active, and that of the Standby).

Well hello there Michael and thank you for stumbling on my article. I had the intention to write more of the series, but as there really wasn't a lot of feedback I didn't think anyone really wanted to hear from me. My goal was to cover how HA behavior works with BIG-IP and BGP in the second in the series. And well, your questions inspired me to pick this back up.

But to quickly summarize. If a BIG-IP HA-Pair is set up properly the kernel routes should only show up in the routing table IF the BIG-IP is active. The standby for the traffic-group will not have the kernel routes to redistribute.

This is why, in my lab which you also stumbled upon; I use aggregate-address. BGP is a different routing protocol in that it generally must learn a route from another routing protocol, or static, or in this case from the kernel. Without the /32 kernel route for the BGP process to learn something in the /24 aggregate it won't advertise the /24. This would leave me to believe that the /32 is active in the kernel on both devices which is why you saw it advertised from both.

One fun fact. When you use aggregate-address it will install a /24 route to null0 in the routing table to advertise. This is also why if there are individual VIPs in the range they all should have advertisement enabled assuming you don't want the route withdrawn if any particular /32 fails.

Now for a little bit on the difference between the network statement in BGP and the aggregate-address statement.

For the network statement to work by default back in the day, when dinosaurs configured routers with their little T-Rex hands there must be an exact match in the routing table.

I say by default; because that was the default behavior and many of us packet heads after configuring our BGP statement, the next immediate command was 'no auto-summary'. Which now is the default and doesn't show up in the configuration anymore.

A lot of times, the /24 virtual server gets created as a \"catch all\" for the subnet. This can be a good thing in that I can log what hits IPs within the range that don't have a more specific virtual server attached to them. This is especially good if you're migrating things around. Like an application moves over to another network and you can see if for some reason that route isn't getting honored somewhere. There's lots of ways to have fun with it.

And yes, if it's tied to the failover traffic-group it will only be a kernel route on the active device. The problem will be that the route won't be withdrawn without some failure situation. In this case I would think about creating a pool and assign a monitor with an alias configuration that monitors something downstream.

Shame on me! I’ve been sitting here patiently waiting on the next installment.

I am currently putting off a BGP config due to HA and lack of clear guidance, I should have just asked!

In my world, I don’t control the upstream neighbor router, so it’s something I have to plan extensively.

When forming a neighbor-ship with the upstream router, and I have a 2 - i7820 HA pair, do I give them the Floating IP? Or do I build basically two neighbors, one in each physical i7820 with their non-floating IP?

Oh my! Now I'm getting motivated to finish part 2 which is focused on how HA works with a BIG-IP and BGP. So thanks for validating that there's some value there.

When forming neighbor relationships with upstream routers, you will establish the neighbor with the local self-IP on each BIG-IP. So the router will have two neighbor statements and the BIG-IP will have 1 or 2 depending on your upstream configuration.

The value of the floating self-IP in this arrangement is that the Active router will advertise the networks you want with a next-hop of the floating self-IP.

Now why would that be good? Well in a failover scenario the next-hop doesn't change. So no matter which router is Active, the next-hop and all flows associated with it won't change. The upstream routers don't need to learn the new next-hop and redirect traffic that way so any flows in-flight aren't interrupted.

MichaelOLeary was nice enough to link to a BGP lab I developed and taught a couple years ago at our user conference. Its retirement is the inspiration for this intended series and it might help you with some additional thoughts. But feel free to fire away questions while I diligently come up with smart-alecky things to say for the second installment.

https://f5-agility-labs-adc.readthedocs.io/en/latest/class5/module1/lab1.html#create-an-aggregate-route-since-many-service-providers-will-not-accept-anything-less-than-a-24

ABOUT DEVCENTRAL

RESOURCES

SUPPORT

PARTNERS

Can I Haz BGP?

Enabling Routing

Considering what you want to advertise.

<rant>

</rant>

Announcing Routes

“So what does BGP have to do with popcorn?”

BUT WAIT! Thar be dragons in that configuration!