Forum Discussion
CirrusDelayed connection when traffic crosses router domains in F5 BIG-IP
CumulonimbusHello fluzocapacitor
When you split your network into multiple route domains (RDs) on an F5 BIG-IP, each domain acts like its own separate “mini-router” with its own routing table. This means traffic doesn’t automatically move between them—even if they’re on the same device.
In your case, when a server in RD2 tries to reach a virtual server in RD0, the F5 doesn’t know how to send the traffic across unless you specifically tell it how. If this isn’t set up, the F5 keeps trying to figure out where to send the traffic, which causes the 10–15 second delay you’re seeing (or even timeouts).
Here are some recommendations:
- Make sure you have explicit routes set up in both RD2 and RD0 that tell the F5 how to reach the other domain.
- Consider creating a “forwarding” virtual server in each route domain. This acts like a bridge, letting traffic pass between domains.
fluzocapacitorThanks for the detailed explanation — we’ll test your suggestions with explicit routes and forwarding virtual servers between route domains.
That said, I’m still puzzled by one thing:
If the F5 doesn’t have a valid route between RD2 and RD0, how is it even able to eventually send the response back at all? I would expect the traffic to consistently fail or time out immediately, since the device shouldn’t know how to reach the destination in another route domain.Yet, in our case, it delays for 10–15 seconds and then succeeds. Could the F5 be falling back to some kind of indirect routing or trying multiple options before failing over to a default behavior?
Also, just to give more context — we introduced multiple route domains on the F5 because previously all servers were in a single VRF on the upstream router. We’ve now segmented them into multiple VRFs, and within these new VRFs, some servers use the F5 as their default gateway, while others route through the firewall.
Thanks again for your help — any insight into how the F5 is even partially succeeding without explicit inter-RD routing would be great to understand.
- Injeyan_Kostas
Nacreous
How do you use F5 as gateway if it doesn't have Routes?
You may don't have a specific route between RD2 and RD0 but RD2 should have a default route and next hop should have a route to RD0.
That said I suspect your delay is because of asymetric routing.
Have you tried a packet capture both to F5 and server?
- fluzocapacitor
Thanks for your reply!
Just to clarify — I didn’t mean to say there are no routes in the F5. There is a default route configured, and the traffic does reach the destination eventually.show net route ------------------------------------------------------------------------------------------------------- Net::Routes Name Destination Type NextHop Origin ------------------------------------------------------------------------------------------------------- Default-VLXX default%2 gw upstream_router_ip%2 static Default default gw f5_self_ip_for_router static
Asymmetric routing is definitely something we’ve had to deal with. When we first separated the VRFs, we ran into issues with the firewall dropping connections due to unknown state. That’s actually one of the reasons we introduced route domains in the F5 — to better isolate traffic paths and avoid that kind of asymmetry.We’ve done captures both on the F5 and on the server side, but so far we haven’t found a clear explanation for the delay when the F5 acts as the gateway and the Virtual Server is in another route domain.
Thanks again!
- VGF5
When the F5 receives traffic that it doesn’t know how to route (for example, from RD2 to RD0 with no explicit route), it will attempt to resolve the next hop using ARP (for IPv4) or Neighbor Discovery (for IPv6) and if there’s no immediate route or ARP entry, the F5 will send ARP requests and wait for a response. During this time, the connection is held open, and the client may experience a delay.
If you want traffic to flow between route domains, you need to do two things:
- Disable Strict Isolation:
Go to Network > Route Domains in the BIG-IP Configuration Utility, select each route domain you want to communicate, and uncheck the “Strict Isolation” box. You must do this on both the source and destination route domains. - Set Up Explicit Routes and Forwarding Virtual Servers:
- Add static routes in each route domain that point to the other, using the correct route domain notation (for example, 10.0.0.0%2 for RD2).
- Create a forwarding (IP) virtual server in each route domain. This allows the BIG-IP to pass traffic between the domains, acting as a bridge.
Here is an article:
- Disable Strict Isolation:
- fluzocapacitor
In our setup, RD2 is configured with RD0 as its parent (it's using the default route domain of the partition), but RD2 also has a default route pointing to an external gateway (outside the F5).
Now I’m wondering:
Could it be that, because of this default route, the F5 is sending traffic out toward the external gateway instead of handling it internally between RD2 and RD0?
Here’s the traffic flow we’re seeing:
Client (192.168.142.4) → F5 RD2 (192.168.142.9) → CORE → FW → F5 VIP RD0 (192.168.158.79) → Pool Member (RD2 - 192.168.152.15) → F5 RD2 → FW → CORE → ClientBut ideally, since the F5 handles both RD2 and RD0 internally, and the pool member is also in RD2, I would expect the connection to be routed entirely inside the F5, without leaving to the external network.
So my question is:
How can I make sure that traffic from RD2 to RD0 (and back) stays entirely within the F5 and doesn’t try to exit via the default route?
Would disabling Strict Isolation and using forwarding virtual servers in both directions solve this, or should we instead remove the default route in RD2 and define explicit internal routes for RD0 destinations?
Appreciate any advice — we’re trying to avoid traffic unnecessarily leaving the box just to re-enter.
- Injeyan_Kostas
Are you doing SNAT on the Virtual Server?
With child/parent relationship forwarded traffic should be already allowed.
And this actually how your RD0 VS can send traffic to RD2 pool member- fluzocapacitor
Yes. When the F5 receives a connection on the VS 192.168.158.79 from 192.168.142.4, it replaces the source IP before forwarding the request to the pool member 192.168.152.15. As a result, the pool member sees the connection as coming from the F5’s self IP (e.g., 192.168.152.7).
