Forum Discussion
CirrusDelayed connection when traffic crosses router domains in F5 BIG-IP
Hi everyone,
I'm experiencing an issue with F5 BIG-IP involving multiple router domains and would appreciate any insight.
We’ve recently moved some VLANs into a separate router domain (RD2) on our internal F5 appliance. The virtual servers that serve our applications are still in the default router domain (RD0).
Here’s the issue:
When a server located in a VLAN within RD2 (e.g., 192.168.142.4) uses the F5 as its default gateway (192.168.142.9), and tries to connect to a virtual server in RD0 (e.g., 192.168.158.79), the connection gets delayed by around 10–15 seconds. Sometimes, the browser even times out.
However, if we change the server’s default gateway to the upstream router (192.168.142.1), bypassing the F5, the same connection to the virtual server is immediate and works without delay.
The traffic path in the delayed case is something like:
192.168.142.4 → F5 RD2 (192.168.142.9) → F5 RD0 (192.168.233.14) → CORE/FW → F5 VIP (192.168.158.79)In the fast case (no delay):
192.168.142.4 → CORE (192.168.142.1) → FW → F5 VIP (192.168.158.79)So far, our assumption is that the traffic from RD2 is not being routed immediately across to RD0 inside the F5, and it somehow gets stuck or delayed before it finds the correct path.
Has anyone experienced this kind of inter-router domain traffic delay in F5 before?
Is there a best practice for handling communication between router domains, especially when the F5 acts as both the gateway and the destination?
Thanks in advance for any suggestions or ideas.
10 Replies
f51Cumulonimbus
Hello fluzocapacitor
When you split your network into multiple route domains (RDs) on an F5 BIG-IP, each domain acts like its own separate “mini-router” with its own routing table. This means traffic doesn’t automatically move between them—even if they’re on the same device.
In your case, when a server in RD2 tries to reach a virtual server in RD0, the F5 doesn’t know how to send the traffic across unless you specifically tell it how. If this isn’t set up, the F5 keeps trying to figure out where to send the traffic, which causes the 10–15 second delay you’re seeing (or even timeouts).
Here are some recommendations:
- Make sure you have explicit routes set up in both RD2 and RD0 that tell the F5 how to reach the other domain.
- Consider creating a “forwarding” virtual server in each route domain. This acts like a bridge, letting traffic pass between domains.
fluzocapacitorThanks for the detailed explanation — we’ll test your suggestions with explicit routes and forwarding virtual servers between route domains.
That said, I’m still puzzled by one thing:
If the F5 doesn’t have a valid route between RD2 and RD0, how is it even able to eventually send the response back at all? I would expect the traffic to consistently fail or time out immediately, since the device shouldn’t know how to reach the destination in another route domain.Yet, in our case, it delays for 10–15 seconds and then succeeds. Could the F5 be falling back to some kind of indirect routing or trying multiple options before failing over to a default behavior?
Also, just to give more context — we introduced multiple route domains on the F5 because previously all servers were in a single VRF on the upstream router. We’ve now segmented them into multiple VRFs, and within these new VRFs, some servers use the F5 as their default gateway, while others route through the firewall.
Thanks again for your help — any insight into how the F5 is even partially succeeding without explicit inter-RD routing would be great to understand.
- Injeyan_Kostas
How do you use F5 as gateway if it doesn't have Routes?
You may don't have a specific route between RD2 and RD0 but RD2 should have a default route and next hop should have a route to RD0.
That said I suspect your delay is because of asymetric routing.
Have you tried a packet capture both to F5 and server?
- fluzocapacitor
In our setup, RD2 is configured with RD0 as its parent (it's using the default route domain of the partition), but RD2 also has a default route pointing to an external gateway (outside the F5).
Now I’m wondering:
Could it be that, because of this default route, the F5 is sending traffic out toward the external gateway instead of handling it internally between RD2 and RD0?
Here’s the traffic flow we’re seeing:
Client (192.168.142.4) → F5 RD2 (192.168.142.9) → CORE → FW → F5 VIP RD0 (192.168.158.79) → Pool Member (RD2 - 192.168.152.15) → F5 RD2 → FW → CORE → ClientBut ideally, since the F5 handles both RD2 and RD0 internally, and the pool member is also in RD2, I would expect the connection to be routed entirely inside the F5, without leaving to the external network.
So my question is:
How can I make sure that traffic from RD2 to RD0 (and back) stays entirely within the F5 and doesn’t try to exit via the default route?
Would disabling Strict Isolation and using forwarding virtual servers in both directions solve this, or should we instead remove the default route in RD2 and define explicit internal routes for RD0 destinations?
Appreciate any advice — we’re trying to avoid traffic unnecessarily leaving the box just to re-enter.
- Injeyan_Kostas
Are you doing SNAT on the Virtual Server?
With child/parent relationship forwarded traffic should be already allowed.
And this actually how your RD0 VS can send traffic to RD2 pool member
- Nikoolayy1
MVP
If you have isolated the route domains then traffic will have to exit F5 and then return again from the network, so latency could happen in that case and it is the design not F5 that is the issue in that case 😉
You can also do tcdumps in the 2 route domains and compare the timestamps:
https://my.f5.com/manage/s/article/K6546
https://techdocs.f5.com/en-us/bigip-14-1-0/big-ip-tmos-routing-administration-14-1-0/route-domains/traffic-forwarding-across-route-domains/about-strict-isolation.html
