Symmetric routing on NGINX Plus with multiple interfaces
Overview
The BIG-IP as a networking appliance has a feature named ‘Auto-Last-Hop’, it allows the BIG-IP to send returning traffic (e.g., VIP to client traffic) directly back to the source MAC address of the matching incoming traffic, as well as pushing returning traffic out the same interface where incoming traffic was received.
With ‘Auto-Last-Hop’, returning traffic is sent to the sender without a route table lookup by the BIG-IP. Please see this ( https://support.f5.com/csp/article/K13876 ) for details on ‘Auto-Last-Hop’.
When you migrate workloads from the BIG-IP to NGINX Plus, as NGINX Plus offloads L2/L3 functions to the underlying Linux machine where it sits on, a different approach is warranted as Linux does not have a feature similar to ‘Auto-Last-Hop’.
Imaging a scenario where the NGINX Plus instance has multiple interfaces and all of which are connected to a firewall. Client (10.0.2.2) sends traffic to the 10.0.1.2 address on the NGINX Plus server. Return traffic from 10.0.1.2 would be sent via eth0 based on the route table look up, lands on firewall on 10.0.0.1 interface and gets dropped for packet spoofing.
Linux Advanced Routing
With the help of Linux advanced routing, we can create additional route tables to steer returning traffic back out the original path.
echo "1 s1" >> /etc/iproute2/rt_tables
ip route add 10.0.1.1 scope link dev eth1
ip rule add from 10.0.1.0/24 table s1
ip route add default via 10.0.1.1 dev eth1 table s1
The above commands (run on Ubuntu) create a route table named ‘s1’, with a numeric ID of 1, as well as telling Linux to route traffic sourced from 10.0.1.0/24 to use s1 and s1 has 10.0.1.1 as its default gateway. The following diagram depicts the new traffic traversal.
Since returning traffic is routed back out via the incoming interface, firewall passes that traffic through.
If you have additional interfaces to cater for more VIP addresses as listeners on NGINX Plus, simply create additional route tables as described above.
For details on Linux advanced routing, please take a look here ( https://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.rpdb.html ).
The bottom line is, despite Linux does not have a feature akin to ‘Auto-Last-Hop’ on the BIG-IP, it has advanced routing functions that achieve a similar result, and this allows you to deploy NGINX Plus more easily without having to change existing networking topologies.
We have used this feature of the Linux OS for many years to allow direct connect between the F5 BIG-IP and the backend servers, while allowing the servers to retain their primary gateway at the router or firewall, which can also allow for SNAT to be disabled on the VS.
Very underrated, wish Windows had it!