Forum Discussion
Dmitri_Ch__1425
Cirrus
CirrusnPath and firewall rules
Hi
Does nPath need special firewall configuration?
I've configured nPath in LAB, everything seems to be working fine. But I cannot get it to work on production network and suspect firewall blocking return traffic from nodes.
Are there any firewall rules needed on for nPath to work properly?
Thank you.
What is interesting is in LAB env. when client ARPs for VS_IP one of the node responds with it's MAC address and everything works OK then. That means the F5 will not be involved at all, and the whole purpose of nPath is for the F5 to be involved!!
In prod, when client ARPs for VS_IP, F5 responds with MAC address of self-ip, which causes the problem, b/c node then trying to send back to client's MAC. That's how nPath should work - client sends to F5, F5 forwards to server, and server responds direct to client.
Also in LAB when I run tcpdump on the F5 I don't see any traffic, only ARP.
Yes because the client is sending only to the node as that is what responded.You need to prevent the servers from responding to the arp requests for the VS_IP (so get rid of that -arp setting), although they do need to have an interface that will respond to unicast traffic to that IP configured. While the F5 and the nodes are all responding to arp it's a race to see who gets there response in first. Once you have the F5 only responding then you'll need to troubleshoot the other problem.
IheartF5_45022Nacreous
It depends where in the network topology the firewall is.
If the firewall is between the F5 and the server then you are likely to have problems.If both F5 and server are behind the firewall, and behind the same interface of the firewall, then you should not have an issue.
Dmitri_Ch__1425Hi, thank you. If there is nothing on firewall then I'll double-check F5 config.
F5's VS, node and firewall are all on the same subnet. The concern I have with firewall is it could drop reply packets coming from the node since firewall doesn't have incoming connection to match. Could this be the problem?
I ran tcpdump on F5 and node and see that node is sending data back (dst is MAC address of FW), but my client doesn't receive those.
- Dmitri_Ch__1425
I took one more look at tcpdump result on the node. I see that node gets SYN from client_IP with MAC address of F5's interface. Then node ARPs for MAC of the client_IP, gets MAC address, which is different from what's in SYN packet and sends SYN,ACK to client_IP /w client_MAC address. And client drop it, which is what I expect in this case.
The node's lo:0 configured as :
ifconfig lo:0 netmask 255.255.255.255 -arp up
Autolasthop is enabled on F5, both global and VS level.
- IheartF5_45022why do you say "And client drop it, which is what I expect in this case"? SYN-ACK with client MAC as dest should not be dropped.
Is the Client on the same subnet as the server? If not, then the server should ARP for the gateway IP, and send packet that has the client IP and gateway MAC.
- Dmitri_Ch__1425Yes, in this test the client is on the same subnet. I'm trying to pinpoint the problem and avoid firewall for now. What is interesting is in LAB env. when client ARPs for VS_IP one of the node responds with it's MAC address and everything works OK then. In prod, when client ARPs for VS_IP, F5 responds with MAC address of self-ip, which causes the problem, b/c node then trying to send back to client's MAC. Also in LAB when I run tcpdump on the F5 I don't see any traffic, only ARP. I don't know if it make any difference - in LAB I'm using Ubuntu nodes, in prod Redhat nodes. Same F5 unit, different VLANs on trunk port...
- IheartF5_45022
What is interesting is in LAB env. when client ARPs for VS_IP one of the node responds with it's MAC address and everything works OK then. That means the F5 will not be involved at all, and the whole purpose of nPath is for the F5 to be involved!!
In prod, when client ARPs for VS_IP, F5 responds with MAC address of self-ip, which causes the problem, b/c node then trying to send back to client's MAC. That's how nPath should work - client sends to F5, F5 forwards to server, and server responds direct to client.
Also in LAB when I run tcpdump on the F5 I don't see any traffic, only ARP.
Yes because the client is sending only to the node as that is what responded.You need to prevent the servers from responding to the arp requests for the VS_IP (so get rid of that -arp setting), although they do need to have an interface that will respond to unicast traffic to that IP configured. While the F5 and the nodes are all responding to arp it's a race to see who gets there response in first. Once you have the F5 only responding then you'll need to troubleshoot the other problem.
- Dmitri_Ch__1425It was 'ARP race' problem indeed. After carefully reviewing tcpdump from both node and F5 I can see that both node and F5 responded on ARP request with it's own MAC address. What I don't understand why is loopback adapter involved in ARP resolution in 1st place and why would lo respond to requests on eth0... '-arp' option on ifconfig suppose to stop interface from responding to ARP (from man). 'ip link lo set arp off' worked better in this case. I'm also looking using arp_announce and arp_ignore sysctl options. Thank you very much for all your help.
- Dmitri_Ch__1425'ip link' command is: ip link set lo:0 arp off
