Forum Discussion

Dmitri_Ch__1425's avatar
Mar 24, 2014

nPath and firewall rules

Hi

 

Does nPath need special firewall configuration?

 

I've configured nPath in LAB, everything seems to be working fine. But I cannot get it to work on production network and suspect firewall blocking return traffic from nodes.

 

Are there any firewall rules needed on for nPath to work properly?

 

Thank you.

 

  • What is interesting is in LAB env. when client ARPs for VS_IP one of the node responds with it's MAC address and everything works OK then. That means the F5 will not be involved at all, and the whole purpose of nPath is for the F5 to be involved!!

     

    In prod, when client ARPs for VS_IP, F5 responds with MAC address of self-ip, which causes the problem, b/c node then trying to send back to client's MAC. That's how nPath should work - client sends to F5, F5 forwards to server, and server responds direct to client.

     

    Also in LAB when I run tcpdump on the F5 I don't see any traffic, only ARP.

     

    Yes because the client is sending only to the node as that is what responded.

     

    You need to prevent the servers from responding to the arp requests for the VS_IP (so get rid of that -arp setting), although they do need to have an interface that will respond to unicast traffic to that IP configured. While the F5 and the nodes are all responding to arp it's a race to see who gets there response in first. Once you have the F5 only responding then you'll need to troubleshoot the other problem.

     

9 Replies

  • It depends where in the network topology the firewall is.

     

    If the firewall is between the F5 and the server then you are likely to have problems.

     

    If both F5 and server are behind the firewall, and behind the same interface of the firewall, then you should not have an issue.

     

  • Hi, thank you. If there is nothing on firewall then I'll double-check F5 config.

     

    F5's VS, node and firewall are all on the same subnet. The concern I have with firewall is it could drop reply packets coming from the node since firewall doesn't have incoming connection to match. Could this be the problem?

     

    I ran tcpdump on F5 and node and see that node is sending data back (dst is MAC address of FW), but my client doesn't receive those.

     

  • I took one more look at tcpdump result on the node. I see that node gets SYN from client_IP with MAC address of F5's interface. Then node ARPs for MAC of the client_IP, gets MAC address, which is different from what's in SYN packet and sends SYN,ACK to client_IP /w client_MAC address. And client drop it, which is what I expect in this case.

     

    The node's lo:0 configured as :

     

    ifconfig lo:0 netmask 255.255.255.255 -arp up

     

    Autolasthop is enabled on F5, both global and VS level.

     

    • IheartF5_45022's avatar
      IheartF5_45022
      Icon for Nacreous rankNacreous
      why do you say "And client drop it, which is what I expect in this case"? SYN-ACK with client MAC as dest should not be dropped.
  • bwolmarans_1284's avatar
    bwolmarans_1284
    Historic F5 Account

    Is the Client on the same subnet as the server? If not, then the server should ARP for the gateway IP, and send packet that has the client IP and gateway MAC.

     

    • Dmitri_Ch__1425's avatar
      Dmitri_Ch__1425
      Icon for Cirrus rankCirrus
      Yes, in this test the client is on the same subnet. I'm trying to pinpoint the problem and avoid firewall for now. What is interesting is in LAB env. when client ARPs for VS_IP one of the node responds with it's MAC address and everything works OK then. In prod, when client ARPs for VS_IP, F5 responds with MAC address of self-ip, which causes the problem, b/c node then trying to send back to client's MAC. Also in LAB when I run tcpdump on the F5 I don't see any traffic, only ARP. I don't know if it make any difference - in LAB I'm using Ubuntu nodes, in prod Redhat nodes. Same F5 unit, different VLANs on trunk port...
  • What is interesting is in LAB env. when client ARPs for VS_IP one of the node responds with it's MAC address and everything works OK then. That means the F5 will not be involved at all, and the whole purpose of nPath is for the F5 to be involved!!

     

    In prod, when client ARPs for VS_IP, F5 responds with MAC address of self-ip, which causes the problem, b/c node then trying to send back to client's MAC. That's how nPath should work - client sends to F5, F5 forwards to server, and server responds direct to client.

     

    Also in LAB when I run tcpdump on the F5 I don't see any traffic, only ARP.

     

    Yes because the client is sending only to the node as that is what responded.

     

    You need to prevent the servers from responding to the arp requests for the VS_IP (so get rid of that -arp setting), although they do need to have an interface that will respond to unicast traffic to that IP configured. While the F5 and the nodes are all responding to arp it's a race to see who gets there response in first. Once you have the F5 only responding then you'll need to troubleshoot the other problem.

     

    • Dmitri_Ch__1425's avatar
      Dmitri_Ch__1425
      Icon for Cirrus rankCirrus
      It was 'ARP race' problem indeed. After carefully reviewing tcpdump from both node and F5 I can see that both node and F5 responded on ARP request with it's own MAC address. What I don't understand why is loopback adapter involved in ARP resolution in 1st place and why would lo respond to requests on eth0... '-arp' option on ifconfig suppose to stop interface from responding to ARP (from man). 'ip link lo set arp off' worked better in this case. I'm also looking using arp_announce and arp_ignore sysctl options. Thank you very much for all your help.