Wildcard forwarding for direct node traffic with PBR

Hi All,

Apologies if this question has been asked before; I've waded my way through a lot of forum posts but haven't seen the problem I'm facing - feel free to prove otherwise.

I am currently have a HA LTM pair with a single trunked interface (eight trunk members aggregated using LACP and with VLANs trunked on the connected switch). Policy based routing is used on the node VLANs to route matching traffic back to the F5 self IP interface for that VLAN. The default gateway on the nodes is not set to the F5 and all other traffic not matched by PBR uses the default gateway.

Everything works fine, however it is occasionally necessary to connect directly to a service on a node rather than the virtual server. In this case, return traffic is still being routed by PBR to the F5's. I have created a forwarding wildcard virtual server on 0.0.0.0/0 and all ports (with loose connection initiation, etc.), but am still not seeing the traffic being forwarded.

I am seeing the "in" traffic in the virtual server statistics and can match it to individual requests, but I am not seeing "out" traffic be incremented. Before I spend hours poring over packet dumps, can anyone suggest what the problem is likely to be? Is it possible that the F5 is not able to route the traffic, and if so where would I see evidence of this (if anywhere)?

Cheers

config

design

21 Replies

The_Bhattman
Nimbostratus
Jun 29, 2011
Hi Ltp,

This problem could be a number of things. It would make this a bit easier if you can provide a network diagram of how it's set up?

Bhattman
The_Bhattman
Nimbostratus
Jun 30, 2011
Hi LTP,

Seems like something is not working on the pbr side. How is your PBR accounting for the flow is between 10.4.0.0/24 towards 10.2.0.0/24?

Are you using Cisco or Nexus ?

If you are using Cisco or Nexus is it possible for you post up the PBR statement along with the access-list and route map commands?

Bhattman
ltp_55848
Nimbostratus
Jun 30, 2011
Hi Bhattman,

We are using Nexus, the following is an example of the applicable parts of the access list, policy and route map we are using (sanitised, made up addresses and names):

ip access-list TEST

10 permit tcp 10.4.0.0/16 eq www any

route-map TEST permit 10

match ip address TEST

set ip next-hop 10.4.0.10

interface VLAN_TEST

ip address 10.4.0.1/16

ip policy route-map TEST

I can see the reply packets from a request directly to the node in a tcpdump on the F5's so it seems as though my forwarding virtual server is not configured correctly.
Hamish
Cirrocumulus
Jun 30, 2011
F5's don't like asymetric routing.

The diagram seems to indicate that the return traffic is coming in a different interface from the outound to the poolmember. The F5 doesn't like this (AFAIK it hasn't been fixed between 9.4 where I first experienced it, and 10.x).

H
ltp_55848
Nimbostratus
Jun 30, 2011
Hi Hamish,

Previously we were using a Cisco CSM module to accomplish this traffic flow using the ROUTE_UNKNOWN_FLOW_PKTS variable to route non-SYN packets that did not match an existing flow. From the documentation I had assumed that the loose initiation option for a custom FastL4 profile would accomplish the same thing, however I may have misunderstood.
Hamish
Cirrocumulus
Jun 30, 2011
Hmm... That might be possible if you have a second network vs that leads back to the client...

However, you'd get two connection table entries... And I'm Not sure that'll be possible because they'd essentially be for the same flow... And would probably conflict... And fail anyway.

IIRC there was a good reason for it not being supported back when I tried (Circa 2007 I think). I don't believe it was going to be looked at for changing... But it's always possible... What version did u say u were running?

H
Luis_San_Miguel
Nimbostratus
Jun 30, 2011
Have you try to disable Auto Last Hop feature ?

http://support.f5.com/kb/en-us/solutions/public/11000/700/sol11796.html
hoolio
Cirrostratus
Jun 30, 2011
The diagram seems to indicate that the return traffic is coming in a different interface from the outbound to the poolmember. The F5 doesn't like this (AFAIK it hasn't been fixed between 9.4 where I first experienced it, and 10.x).

You can set a db key to have TMM accept responses into a different VLAN than they went out of:

b db Connection.VlanKeyed disable

Aaron
The_Bhattman
Nimbostratus
Jun 30, 2011
Hi Ltp,

Cisco did some changes with pBR within Nexus vs the IOS. Especially when ACL's do not allow DENY statements when using it under pBR.

Try the following as a test.

ip access-list TEST_deny

10 permit tcp 10.4.0.0/16 10.2.0.0/16

ip access-list TEST_allow

10 permit tcp 10.4.0.0/16 eq www any

route-map TEST deny 10

match ip address TEST_Deny

route-map TEST permit 20

match ip address TEST_alllow

set ip next-hop 10.4.0.10

I hope this helps

Bhattman
ltp_55848
Nimbostratus
Jun 30, 2011
Thanks for the responses guys. To clarify; I'm running 10.2.1, I tried disabling the auto last-hop feature and VLAN keyed connections without success.

I also came across this SOL (http://support.f5.com/kb/en-us/solutions/public/6000/400/sol6417.html) that suggests PVA is not compatible with asymmetric connections across multiple VLANs for version 10.2.0 but I'm not sure whether this would also apply to version 10.2.1.

Bhattman, thanks, I'll follow up the PBR configuration with one of our networking guys today. Also, do you have a link to any documentation detailing the differences between PBR on Nexus and IOS?