Two BIG-IP/Viprion LTMs and routing

Hi there,

I am trying to resolve the following issue:

we have two BIG-IP LTMs HA pairs (two 8800s and two Viprions, 4 units in total) installed. The setup is as follows, each HA pair is displayed as one box for simplicity:

The internet (ex)   : 2001:ad0::/32
Company Ipv6 prefix : 2001:db8::/32
  The office IPv6   : 2001:db8:4000::/48
  Datacenter IPv6   : 2001:db8:0001::/48
    Public IPv6     : 2001:db8:0001:0000::/112
    Private VLAN84  : 2001:db8:0001:0084::/64 
                    +------------+
                    |  Internet  |
                    +------------+
                          |
                 +-------------------+
                 |                   |     +-------------------------+
                 |      Router       +-----|  Office                 |
                 |                   |     | 2001:db8:4000:100::1/64 |
                 |                   |     +-------------------------+
                 +----+--------+-----+
                      |        |
     +----------------+    [ VLAN256 ]
     |                     [ 2001:db8:1::1/112 ]
 [ VLAN84 ]                    |
 [ 2001:db8:1:84::1/64 ]       |
     |                         |
     |                         |
     |               +---------+-- [ VLAN256 ] ----------+
     |               |                                   |
     |               | [ VS A: 2001:db8:1::120/112 ]     | [ VS B: 2001:db8:1::130/112 ]
     |           +---+----------------------------+  +---+------------------------------+
     |           |   Viprion w/Virtual Server A   |  |  BIG-IP 8800 w/Virtual Server B  |
     |           +---+----------------------------+  +---+------------------------------+
     |               | [ 2001:db8:1:84::f0/64 ]          | [ 2001:db8:1:84::e0/64 ]
     |               |                                   | 
     |               |                                   |
     +------+--------+-- [ VLAN84 ] ---+---- [ VLAN84 ] -+-----+------ [ VLAN84 ] -------+
            |                          |                       |                         |
        +---+------+               +---+------+            +---+------+              +---+------+
        | Linux A1 |               | Linux A2 |            | Linux B1 |              | Linux B2 |
        +----------+               +----------+            +----------+              +----------+
  [ 2001:db8:1:84::121/64 ]  [ 2001:db8:1:84::122/64 ]  [ 2001:db8:1:84::131/64 ]  [ 2001:db8:1:84::132/64 ]
  [ ---------- gateway: 2001:db8:1:84::f0 ---------- ]  [ ---------- gateway: 2001:db8:1:84::e0 ---------- ]

I am currently trying to configure IPv6 on the network, which includes load balancing. So far I have acchieved the following:

- I can access the servers from office (path Office->Router->Linux* works perfectly)

- I can access the public IPv6 addresses from bothe the office and outside (Internet/Office->Router->VLAN256 is fine)

- I can access the public IPv6 address 2001:db8:1::111 of Virtual server A from the linux server behind LTM A (and also IPv6 address 2001:db8:1::112 of Virtual server B from the linux server behind the LTM B) -- see note 1

- I cannot access the public IPv6 address 2001:db8:1::111 of Virtual server B from the linux server behind LTM B and vice versa. It looks like the LTMs do not forward packets accordingly, or like there is a routing misconfiguration somewhere. As you can see, the IPv6 addresses are in the same subnet, but still not accessible. See note 2

 

*Note 1:

LTM's IPv6 NAT is in place. The packet comes from the internet/office/other source via VLAN256 with destination address=2001:db8:1::130 or 2001:db8:1::120. One of the LTMs accepts the packet (because it has the corresponding address on it) and rewrites destination address to either Linux A1/A2's address (:121/:122, in case of Virtual server A) or Linux B1/B2's address (:131/:132, in case of Virtual server B).

For this NAT to work from within the office, we had to set up the following routing environment:

- Linux boxes: default gw points to a floating IP on the corresponding LTM box (LTM A for Linux A*, LTM B for Linux B*)

- Linux boxes: route to addresses from VLAN256 (2001:db8:1::/112) points to a floating IP on the corresponding LTM box

- Linux boxes: company network prefix (2001:db8::/32) points to router's IP (2001:db8:1:84::1)

- LTM boxes: an iRule is in place for company network prefix (2001:db8::/32) that translates source address of the packets coming from network 2001:db8::/32 and trying to reach the virtual server at 2001:db8:1::1x0. The new source address is the same as the virtual server's address (e.g. 2001:db8:1::1x0).

Without the iRule, we wouldn't be able to access virtual servers, because Linux A*/Linux B* would send the packets directly via Router with wrong source port numbers (the virtual servers' port numbers differ from the port numbers on linux boxes). Without the route of 2001:db8::/32 on the linux boxes, we wouldn't be able to access these boxes via IPv6 from within the office (the LTM boxes apparently do not route traffic through).

*Note 2:

The problem might be with LTM routing. We have wildcard virtual servers set up (type Forwarding (IP), dst net: ::, dst netmask: ::), however this does not seem to fix the issue. The packet flow seems to be as follows:

Linux A1->LTM A->LTM B (dst address is rewritten, source address is rewritten)->Linux B1

Somewhere in this chain the problem occurs, leading to packets not coming back to Linux A1.

This is driving me nuts. In IPv6, NATs are not used (and I do not approve them), however I am not sure if there is a possibility to configure load-balancing using Virtual server of type Performance (L4) without NAT.

If we don't get this issue solved, we may need to set up a different VLAN/different IPv6 network for the second load balancer. The problem might disappear then... or not. We haven't tried this set up yet.

If there is a better/recommended set up for IPv6-IPv6 load balancing, please, suggest it here.

Thank you all.