looking inside persistence table, through different Virtual Servers

Do you have a universal persistence profile with "Match Across Virtual Servers" (and possibly "Match across services" if the pool on the incoming VIP is defined on a specific port) enabled, to both virtual servers?

 

 

Are you using 'persist lookup uie $IP all' to get the existing record?

 

 

Can you post your rule, vips and profiles with logging?

 

 

Aaron

Here is a part of the bigip_base.conf from my test lab:

 

 

-----------------------------------

 

 

vlan PROD {

 

tag 4094

 

interfaces 1.1

 

}

 

vlan VLAN-RTR-Cisco {

 

tag 4093

 

interfaces 1.3

 

}

 

stp instance 0 {

 

vlans

 

PROD

 

VLAN-RTR-Cisco

 

interfaces

 

1.1

 

external path cost 20K

 

internal path cost 20K

 

1.3

 

external path cost 20K

 

internal path cost 20K

 

}

 

self allow {

 

default

 

tcp ssh

 

tcp domain

 

tcp snmp

 

tcp https

 

tcp 4353

 

udp domain

 

udp snmp

 

udp efs

 

udp 1026

 

udp 4353

 

proto ospf

 

}

 

self 192.168.100.141 {

 

netmask 255.255.255.0

 

vlan VLAN-RTR-Cisco

 

allow default

 

}

 

self 192.168.155.141 {

 

netmask 255.255.255.0

 

vlan PROD

 

allow default

 

 

 

 

 

and the bigip.conf :

 

-----------------------------

 

 

profile persist pers_univ {

 

defaults from universal

 

mode universal

 

timeout indefinite

 

across services enable

 

across virtuals enable

 

across pools disable

 

rule Persist-univ

 

}

 

 

pool RTR_OUT {

 

monitor all gateway_icmp

 

members 192.168.100.200:http

 

}

 

pool rtr_f5 {

 

monitor all gateway_icmp

 

members 192.168.155.254:any

 

}

 

rule Persist-univ {

 

when CLIENT_ACCEPTED {

 

set adr ""

 

log local0. "LINK vlan_id : [LINK::vlan_id]"

 

if {[LINK::vlan_id] == "4094"}

 

{

 

set adr [IP::client_addr]

 

log local0. "vlan 4094 from client to servers - $adr"

 

}

 

else

 

{

 

set adr [IP::local_addr]

 

log local0. "vlan 4093 from server to client - $adr"

 

}

 

set mac1 "00:15:c5:16:94:47"

 

This mac adress is hardcoded, for tests only

 

set pipo [concat [string range $adr 0 end]$mac1]

 

set toto [persist lookup uie $pipo all]

 

if { $toto == "" }

 

{

 

log local0. "UIE PERS - New persistence entry created"

 

persist uie $pipo

 

}

 

else

 

{

 

log local0. "UIE PERS - Persistence existing in the table"

 

}

 

}

 

}

 

rule iRule_Cli2Srv {

 

when CLIENT_ACCEPTED {

 

log local0. "client accepted"

 

adr is storing the IP of the client

 

set adr [IP::client_addr]

 

The mac is hardcoded for tests lab only

 

set mac1 "00:15:c5:16:94:47"

 

the uie stores the client IP and Mac of the last hop gateway

 

set test [concat [string range $adr 0 end]$mac1]

 

log local0. "test variable content : $test"

 

set toto [persist lookup uie $pipo]

 

if { $toto ==""}

 

{

 

log local0. "No persistence found in the uie table"

 

persist uie $test

 

log local0. "uie value stored in the table = $test"

 

}

 

else

 

{

 

log local0. "Persistence existing in the table : $test"

 

}

 

 

}

 

}

 

rule iRule_Srv2Cli {

 

when CLIENT_ACCEPTED {

 

log local0. "client accepted"

 

set adr1 [IP::local_addr]

 

set mac1 "00:15:c5:16:94:47"

 

set test [concat [string range $adr1 0 end]$mac1]

 

log local0. "Content of the variable test = $test"

 

set toto [persist lookup uie $pipo]

 

if { $toto == "" }

 

{

 

log local0. "No persistence Found"

 

persist uie $test

 

log local0. "persistence stored in the uie table = $test"

 

}

 

else

 

{

 

log local0. "Persistence found in the uie table = $test "

 

}

 

}

 

}

 

virtual address any {

 

mask none

 

}

 

virtual VIP_OUT {

 

pool rtr_f5

 

destination any:any

 

mask none

 

vlans VLAN-RTR-Cisco enable

 

rules iRule_Srv2Cli

 

persist pers_univ

 

}

 

virtual VS_IN {

 

snat automap

 

pool RTR_OUT

 

destination 192.168.155.108:http

 

ip protocol tcp

 

vlans PROD enable

 

rules iRule_Cli2Srv

 

persist pers_univ

 

}

 

 

 

 

 

AND HERE ARE THE LOGS OF MY TESTS:

 

----------------------------------------------------------

 

First connection to the VIP of the servers

 

 

Sep 17 18:44:26 tmm tmm[1666]: Rule: Persist-univ vlan 4094 from client to servers - 192.168.155.57

 

Sep 17 18:44:26 tmm tmm[1666]: Rule: iRule_Cli2Srv test variable content : 192.168.155.5700:15:c5:16:94:47

 

Sep 17 18:44:26 tmm tmm[1666]: Rule: iRule_Cli2Srv uie value stored in the table = 192.168.155.5700:15:c5:16:94:47

 

 

 

Second connection to the VIP of the servers

 

 

Sep 17 18:44:32 tmm tmm[1666]: Rule: Persist-univ vlan 4094 from client to servers - 192.168.155.57

 

Sep 17 18:44:32 tmm tmm[1666]: Rule: iRule_Cli2Srv test variable content : 192.168.155.5700:15:c5:16:94:47

 

Sep 17 18:44:32 tmm tmm[1666]: Rule: iRule_Cli2Srv Persistence existing in the table : 192.168.155.5700:15:c5:16:94:47

 

 

 

First connection From the Server to the client (.57)

 

 

Sep 17 18:44:57 tmm tmm[1666]: Rule: Persist-univ LINK vlan_id : 4093

 

Sep 17 18:44:57 tmm tmm[1666]: Rule: Persist-univ vlan 4093 from server to client - 192.168.155.57

 

Sep 17 18:44:57 tmm tmm[1666]: Rule: Persist-univ UIE PERS - New persistence entry created

 

Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli client accepted

 

Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli Content of the variable test = 192.168.155.5700:15:c5:16:94:47

 

Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli No persistence Found

 

Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli persistence stored in the uie table = 192.168.155.5700:15:c5:16:94:47

 

 

 

What is the content of the persistence table ???

 

2 entries with the same uie persistence value.

 

 

 

[root@bip:Active] config b persist all show all

 

PERSISTENT CONNECTIONS

 

| Mode universal Value 192.168.155.5700:15:c5:16:94:47

 

| virtual any:any node 192.168.155.254:any age 30sec

 

| Mode universal Value 192.168.155.5700:15:c5:16:94:47

 

| virtual 192.168.155.108:http node 192.168.100.200:http age 55sec

 

 

 

The config shown is the inside pair of a firewall sandwich.

The requirement here is to force server-initiated outbound connections to traverse the same firewall as inbound connections initiated by the same client.

I think the following approach would work:

On the external pair, persist inbound traffic by source IP.

On the internal pair, first create this class cross-referencing server MAC to IP address. Class type is "string". Insert the appropriate MAC address for each internal firewall IP (must match MAC address format as returned by LINK::lasthop):

class fw_MAC2IP {
  00:00:00:00:00:00 192.168.100.200
  00:00:00:00:00:00 192.168.100.201
}

The class will be used for direct outbound node selection by IP based on inbound lasthop MAC.

On the internal pair, persist inbound & outbound using the following iRule applied to both VS:

--- For inbound connections, it enforces simple persistence and saves lasthop mac by src IP for reciprocal traffic.

--- For outbound connections, it looks up dest IP in session table to get fw MAC, then looks up fw MAC in class to get fw IP.

rule bidirectional_firewall_persistence {
when RULE_INIT {
  set ::timeout 86400
}
when CLIENT_ACCEPTED {
  set server ""
  set origin_vlan [LINK::vlan_id]
  log local0. "vlan ID: $origin_vlan  client: [IP::remote_addr]"
  
  if {$origin_vlan == "4094"}{
     for client originated connections, use server pool & simple persistence 
    set session_key [IP::remote_addr]
    log local0. "client connecting from vlan $origin_vlan - session_key = >$session_key<"
     select inbound pool by name & apply simple persistence
    pool RTR-OUT
    persist source_addr 86400
     create/update session table entry for reciprocal traffic
    session add uie {$session_key any virtual} [LINK::lasthop] $::timeout
    log local0. "RTR-OUT pool selected, source_addr persistence & session table entry added for $session_key"
  } else {
     for server originated conns, look up nexthop MAC for that dest IP in session table
    set session_key [IP::local_addr]
    log local0. "server connecting from vlan $origin_vlan - session_key = >$session_key<"
    set fwMAC [session lookup uie {$session_key any virtual}]
    log local0. "fwMAC found: >$fwMAC< - session_key = >$session_key<"
    if {$fwMAC != ""}{
       If session table entry already exists, refresh to update timeout
      session add uie {$session_key any virtual} $fwMAC $::timeout
      log local0. "session table entry refreshed - session_key = >$session_key<"
       then look up fw IP address in class by MAC
      set fwIP [findclass $fwMAC $::fw_MAC2IP " "]
      log local0. "fw IP >$fwIP< found for MAC $fwMAC - session_key = >$session_key<"
      if {$fwIP != ""}{
         if fw IP address is found in the class for this MAC, directly select fw node
        log local0. "directly selecting fw node $fwIP - session_key = >$session_key<"
        node $fwIP 0
      } else {
         if no fw IP address was found in the class for this MAC, log an error
         This should never happen if class list contains the correct data in the correct format
        log local0. "No IP address found for MAC $fwMAC  Check class list for proper entries."
        log local0. "LB'ing outbound conn to rtr_f5 pool - session_key = >$session_key<"
         select outbound pool by name
        pool rtr_f5
      }
    } else {
       if no session table entry found, load balance the connection
      log local0. "No session table entry for $session_key. LB'ing outbound conn to rtr_f5 pool."
       select outbound pool by name
      pool rtr_f5
    }
  }
}
when LB_FAILED {
   detach and choose a new server if the selected server fails to respond.
  LB::detach
  LB::reselect
}
}

HTH

/deb

The result of this iRule is the following:

 

 

[root@Internal:Active] config b persist all show all

 

PERSISTENT CONNECTIONS

 

| Mode source addr Value 10.3.3.252

 

| virtual 10.3.3.50:80 node 10.5.5.100:80 age 144sec

 

| Mode universal Value $session_key

 

| virtual 10.3.3.50:80 node any:any age 102sec

 

 

 

I think there is a mistake in the line :

 

 

session add uie {$session_key any virtual} ....

 

 

the $session_key is not interpreted as a variable.

 

 

hmm, I have used that syntax before & not had a issue w/variable expansion... What version are you on?

You can try changing the 2 lines with that command to explicitly expand the variable:

session add uie {[list $session_key] any virtual} ....

If that doesn't work, seeing the logs along with the persist table would be helpful in debugging the situation.

HTH

/deb

Hi deb,

 

I test this irule. I have some problem. I can find the lasthop MAC, But I can't find out the lasthop's IP.

 

When I use real lasthop MAC to replace $fwMAC,I can find out the lasthop's IP.

 

-----------------------------------------------------------------------

 

set fwIP [findclass 00:d0:c9:96:83:f7 $::fw_MAC2IP " "] is OK.

 

set fwIP1 [findclass $fwMAC $::fw_MAC2IP " "] does't OK.

 

------------------------------------------------------------------------------

 

 

So I think there is some problem in $fwMAC.

 

I check below rule:

 

---------------------------------------------------------------------------

 

if {$origin_vlan == "4093"}{

 

for client originated connections, use server pool & simple persistence

 

set session_key [IP::remote_addr]

 

log local0. "client connecting from vlan $origin_vlan - session_key = >$session_key<"

 

select inbound pool by name & apply simple persistence

 

pool pool2-https

 

persist source_addr 86400

 

create/update session table entry for reciprocal traffic

 

session add uie {$session_key any virtual} [LINK::lasthop] $::timeout

 

log local0. "http_pool1 pool selected, source_addr persistence & session table entry added for $session_key"

 

set fwMAC [session lookup uie {$session_key any virtual}]

 

log local0. "session add uie is --->$fwMAC<---------"

 

------------------------------------------------------------------------------

 

when testing, the display is:

 

 

Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : vlan ID: 4093 client: 10.9.1.11

 

Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : client connecting from vlan 4093 - session_key = >10.9.1.11<

 

Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : http_pool1 pool selected, source_addr persistence & session table entry added for 10.9.1.11

 

Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : session add uie is --->00:d0:c9:96:83:f7

 

Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : vlan ID: 4093 client: 10.9.1.11

 

Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : client connecting from vlan 4093 - session_key = >10.9.1.11<

 

Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : http_pool1 pool selected, source_addr persistence & session table entry added for 10.9.1.11

 

Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : session add uie is --->00:d0:c9:96:83:f7

 

 

 

The "session add uie is --->00:d0:c9:96:83:f7" does not include" <-----"

 

So I think there is some problem about the $fwMAC.

 

 

Would you pls help check what has happen? For I am urgent to use this iRule.

 

Thanks.

 

 

 

 

hmm....

The variable itself seems to be expanding just fine, but I have no idea why the trailing characters are not logged.

It might be worth looking at the output of "b list " to see if those characters are in the config. If the rule looks correct in the config (well really either way), I'd recommend contacting F5 Support to look into the reason for the missing characters in the log command output.

You could try using the MAC value in a slightly different way, I guess. Maybe just checking the last 2 characters in the MAC?

 set fwIP [findclass [getfield $fwMAC ":" 6] $::fw_MAC2IP " "] 

against a modified class:

class fw_MAC2IP {
  10.9.1.11 f7
  10.9.1.12 f8
}

HTH

/deb

Here's an updated but untested version of the iRule for 10.1.0 or higher:

rule bidirectional_firewall_persistence {

when RULE_INIT {

 Log debug to /var/log/ltm? (2=all including debug, 1=only log missing mac2ip datagroup entries, 0=none)
set static::fw_debug 1

 Time in seconds to store mappings for
set static::timeout 86400

 Pool name for outbound routers
set static::outbound_pool "outbound_pool"

 Pool name for inbound routers
set static::inbound_pool "inbound_pool"

 Name of string datagroup which maps mac addresses to IP addresses for the routers
set static::mac2ip_class "mac2ip_class"

 VLAN ID of upstream clients
set static::client_vlanid 4094
}

when CLIENT_ACCEPTED {

set server ""
set origin_vlan [LINK::vlan_id]
if {$static::fw_debug}{log local0. "vlan ID: $origin_vlan client: [IP::client_addr]"}

 Check if this a client originated connection by VLAN ID
if {$origin_vlan == $static::client_vlanid}{

 for client originated connections, use server pool & simple persistence 
set session_key [IP::client_addr]
if {$static::fw_debug}{log local0. "client connecting from vlan $origin_vlan - session_key = \"$session_key\""}

 select inbound pool by name & apply simple persistence
pool $static::ingbound_pool
persist source_addr 86400

 create/update session table entry for reciprocal traffic
session add uie [list $session_key any virtual] [LINK::lasthop] $static::timeout
if {$static::fw_debug}{log local0. "$static::router_out_pool pool selected, source_addr persistence & session table entry added for $session_key"}

} else {

 for server originated conns, look up nexthop MAC for that dest IP in session table
set session_key [IP::local_addr]
if {$static::fw_debug}{log local0. "server connecting from vlan $origin_vlan - session_key = \"$session_key\""}

set fwMAC [session lookup uie [list $session_key any virtual]]
if {$static::fw_debug}{log local0. "fwMAC found: \"$fwMAC\" - session_key = \"$session_key\""}
if {$fwMAC ne ""}{

 If session table entry already exists, refresh to update timeout
session add uie [list $session_key any virtual] $fwMAC $static::timeout
if {$static::fw_debug}{log local0. "session table entry refreshed - session_key = \"$session_key\""}

 then look up firewall MAC address in the datagroup to get the firewall IP address
set fwIP [class match -value $fwMAC equals $static::mac2ip_class]
if {$static::fw_debug}{log local0. "fw IP \"$fwIP\" found for MAC $fwMAC - session_key = \"$session_key\""}
if {$fwIP ne ""}{

 if fw IP address is found in the class for this MAC, directly select fw node
if {$static::fw_debug}{log local0. "directly selecting fw node $fwIP - session_key = \"$session_key\""}
node $fwIP 0

} else {

 if no fw IP address was found in the class for this MAC, log an error
 This should never happen if class list contains the correct data in the correct format
if {$static::fw_debug}{log local0. "No IP address found for MAC $fwMAC Check class list for proper entries."}
if {$static::fw_debug}{log local0. "LB'ing outbound conn to $static::outbound_pool pool - session_key = \"$session_key\""}
 select outbound pool by name
pool $static::outbound_pool
}
} else {
 if no session table entry found, load balance the connection
if {$static::fw_debug}{log local0. "No session table entry for $session_key. LB'ing outbound conn to $static::outbound_pool pool."}
 select outbound pool by name
pool $static::outbound_pool
}
}
}

when LB_FAILED {
 detach and choose a new server if the selected server fails to respond.
LB::detach
LB::reselect
}
}

Aaron