Forum Discussion
Philippe_CLOUP
Employee
Employeelooking inside persistence table, through different Virtual Servers
I have created an irule to store, using universal persistence, the 2 things:
- the Client-IP adress connected to the VS.
- the Mac adress of the "lasthop" that sent the request to the VS.
Why ?
Because i need to send all the trafic generated from the server inside the pool, to the client IP, THROUGH THE SAME EQUIPMENT THAT SENT the incoming connection.
I created a second iRule, to catch the persistence created on the incoming VS, each time the server is generating an outgoing connection (created a wildcard VS, in perf L4, with "any protocols" - CLIENT_ACCEPTED event, persist across virtuals selected).
The problem is that i cannot find the entry in the persistence table (persist lookup uie ....). If i try to add it in the table via this irule, the only entry i find is the incoming one, with the "aging time" resetted.
DO YOU THINK THE PERSISTENCE TABLE CANNOT BE SHARED THROUGH THE OTHER VS, IF THEY ARE ON DIFFERENT VLANS WITH DIFFERENT NODES AND POOLS ?
Thanks.
8 Replies
hoolioCirrostratus
Do you have a universal persistence profile with "Match Across Virtual Servers" (and possibly "Match across services" if the pool on the incoming VIP is defined on a specific port) enabled, to both virtual servers?
Are you using 'persist lookup uie $IP all' to get the existing record?
Can you post your rule, vips and profiles with logging?
Aaron- Philippe_CLOUPHere is a part of the bigip_base.conf from my test lab:
-----------------------------------
vlan PROD {
tag 4094
interfaces 1.1
}
vlan VLAN-RTR-Cisco {
tag 4093
interfaces 1.3
}
stp instance 0 {
vlans
PROD
VLAN-RTR-Cisco
interfaces
1.1
external path cost 20K
internal path cost 20K
1.3
external path cost 20K
internal path cost 20K
}
self allow {
default
tcp ssh
tcp domain
tcp snmp
tcp https
tcp 4353
udp domain
udp snmp
udp efs
udp 1026
udp 4353
proto ospf
}
self 192.168.100.141 {
netmask 255.255.255.0
vlan VLAN-RTR-Cisco
allow default
}
self 192.168.155.141 {
netmask 255.255.255.0
vlan PROD
allow default
and the bigip.conf :
-----------------------------
profile persist pers_univ {
defaults from universal
mode universal
timeout indefinite
across services enable
across virtuals enable
across pools disable
rule Persist-univ
}
pool RTR_OUT {
monitor all gateway_icmp
members 192.168.100.200:http
}
pool rtr_f5 {
monitor all gateway_icmp
members 192.168.155.254:any
}
rule Persist-univ {
when CLIENT_ACCEPTED {
set adr ""
log local0. "LINK vlan_id : [LINK::vlan_id]"
if {[LINK::vlan_id] == "4094"}
{
set adr [IP::client_addr]
log local0. "vlan 4094 from client to servers - $adr"
}
else
{
set adr [IP::local_addr]
log local0. "vlan 4093 from server to client - $adr"
}
set mac1 "00:15:c5:16:94:47"
This mac adress is hardcoded, for tests only
set pipo [concat [string range $adr 0 end]$mac1]
set toto [persist lookup uie $pipo all]
if { $toto == "" }
{
log local0. "UIE PERS - New persistence entry created"
persist uie $pipo
}
else
{
log local0. "UIE PERS - Persistence existing in the table"
}
}
}
rule iRule_Cli2Srv {
when CLIENT_ACCEPTED {
log local0. "client accepted"
adr is storing the IP of the client
set adr [IP::client_addr]
The mac is hardcoded for tests lab only
set mac1 "00:15:c5:16:94:47"
the uie stores the client IP and Mac of the last hop gateway
set test [concat [string range $adr 0 end]$mac1]
log local0. "test variable content : $test"
set toto [persist lookup uie $pipo]
if { $toto ==""}
{
log local0. "No persistence found in the uie table"
persist uie $test
log local0. "uie value stored in the table = $test"
}
else
{
log local0. "Persistence existing in the table : $test"
}
}
}
rule iRule_Srv2Cli {
when CLIENT_ACCEPTED {
log local0. "client accepted"
set adr1 [IP::local_addr]
set mac1 "00:15:c5:16:94:47"
set test [concat [string range $adr1 0 end]$mac1]
log local0. "Content of the variable test = $test"
set toto [persist lookup uie $pipo]
if { $toto == "" }
{
log local0. "No persistence Found"
persist uie $test
log local0. "persistence stored in the uie table = $test"
}
else
{
log local0. "Persistence found in the uie table = $test "
}
}
}
virtual address any {
mask none
}
virtual VIP_OUT {
pool rtr_f5
destination any:any
mask none
vlans VLAN-RTR-Cisco enable
rules iRule_Srv2Cli
persist pers_univ
}
virtual VS_IN {
snat automap
pool RTR_OUT
destination 192.168.155.108:http
ip protocol tcp
vlans PROD enable
rules iRule_Cli2Srv
persist pers_univ
}
AND HERE ARE THE LOGS OF MY TESTS:
----------------------------------------------------------
First connection to the VIP of the servers
Sep 17 18:44:26 tmm tmm[1666]: Rule: Persist-univ vlan 4094 from client to servers - 192.168.155.57
Sep 17 18:44:26 tmm tmm[1666]: Rule: iRule_Cli2Srv test variable content : 192.168.155.5700:15:c5:16:94:47
Sep 17 18:44:26 tmm tmm[1666]: Rule: iRule_Cli2Srv uie value stored in the table = 192.168.155.5700:15:c5:16:94:47
Second connection to the VIP of the servers
Sep 17 18:44:32 tmm tmm[1666]: Rule: Persist-univ vlan 4094 from client to servers - 192.168.155.57
Sep 17 18:44:32 tmm tmm[1666]: Rule: iRule_Cli2Srv test variable content : 192.168.155.5700:15:c5:16:94:47
Sep 17 18:44:32 tmm tmm[1666]: Rule: iRule_Cli2Srv Persistence existing in the table : 192.168.155.5700:15:c5:16:94:47
First connection From the Server to the client (.57)
Sep 17 18:44:57 tmm tmm[1666]: Rule: Persist-univ LINK vlan_id : 4093
Sep 17 18:44:57 tmm tmm[1666]: Rule: Persist-univ vlan 4093 from server to client - 192.168.155.57
Sep 17 18:44:57 tmm tmm[1666]: Rule: Persist-univ UIE PERS - New persistence entry created
Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli client accepted
Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli Content of the variable test = 192.168.155.5700:15:c5:16:94:47
Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli No persistence Found
Sep 17 18:44:57 tmm tmm[1666]: Rule: iRule_Srv2Cli persistence stored in the uie table = 192.168.155.5700:15:c5:16:94:47
What is the content of the persistence table ???
2 entries with the same uie persistence value.
[root@bip:Active] config b persist all show all
PERSISTENT CONNECTIONS
| Mode universal Value 192.168.155.5700:15:c5:16:94:47
| virtual any:any node 192.168.155.254:any age 30sec
| Mode universal Value 192.168.155.5700:15:c5:16:94:47
| virtual 192.168.155.108:http node 192.168.100.200:http age 55sec - The config shown is the inside pair of a firewall sandwich.
The requirement here is to force server-initiated outbound connections to traverse the same firewall as inbound connections initiated by the same client.
I think the following approach would work:
On the external pair, persist inbound traffic by source IP.
On the internal pair, first create this class cross-referencing server MAC to IP address. Class type is "string". Insert the appropriate MAC address for each internal firewall IP (must match MAC address format as returned by LINK::lasthop):
The class will be used for direct outbound node selection by IP based on inbound lasthop MAC.class fw_MAC2IP { 00:00:00:00:00:00 192.168.100.200 00:00:00:00:00:00 192.168.100.201 }
On the internal pair, persist inbound & outbound using the following iRule applied to both VS:
--- For inbound connections, it enforces simple persistence and saves lasthop mac by src IP for reciprocal traffic.
--- For outbound connections, it looks up dest IP in session table to get fw MAC, then looks up fw MAC in class to get fw IP.rule bidirectional_firewall_persistence { when RULE_INIT { set ::timeout 86400 } when CLIENT_ACCEPTED { set server "" set origin_vlan [LINK::vlan_id] log local0. "vlan ID: $origin_vlan client: [IP::remote_addr]" if {$origin_vlan == "4094"}{ for client originated connections, use server pool & simple persistence set session_key [IP::remote_addr] log local0. "client connecting from vlan $origin_vlan - session_key = >$session_key<" select inbound pool by name & apply simple persistence pool RTR-OUT persist source_addr 86400 create/update session table entry for reciprocal traffic session add uie {$session_key any virtual} [LINK::lasthop] $::timeout log local0. "RTR-OUT pool selected, source_addr persistence & session table entry added for $session_key" } else { for server originated conns, look up nexthop MAC for that dest IP in session table set session_key [IP::local_addr] log local0. "server connecting from vlan $origin_vlan - session_key = >$session_key<" set fwMAC [session lookup uie {$session_key any virtual}] log local0. "fwMAC found: >$fwMAC< - session_key = >$session_key<" if {$fwMAC != ""}{ If session table entry already exists, refresh to update timeout session add uie {$session_key any virtual} $fwMAC $::timeout log local0. "session table entry refreshed - session_key = >$session_key<" then look up fw IP address in class by MAC set fwIP [findclass $fwMAC $::fw_MAC2IP " "] log local0. "fw IP >$fwIP< found for MAC $fwMAC - session_key = >$session_key<" if {$fwIP != ""}{ if fw IP address is found in the class for this MAC, directly select fw node log local0. "directly selecting fw node $fwIP - session_key = >$session_key<" node $fwIP 0 } else { if no fw IP address was found in the class for this MAC, log an error This should never happen if class list contains the correct data in the correct format log local0. "No IP address found for MAC $fwMAC Check class list for proper entries." log local0. "LB'ing outbound conn to rtr_f5 pool - session_key = >$session_key<" select outbound pool by name pool rtr_f5 } } else { if no session table entry found, load balance the connection log local0. "No session table entry for $session_key. LB'ing outbound conn to rtr_f5 pool." select outbound pool by name pool rtr_f5 } } } when LB_FAILED { detach and choose a new server if the selected server fails to respond. LB::detach LB::reselect } }
HTH
/deb - Philippe_CLOUPThe result of this iRule is the following:
[root@Internal:Active] config b persist all show all
PERSISTENT CONNECTIONS
| Mode source addr Value 10.3.3.252
| virtual 10.3.3.50:80 node 10.5.5.100:80 age 144sec
| Mode universal Value $session_key
| virtual 10.3.3.50:80 node any:any age 102sec
I think there is a mistake in the line :
session add uie {$session_key any virtual} ....
the $session_key is not interpreted as a variable. - hmm, I have used that syntax before & not had a issue w/variable expansion... What version are you on?
You can try changing the 2 lines with that command to explicitly expand the variable:session add uie {[list $session_key] any virtual} ....
If that doesn't work, seeing the logs along with the persist table would be helpful in debugging the situation.
HTH
/deb - Hi deb,
I test this irule. I have some problem. I can find the lasthop MAC, But I can't find out the lasthop's IP.
When I use real lasthop MAC to replace $fwMAC,I can find out the lasthop's IP.
-----------------------------------------------------------------------
set fwIP [findclass 00:d0:c9:96:83:f7 $::fw_MAC2IP " "] is OK.
set fwIP1 [findclass $fwMAC $::fw_MAC2IP " "] does't OK.
------------------------------------------------------------------------------
So I think there is some problem in $fwMAC.
I check below rule:
---------------------------------------------------------------------------
if {$origin_vlan == "4093"}{
for client originated connections, use server pool & simple persistence
set session_key [IP::remote_addr]
log local0. "client connecting from vlan $origin_vlan - session_key = >$session_key<"
select inbound pool by name & apply simple persistence
pool pool2-https
persist source_addr 86400
create/update session table entry for reciprocal traffic
session add uie {$session_key any virtual} [LINK::lasthop] $::timeout
log local0. "http_pool1 pool selected, source_addr persistence & session table entry added for $session_key"
set fwMAC [session lookup uie {$session_key any virtual}]
log local0. "session add uie is --->$fwMAC<---------"
------------------------------------------------------------------------------
when testing, the display is:
Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : vlan ID: 4093 client: 10.9.1.11
Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : client connecting from vlan 4093 - session_key = >10.9.1.11<
Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : http_pool1 pool selected, source_addr persistence & session table entry added for 10.9.1.11
Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : session add uie is --->00:d0:c9:96:83:f7
Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : vlan ID: 4093 client: 10.9.1.11
Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : client connecting from vlan 4093 - session_key = >10.9.1.11<
Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : http_pool1 pool selected, source_addr persistence & session table entry added for 10.9.1.11
Dec 4 18:04:45 tmm tmm[1052]: Rule bidirectional_firewall_persistence : session add uie is --->00:d0:c9:96:83:f7
The "session add uie is --->00:d0:c9:96:83:f7" does not include" <-----"
So I think there is some problem about the $fwMAC.
Would you pls help check what has happen? For I am urgent to use this iRule.
Thanks. - hmm....
The variable itself seems to be expanding just fine, but I have no idea why the trailing characters are not logged.
It might be worth looking at the output of "b list " to see if those characters are in the config. If the rule looks correct in the config (well really either way), I'd recommend contacting F5 Support to look into the reason for the missing characters in the log command output.
You could try using the MAC value in a slightly different way, I guess. Maybe just checking the last 2 characters in the MAC?
against a modified class:set fwIP [findclass [getfield $fwMAC ":" 6] $::fw_MAC2IP " "]class fw_MAC2IP { 10.9.1.11 f7 10.9.1.12 f8 }
HTH
/deb - hoolioHere's an updated but untested version of the iRule for 10.1.0 or higher:
rule bidirectional_firewall_persistence { when RULE_INIT { Log debug to /var/log/ltm? (2=all including debug, 1=only log missing mac2ip datagroup entries, 0=none) set static::fw_debug 1 Time in seconds to store mappings for set static::timeout 86400 Pool name for outbound routers set static::outbound_pool "outbound_pool" Pool name for inbound routers set static::inbound_pool "inbound_pool" Name of string datagroup which maps mac addresses to IP addresses for the routers set static::mac2ip_class "mac2ip_class" VLAN ID of upstream clients set static::client_vlanid 4094 } when CLIENT_ACCEPTED { set server "" set origin_vlan [LINK::vlan_id] if {$static::fw_debug}{log local0. "vlan ID: $origin_vlan client: [IP::client_addr]"} Check if this a client originated connection by VLAN ID if {$origin_vlan == $static::client_vlanid}{ for client originated connections, use server pool & simple persistence set session_key [IP::client_addr] if {$static::fw_debug}{log local0. "client connecting from vlan $origin_vlan - session_key = \"$session_key\""} select inbound pool by name & apply simple persistence pool $static::ingbound_pool persist source_addr 86400 create/update session table entry for reciprocal traffic session add uie [list $session_key any virtual] [LINK::lasthop] $static::timeout if {$static::fw_debug}{log local0. "$static::router_out_pool pool selected, source_addr persistence & session table entry added for $session_key"} } else { for server originated conns, look up nexthop MAC for that dest IP in session table set session_key [IP::local_addr] if {$static::fw_debug}{log local0. "server connecting from vlan $origin_vlan - session_key = \"$session_key\""} set fwMAC [session lookup uie [list $session_key any virtual]] if {$static::fw_debug}{log local0. "fwMAC found: \"$fwMAC\" - session_key = \"$session_key\""} if {$fwMAC ne ""}{ If session table entry already exists, refresh to update timeout session add uie [list $session_key any virtual] $fwMAC $static::timeout if {$static::fw_debug}{log local0. "session table entry refreshed - session_key = \"$session_key\""} then look up firewall MAC address in the datagroup to get the firewall IP address set fwIP [class match -value $fwMAC equals $static::mac2ip_class] if {$static::fw_debug}{log local0. "fw IP \"$fwIP\" found for MAC $fwMAC - session_key = \"$session_key\""} if {$fwIP ne ""}{ if fw IP address is found in the class for this MAC, directly select fw node if {$static::fw_debug}{log local0. "directly selecting fw node $fwIP - session_key = \"$session_key\""} node $fwIP 0 } else { if no fw IP address was found in the class for this MAC, log an error This should never happen if class list contains the correct data in the correct format if {$static::fw_debug}{log local0. "No IP address found for MAC $fwMAC Check class list for proper entries."} if {$static::fw_debug}{log local0. "LB'ing outbound conn to $static::outbound_pool pool - session_key = \"$session_key\""} select outbound pool by name pool $static::outbound_pool } } else { if no session table entry found, load balance the connection if {$static::fw_debug}{log local0. "No session table entry for $session_key. LB'ing outbound conn to $static::outbound_pool pool."} select outbound pool by name pool $static::outbound_pool } } } when LB_FAILED { detach and choose a new server if the selected server fails to respond. LB::detach LB::reselect } }
Aaron
