Dynamic FQDN Node DNS Resolution based on URI with Route Domains and Caching iRule
Problem this snippet solves: Following on from the code share Ephemeral Node FQDN Resolution with Route Domains - DNS Caching iRule that I posted, I have made a few modifications based on further development and comments/questions in the original share. On an incoming HTTP request, this iRule will dynamically query a DNS server (out of the appropriate route domain) defined in a pool to determine the IP address of an FQDN ephemeral node depending on the requesting URI. The IP address will be cached in a subtable so to prevent querying on every HTTP request It will also append the route domain to the node and replace the HTTP Host header. Features of this iRule * Dynamically resolves FQDN node from requesting URI * Iterates through other DNS servers in a pool if response is invalid or no response * Caches response in the session table using custom TTL * Uses cached result if present (prevents DNS lookup on each HTTP_REQUEST event) * Appends route domain to the resolved node * Replaces host header for outgoing request How to use this snippet: Add a DNS pool with appropriate health monitors, in this example the pool is called dns_pool Add the lookup datagroup, dns_lookup_dg . This defines the parameters of the DNS query. The values in the datagroup will built up an array using the key in capitals to define the array object e.g. $myArray(FQDN) Modify the values as required: FQDN: the FQDN of the node to load balance to. DNS-RD: the outbound route domain to reach the DNS servers NODE-RD: the outbound route domain to reach the node TTL: TTL value for the DNS cache in seconds ltm data-group internal dns_lookup_dg { records { /app1 { data "FQDN app1.my-domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 8443" } /app2 { data "FQDN app2.my-other-domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 8080" } default { data "FQDN default.domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 443" } } type string } Code : ltm data-group internal dns_lookup_dg { records { /app1 { data "FQDN app1.my-domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 8443" } /app2 { data "FQDN app2.my-other-domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 8080" } default { data "FQDN default.domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 443" } } type string } when CLIENT_ACCEPTED { set dnsPool "dns_pool" set dnsCache "dns_cache" set dnsDG "dns_lookup_dg" } when HTTP_REQUEST { # set datagroup values in an array if {[class match -value [HTTP::uri] "starts_with" $dnsDG]} { set dnsValues [split [string trim [class match -value [HTTP::uri] "starts_with" $dnsDG]] "| "] } else { # set to default if URI is not defined in DG set dnsValues [split [string trim [class match -value "default" "equals" $dnsDG]] "| "] } if {([info exists dnsValues]) && ($dnsValues ne "")} { if {[catch { array set dnsArray $dnsValues set fqdn $dnsArray(FQDN) set dnsRd $dnsArray(DNS-RD) set nodeRd $dnsArray(NODE-RD) set ttl $dnsArray(TTL) set port $dnsArray(PORT) } catchErr ]} { log local0. "failed to set DNS variables - error: $catchErr" event disable return } } # check if fqdn has been previously resolved and cached in the session table if {[table lookup -notouch -subtable $dnsCache $fqdn] eq ""} { log local0. "IP is not cached in the subtable - attempting to resolve IP using DNS" # initialise loop vars set flgResolvError 0 if {[active_members $dnsPool] > 0} { foreach dnsServer [active_members -list $dnsPool] { set dnsResolvIp [lindex [RESOLV::lookup @[lindex $dnsServer 0]$dnsRd -a $fqdn] 0] # verify result of dns lookup is a valid IP address if {[scan $dnsResolvIp {%d.%d.%d.%d} a b c d] == 4} { table set -subtable $dnsCache $fqdn $dnsResolvIp $ttl # clear any error flag and break out of loop set flgResolvError 0 set nodeIpRd $dnsResolvIp$nodeRd break } else { call ${partition}log local0."$dnsResolvIp is not a valid IP address, attempting to query other DNS server" set flgResolvError 1 } } } else { call ${partition}log local0."ERROR no DNS servers are availible" event disable return } #log error if query answers are invalid if {$flgResolvError} { call ${partition}log local0."ERROR unable to resolve $fqdn" unset -nocomplain dnsServer dnsResolvIp dnsResolvIp event disable return } } else { # retrieve cached IP from subtable (verification of resolved IP compelted before being commited to table) set dnsResolvIp [table lookup -notouch -subtable $dnsCache $fqdn] set nodeIpRd $dnsResolvIp$nodeRd log local0. "IP found in subtable: $dnsResolvIp with TTL: [table timeout -subtable $dnsCache -remaining $fqdn]" } # rewrite outbound host header and set node IP with RD and port HTTP::header replace Host $fqdn node $nodeIpRd $port log local0. "setting node to $nodeIpRd $port" } Tested this on version: 12.1Accept LTM traffic based on IP and variable datagroup
Hi, we are trying to do the following thing on our platform. I'll explain the flow first: Traffic enters the box using a single VS (EntryVS). An iRule triggers, looks at the host-header and directs the traffic to the correct VS for that site (ApplicationVS) A policy triggers there that rewrites the external received host-header to an internally used host-header. Traffic is forwarded to the backend pool members So far so good and is working fine. But we need access-lists per ApplicationVS based on Client-IP. I found an example to do that using in iRule, but we have hundreds of ApplicationVSses and many different access-lists and i don't want to create that many iRules. Many data-groups are acceptable but the iRule then must be aware of which data-group it has to use based on the host-header which, i think, is not available at the CLIENT_ACCEPTED stage. This is the iRule i have so far: when RULE_INIT { Drop unknown source IP addresses? 0 = no, 1 = yes set static::drop_unknown_sources 1 Log accepted requests? 0 = no, 1 = yes set static::log_accepted_requests 1 Log accepted requests? 0 = no, 1 = yes set static::log_dropped_requests 1 } This event is triggered when a client - BIG-IP TCP connection is established when CLIENT_ACCEPTED { Is client IP address defined in the admin_datagroup? if { [class match [IP::client_addr] starts_with DATAGROUP_NAME] }{ Log accepted admin request if {$static::log_accepted_requests==1}{ Log the client IP address:port -> destination IP address:port log local0. "request accepted from client: \ [IP::client_addr]:[TCP::client_port] -> [IP::local_addr]:[TCP::local_port]" } } else { Request didn't match the conditions for allowing the request if {$static::log_dropped_requests==1}{ Log the client IP address:port -> destination IP address:port log local0. "unknown request rejected from client: \ [IP::client_addr]:[TCP::client_port] -> [IP::local_addr]:[TCP::local_port]" } Drop request if the option is enabled. Could also send a reset using 'reject' if {$static::drop_unknown_sources==1}{ drop } } } How can i make the dategroup variable so that on each request, a host-header is verified and a matching datagroup is used like FQDN_access_DG? Or is there a better solution to this issue ? Regards, René
Accept LTM traffic based on IP and variable datagroup
Hi, we are trying to do the following thing on our platform. I'll explain the flow first: Traffic enters the box using a single VS (EntryVS). An iRule triggers, looks at the host-header and directs the traffic to the correct VS for that site (ApplicationVS) A policy triggers there that rewrites the external received host-header to an internally used host-header. Traffic is forwarded to the backend pool members So far so good and is working fine. But we need access-lists per ApplicationVS based on Client-IP. I found an example to do that using in iRule, but we have hundreds of ApplicationVSses and many different access-lists and i don't want to create that many iRules. Many data-groups are acceptable but the iRule then must be aware of which data-group it has to use based on the host-header which, i think, is not available at the CLIENT_ACCEPTED stage. This is the iRule i have so far: when RULE_INIT { Drop unknown source IP addresses? 0 = no, 1 = yes set static::drop_unknown_sources 1 Log accepted requests? 0 = no, 1 = yes set static::log_accepted_requests 1 Log accepted requests? 0 = no, 1 = yes set static::log_dropped_requests 1 } This event is triggered when a client - BIG-IP TCP connection is established when CLIENT_ACCEPTED { Is client IP address defined in the admin_datagroup? if { [class match [IP::client_addr] starts_with DATAGROUP_NAME] }{ Log accepted admin request if {$static::log_accepted_requests==1}{ Log the client IP address:port -> destination IP address:port log local0. "request accepted from client: \ [IP::client_addr]:[TCP::client_port] -> [IP::local_addr]:[TCP::local_port]" } } else { Request didn't match the conditions for allowing the request if {$static::log_dropped_requests==1}{ Log the client IP address:port -> destination IP address:port log local0. "unknown request rejected from client: \ [IP::client_addr]:[TCP::client_port] -> [IP::local_addr]:[TCP::local_port]" } Drop request if the option is enabled. Could also send a reset using 'reject' if {$static::drop_unknown_sources==1}{ drop } } } How can i make the dategroup variable so that on each request, a host-header is verified and a matching datagroup is used like FQDN_access_DG? Or is there a better solution to this issue ? Regards, René
iControl - Datagrouplist - address type with country
I need to detect if the incoming IP in the request is in the subnets listed and fetch the corresponding country. For this I created an address type list with subnets and the corresponding country (see sample 1) and you can see the list created (sample 2). Then I created an iRule to check If the incoming IP falls in the subnet and get the corresponding country [Sample 3]. All good till now. However, the list is getting longer and I will have to use iControl to automate the addition. How do I add/delete/retrieve the mask/country members (same format as I added them using "tmsh create ltm data-group") via iControl? As you can see the Sample 4, I am trying to retrieve the country value and all am getting is the address and network mask although the list has the country name (see Sample 5). How can I then add a new one in the same format using iControl. Sample 1 create ltm data-group internal OFAC_BLOCKED_CIDR_LIST records add { 5.100.192.0/19 { data "Belarus" } 41.66.0.0/18 {data "IvoryCoast"} 152.206.0.0/15 {data "Cuba"} 41.75.64.0/20 {data "Congo"} 91.186.192.0/19 { data "Iran" } 91.243.160.0/20 { data "Iran" } 5.42.192.0/19 {data "Iraq"} 41.57.80.0/20 {data "Liberia"} 115.84.64.0/18 {data "NorthKorea"} 41.67.0.0/18 {data "Sudan"} 5.0.0.0/16 {data "Syria"} 41.57.64.0/20 {data "Zimbabwe"}} type ip Sample 2 (/Common)(tmos) ltm data-group internal OFAC_BLOCKED_CIDR { 5.42.192.0/19 { data Iraq } 5.100.192.0/19 { data Belarus } 41.57.64.0/20 { data Zimbabwe } 41.57.80.0/20 { data Liberia } 41.66.0.0/18 { data IvoryCoast } 41.67.0.0/18 { data Sudan } 41.75.64.0/20 { data Congo } 91.186.192.0/19 { data Iran } 91.243.160.0/20 { data Iran } 115.84.64.0/18 { data NorthKorea } 152.206.0.0/15 { data Cuba } } type ip } Sample 3 set client_ip [getfield [IP::client_addr] "%" 1] set origin_country [class match -value ${fake_client_ip} equals OFAC_BLOCKED_CIDR] log local0. "Country - ${origin_country} IP address - ${client_ip}" if { ${origin_country} ne "" } { log local0. "Client Source IP: ${client_ip} from country ${origin_country} has been denied access to: [IP::local_addr]:[TCP::local_port]" HTTP::respond 200 content [ifile get noaccessimage] "Content-Type" "image/png" } else { log local0. "Client Source IP: ${client_ip} has been allowed access to: [IP::local_addr]:[TCP::local_port]" } Sample 4 $DataGroup = "OFAC_BLOCKED_CIDR" $ClassInfo = $ic.LocalLBClass.get_address_class($DataGroup); $name = $ClassInfo[0].name; Write-Host "Class: $name"; $members = $ClassInfo[0].members Write-host $members[0].value foreach($member in $members) { Write-Host "$member " $a = $member.address Write-Host " $a"; $b = $member.netmask } Sample 5 Class: OFAC_BLOCKED_CIDR 5.0.0.0 255.255.0.0 5.42.192.0 255.255.224.0 5.100.192.0 255.255.224.0 41.57.64.0 255.255.240.0 41.57.80.0 255.255.240.0 41.66.0.0 255.255.192.0 41.67.0.0 255.255.192.0 41.75.64.0 255.255.240.0
301 Redirect Using Data Group External File
This should be easy...and I see some posts on it, but I can't seem to get it to work. We were given a list of hundreds of URL's that need 301 redirects. The host and URI in most cases is entirely different and unique so I can't just wildcard a redirect for everything to a single destination. example: https://oldsite.com/pdf/111 --> http://newsite.com/doc/222 https://oldsite.com/jpg/222 --> http://newsite.com/bmp/111 etc. I have all of the old site to new site mappings and I read another post that said I should create an external data group as a string that has := as the separator, so I have a datagroup called "testuri" and it looks like this: "; := "; And I have an irule that says: when HTTP_REQUEST { if {[set code_url [class match -value -- [string tolower [HTTP::host][HTTP::uri]] equals testuri]] ne ""}{ HTTP::respond [getfield code_url " " 1] Location [getfield code_url " " 2] } } ...but it doesn't work. In the end all I need is an irule that looks for the inbound request and if it's in the data group then returns the new value that it should be mapped to. This seems simple, but it's turning not to be as simple as I had hoped.
Can anyone help me with single apology page using data groups?
Hi, I have a VS that serves several sites. Some times, one of these sites malfunction and I want to be able to manually put it in apology in a easy way. Im thinking a datagroup with a string and a value will be easy enough, like this: site1.com | no site2.com | no site3.com | yes site4.com | no site5.com | no Site 3 will be put in apology (just a 301 redirect to a static URL. Every site has a different URL.) My logic is: If HTTP::host equals class match dg_apology_single and class match apology_dg_single value equals "yes" then: 301 to http://apology.com/site3 else: do nothing. Im having some problems creating the logic. Any help would be appreciated!
iRules Optimization for MAC filtering with Data Groups (If/else)
Hello Everyone, I'm trying to figure out an optimized version of the following (currently working) iRules, in order to validate via Machine Info, the incoming MAC Address from different customers to a BIG-IP APM Access Policy. The iRules have been validated in version 12.1 and 13. Any advice/recommendation will be welcome. Here we have an example of the LTM Data Groups deployed: pedro.haoa@(f5chile)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm data-group one-line ltm data-group internal MACGRP_1001_external_chile { records { F4:15:63:11:22:33 { } F4:15:63:11:22:34 { } F4:15:63:11:22:35 { } } type string } . .(Output Omitted) . ltm data-group internal MACGRP_1370_external_chile { records { F4:15:63:44:55:66 { } F4:15:63:44:55:67 { } F4:15:63:44:55:68 { } } type string } . .(Output Omitted) . ltm data-group internal MACGRP_2001_external_bolivia { records { 00:23:E9:22:33:44 { } 00:23:E9:22:33:44 { } 00:23:E9:22:33:44 { } } type string } . .(Output Omitted) . ltm data-group internal MACGRP_2350_external_bolivia { records { 00:23:E9:55:66:77 { } 00:23:E9:55:66:78 { } 00:23:E9:55:66:79 { } } type string } . .(Output Omitted) . And here we have two iRules to validate more than 700 different Data Groups: BIG-IP APM Event when ACCESS_POLICY_AGENT_EVENT priority 410 { Access Policy Branch Filter if { [ACCESS::policy agent_id] eq "macgrp" } { Variables for LAN/WLAN Interfaces set mac0 [ACCESS::session data get "session.machine_info.last.net_adapter.list.\[0\].mac_address"] set mac1 [ACCESS::session data get "session.machine_info.last.net_adapter.list.\[1\].mac_address"] Variable to reduce data along the iRule due to the 64k limit. set s session.logon.custom.macgrp if/else statements to validate the MAC addresses contained within each data group if {[class match $mac0 eq MACGRP_1001_external_chile]||[class match $mac1 eq MACGRP_1001_external_chile]} {ACCESS::session data set $s 1} elseif {[class match $mac0 eq MACGRP_1002_external_chile]||[class match $mac1 eq MACGRP_1002_external_chile]} {ACCESS::session data set $s 1} elseif {[class match $mac0 eq MACGRP_1003_external_chile]||[class match $mac1 eq MACGRP_1003_external_chile]} {ACCESS::session data set $s 1} . .(Output Omitted) . elseif {[class match $mac0 eq MACGRP_1369_external_chile]||[class match $mac1 eq MACGRP_1369_external_chile]} {ACCESS::session data set $s 1} elseif {[class match $mac0 eq MACGRP_1370_external_chile]||[class match $mac1 eq MACGRP_1370_external_chile]} {ACCESS::session data set $s 1} } } Second iRule (Split mode): BIG-IP APM Event when ACCESS_POLICY_AGENT_EVENT priority 420 { Access Policy Branch Filter if { [ACCESS::policy agent_id] eq "macgrp" } { Variables for LAN/WLAN Interfaces set mac0 [ACCESS::session data get "session.machine_info.last.net_adapter.list.\[0\].mac_address"] set mac1 [ACCESS::session data get "session.machine_info.last.net_adapter.list.\[1\].mac_address"] Variable to reduce data along the iRule due to the 64k limit. set s session.logon.custom.macgrp if/else statements to validate the MAC addresses contained within each data group if {[class match $mac0 eq MACGRP_2001_external_bolivia]||[class match $mac1 eq MACGRP_2001_external_bolivia]} {ACCESS::session data set $s 1} elseif {[class match $mac0 eq MACGRP_2002_external_bolivia]||[class match $mac1 eq MACGRP_2002_external_bolivia]} {ACCESS::session data set $s 1} elseif {[class match $mac0 eq MACGRP_2003_external_bolivia]||[class match $mac1 eq MACGRP_2003_external_bolivia]} {ACCESS::session data set $s 1} . .(Output Omitted) . elseif {[class match $mac0 eq MACGRP_2349_external_bolivia]||[class match $mac1 eq MACGRP_2349_external_bolivia]} {ACCESS::session data set $s 1} elseif {[class match $mac0 eq MACGRP_2350_external_bolivia]||[class match $mac1 eq MACGRP_2350_external_bolivia]} {ACCESS::session data set $s 1} elseif {[class match $mac0 eq MACADM_CHECK]||[class match $mac1 eq MACADM_CHECK]} {ACCESS::session data set $s 1} } } Cheers!iRule using DataGroup to bypass APM Policy
Hi, We have an APM policy to collect the machine name, set it to our domain, and perform an AD lookup. We have to utilize Datagroups to pre-authorize a few users that do not fall into proper AD groups without moving the computer account. Our current iRule is below. I've created the datagroup and specified the machine names. I'm trying to create an irule that collects the machine name like normal but, bypasses the APM posture check if defined. If not defined, follow standard policy. Our APM policy --> Windows Info --> Irule Event --> AD Query --> Allow when RULE_INIT { set static::THIS_DOMAIN ".ourdomain.com" } end rule_init when ACCESS_POLICY_AGENT_EVENT { if { [ACCESS::policy agent_id] eq "get_computer_name" } { set computer [string tolower [ACCESS::session data get "session.windows_info_os.last.computer"]] foreach x [split $computer "|"] { if { $x ends_with $static::THIS_DOMAIN } { set machinename [lindex [split $x "."] 0] ACCESS::session data set session.custom.computer $machinename return } end if this_domain } end set computer } end ACCESS::policy agent_id } end access policy agent eventiRule(s) using Data Group based on host and uri
So we are trying to implement single VIP (one Virtual IP) for multiple clients (websites) and would like to achieve that using iRules that has Data Group assigned. I was able to get the iRule working which has DG based on host. However I need to be able to accept certain requests based on couple of URIs as well. One of the URIs should work on port 80 and other on 85. Below is the iRule I have which works fine based on host for 443. when HTTP_REQUEST { set pool [class match -value -- [HTTP::host] equals test_url] if {$pool ne ""} { pool $pool } } How do I append or add more iRules to manage my URI based traffic? I believe iRule for port 80 URI needs to be on 80_VIP. Also we need to have http to https redirect rule for everything other than above specified URI. URIs are as below -- /XYZ.svc* -- port 80 /example/attach/ABC.svc* -- port 85 URI (string) will be same for all clients but the pool (value) is different. So how do I get the request go to the right pool based on URI as well as host in the prefix? In my tests, all requests made for different client websites are going to the first pool in the data group.