[Workaround]: Prevent iRule Managers to accidentally enter invalid characters into a Data Group
We realized some time ago that the BIG-IP GUI does not handle UTF-8 encoded strings very well in Data Groups which result in ever growing data records for everytime the Data Group is saved. This is an example on how to reproduce the problem and to see it in action: Start by creating an empty new data group named TEST of the type string. Add a line with the key test1 and the value abcåäö123 (copy and paste if you don't have a nordic keyboard) Click finish Check with tmsh list ltm data-group internal TEST You will see that it doesn't handle the UTF-8 data and you will se two characters for each > 7-bit character. Now go to the GUI again and add a second line with the key test2 and the same value abcåäö123 Click update Once again check tmsh and you will see that not only has test2 been added but test1 has also been updated, now including even more wrong characters. This is a screenshot of how it will look: The result of this is that eventually you will receive the following error from the BIG-IP (even if you only update lines that are not containing 8-bit ascii characters). Workaround Luckily we require the iRule Managers to login to an APM portal (webtop) where we have published a Portal Access to the administrative GUI of the BIG-IP, and since we do that, we can inject some JavaScript the will check the data before it saved. iRule PREVENT_INVALID_DATA_IN_DATAGROUP-IRULE This iRule will inject some JavaScript code that will prevent these mistakes from happening when HTTP_REQUEST { set inject_code 0 if {[HTTP::path] ends_with "/tmui/tmui/skins/Default/scripts/skin.js"} { set inject_code 1 STREAM::disable HTTP::header remove "Accept-Encoding" } } when HTTP_RESPONSE { if {$inject_code == 1} { set s1 "this.form.submitError\ =\ formSubmitError;" set r1 "this.form.submitError\ =\ formSubmitError;\n" append r1 "if\ (document.location.href.indexOf('datagroup')\ >\ -1)\ {\n" append r1 "select\ =\ document.getElementById('class_string_item');\n" append r1 "if\ (select\ !=\ null)\ {\n" append r1 "console.log('We\ found\ the\ select');\n" append r1 "for\ (option\ of\ select.children)\ {\n" append r1 "key\ =\ option.value.substr(0,\ option.value.indexOf('\\\\x0a'));\n" append r1 "if\ (!\ /^\[\\x00-\\x7F\]*\$/.test(key))\ {\n" append r1 "alert('String\ \"'+key+'\"\ contains\ invalid\ characters!');\n" append r1 "return\ false;\n" append r1 "}\n" append r1 "if\ (!\ /^\[\\x00-\\x7F\]*\$/.test(option.value))\ {\n" append r1 "alert('Value\ of\ \"'+key+'\"\ contains\ invalid\ characters!');\n" append r1 "return\ false;\n" append r1 "}\n" append r1 "}\n" append r1 "}\n" append r1 "}\n" STREAM::expression "@$s1@$r1@" STREAM::enable } } Now if you add a streaming profile to your APM Virtual Server and add this iRule the user will be presented with a popup informing them about the invalid data and also prevent them from submitting it. Hope this can be of help until F5 fixes this in the BIG-IP
Dynamic FQDN Node DNS Resolution based on URI with Route Domains and Caching iRule
Problem this snippet solves: Following on from the code share Ephemeral Node FQDN Resolution with Route Domains - DNS Caching iRule that I posted, I have made a few modifications based on further development and comments/questions in the original share. On an incoming HTTP request, this iRule will dynamically query a DNS server (out of the appropriate route domain) defined in a pool to determine the IP address of an FQDN ephemeral node depending on the requesting URI. The IP address will be cached in a subtable so to prevent querying on every HTTP request It will also append the route domain to the node and replace the HTTP Host header. Features of this iRule * Dynamically resolves FQDN node from requesting URI * Iterates through other DNS servers in a pool if response is invalid or no response * Caches response in the session table using custom TTL * Uses cached result if present (prevents DNS lookup on each HTTP_REQUEST event) * Appends route domain to the resolved node * Replaces host header for outgoing request How to use this snippet: Add a DNS pool with appropriate health monitors, in this example the pool is called dns_pool Add the lookup datagroup, dns_lookup_dg . This defines the parameters of the DNS query. The values in the datagroup will built up an array using the key in capitals to define the array object e.g. $myArray(FQDN) Modify the values as required: FQDN: the FQDN of the node to load balance to. DNS-RD: the outbound route domain to reach the DNS servers NODE-RD: the outbound route domain to reach the node TTL: TTL value for the DNS cache in seconds ltm data-group internal dns_lookup_dg { records { /app1 { data "FQDN app1.my-domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 8443" } /app2 { data "FQDN app2.my-other-domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 8080" } default { data "FQDN default.domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 443" } } type string } Code : ltm data-group internal dns_lookup_dg { records { /app1 { data "FQDN app1.my-domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 8443" } /app2 { data "FQDN app2.my-other-domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 8080" } default { data "FQDN default.domain.com|DNS-RD %10|NODE-RD %20|TTL 300|PORT 443" } } type string } when CLIENT_ACCEPTED { set dnsPool "dns_pool" set dnsCache "dns_cache" set dnsDG "dns_lookup_dg" } when HTTP_REQUEST { # set datagroup values in an array if {[class match -value [HTTP::uri] "starts_with" $dnsDG]} { set dnsValues [split [string trim [class match -value [HTTP::uri] "starts_with" $dnsDG]] "| "] } else { # set to default if URI is not defined in DG set dnsValues [split [string trim [class match -value "default" "equals" $dnsDG]] "| "] } if {([info exists dnsValues]) && ($dnsValues ne "")} { if {[catch { array set dnsArray $dnsValues set fqdn $dnsArray(FQDN) set dnsRd $dnsArray(DNS-RD) set nodeRd $dnsArray(NODE-RD) set ttl $dnsArray(TTL) set port $dnsArray(PORT) } catchErr ]} { log local0. "failed to set DNS variables - error: $catchErr" event disable return } } # check if fqdn has been previously resolved and cached in the session table if {[table lookup -notouch -subtable $dnsCache $fqdn] eq ""} { log local0. "IP is not cached in the subtable - attempting to resolve IP using DNS" # initialise loop vars set flgResolvError 0 if {[active_members $dnsPool] > 0} { foreach dnsServer [active_members -list $dnsPool] { set dnsResolvIp [lindex [RESOLV::lookup @[lindex $dnsServer 0]$dnsRd -a $fqdn] 0] # verify result of dns lookup is a valid IP address if {[scan $dnsResolvIp {%d.%d.%d.%d} a b c d] == 4} { table set -subtable $dnsCache $fqdn $dnsResolvIp $ttl # clear any error flag and break out of loop set flgResolvError 0 set nodeIpRd $dnsResolvIp$nodeRd break } else { call ${partition}log local0."$dnsResolvIp is not a valid IP address, attempting to query other DNS server" set flgResolvError 1 } } } else { call ${partition}log local0."ERROR no DNS servers are availible" event disable return } #log error if query answers are invalid if {$flgResolvError} { call ${partition}log local0."ERROR unable to resolve $fqdn" unset -nocomplain dnsServer dnsResolvIp dnsResolvIp event disable return } } else { # retrieve cached IP from subtable (verification of resolved IP compelted before being commited to table) set dnsResolvIp [table lookup -notouch -subtable $dnsCache $fqdn] set nodeIpRd $dnsResolvIp$nodeRd log local0. "IP found in subtable: $dnsResolvIp with TTL: [table timeout -subtable $dnsCache -remaining $fqdn]" } # rewrite outbound host header and set node IP with RD and port HTTP::header replace Host $fqdn node $nodeIpRd $port log local0. "setting node to $nodeIpRd $port" } Tested this on version: 12.1Print string found in Data Group to Log
I have an iRule that is looking in the HTTP POST request method data payload for a string that is defined in a data-group. I would like to print to the log whichever string from the referenced data-group is found. # See https://devcentral.f5.com/s/question/0D51T00006i7hpJSAQ/irule-to-block-requests-with-specific-word # #ltm data-group internal restricted_dg { #records { #restricted {} #} #type string #} when HTTP_REQUEST_DATA { set payload [HTTP::payload] if {[class match [string tolower $payload] contains "restricted_dg"]} { # set variable named restricted_text to the string found in $payload # that matches something in data-group restricted_dg log local0. "Rejecting restricted content $restricted_text" reject } }
