ASM/WAF Management Automation - TMOS
This post is to go over some of my thoughts on ASM/WAF management, and some custom solutions I've made to make it easier and more accurate. This will be highly technical and will apply to most use-cases. Written based on TMOS v15 software, but will be mostly applicable to all recent TMOS versions. Automatic Learning Informational Automatic Learning will automatically accept most Learning Suggestions that hit 100% confidence Fully Automatic Learning will automatically accept all Learning Suggestions that hit 100% confidence If your application open to the public, it will likely get scanned a lot. Some of the Learning Suggestions will be to loosen your policies based on scanner traffic it sees, like this one that would've accepted a SQL injection attack suggestion from traffic that's clearly from a Veracode scanner Tips Reduce scanner traffic that hits ASM policy: Apply an iRule to your VIPs that have ASM policies that drops external IP-based requests. A lot of scanner traffic hits your VIPs by IP address instead of by hostname, and iRules process before ASM policies. So dropping IP-based traffic via iRule will drastically reduce the amount of bad ASM Learning Suggestions you get I use the attached iRule "irule_all_asm_VIPs" on all ASM VIPs. It has a line to increment the "triggered" field in a Statistics profile named "asm_irule_triggered," so you'll have to create the Statistics profile with the "triggered" field and attach it to the VIP before it will let you attach the iRule to the VIP. This makes sure you don't lose track of how many bad requests are hitting the device If ASM VIP has an existing iRule, irule_all_asm_VIPs must be attached first in order. See below for an automated way to attach this iRule to all ASM VIPs. Ongoing Maintenance Tasks: Enforce Ready Signatures ASM policies have a configured staging period (AKA Enforcement Readiness Period) for all new and modified Attack Signatures. Once this staging period is complete, they must be enforced manually Under Security > Application Security > Policy Building > Traffic Learning, there's an Enforcement Readiness Summary section where you can see ASM signatures that are Ready To Be Enforced. You can also Enforce Ready Entities on this page See here for a script I made that can be pushed from the Big-IQ to Enforce Ready Entities on all ASM Policies: https://community.f5.com/discussions/technicalforum/automate-asm-ready-to-be-enforced-attack-signatures/245055 If using Automatic Learning (not Fully Automatic Learning), periodically review Learning Suggestions at 100% confidence Manual Learning Informational Manual Learning doesn’t accept any Learning Suggestions automatically Without the automation explained in this post or a full WAF team, it can be hard to manage many policies manually Tips Reduce scanner traffic that hits ASM policy. <see above> Ongoing Maintenance Enforce Ready Signatures <see above> Periodically Review Learning Suggestions at 100% confidence <see below for automation> Big-IQ Script Automation Enforce Ready Entities <see above> https://community.f5.com/discussions/technicalforum/automate-asm-ready-to-be-enforced-attack-signatures/245055 Review Learning Suggestions Periodic reviews of Learning Suggestions can be tedious because there isn't a main overview page that tells you how many Learning Suggestions each policy has at 100% confidence. So without automation, you have to click through each policy one-by-one to see which policies need to be reviewed. I made the attached "Email Ready Learning Suggestions" script, that builds an HTML table line-by-line showing how many Learning Suggestions each policy has at 100% confidence, then uses the sendmail command to email it. There are 2 initial one-time setup commands at the top of the script you'll have to check first to make sure your F5 can send emails, and you'll have to replace email@example.com with your own email address in at the bottom. If it doesn't work after that, it's likely that your F5 SMTP traffic isn't allowed through a firewall Attach iRule and Statistics Profile to all ASM VIPs I made the attached "Attach iRule and Statistics Profile to all ASM VIPs" script that can be pushed from the Big-IQ to make a list of all VIPs with ASM policies, then attach the iRule if the VIP doesn't already have it. It doesn't remove existing iRules, and it puts this one first in the list if there are existing iRules Obviously for this to work, your F5 needs to already have the "irule_all_asm_VIPs" iRule and "asm_irule_triggered" Statistics profile created. The Statistics profile needs a field called "triggered" Cron Job Automation Reset Log Statistics Profile Count, and Reset I use these 3 commands in a daily cron job that runs at midnight to send a syslog entry with the number of times an IP-Based request got dropped by the iRule, then reset the counter EXECUTIONS=$(tmsh show ltm profile statistics asm_irule_triggered | grep triggered | tail -n1 | sed s/"triggered"//) logger -p local0.notice "Daily number of IP-based requests that hit ASM iRule: $EXECUTIONS" tmsh reset-stats ltm profile statistics asm_irule_triggered69Views0likes0CommentsGTM return LDNS IP to client
Problem this snippet solves: We do a lot of our load balancing based on topology rules, so it's often very useful to know where the DNS request is actually coming from rather than just the client's IP and the DNS servers they have configured. Especially if they're behind an ADSL router doing NAT or some other similar set up. This rule simply returns the IP address of the LDNS that eventually made the query to the GTM device in the response to a lookup for the WideIP using the rule, as well as logging the response and perceived location. Code : rule "DNS_debug" partition "Common" { when DNS_REQUEST { host [IP::client_addr] log local0.err "Debug address : [IP::client_addr] [whereis [IP::client_addr]]" } }817Views1like2CommentsCreate DNS monitor without external dig or nslookup
Problem this snippet solves: NOTE: This was written in 10.x and early 11.x days. Since then, a native DNS monitor has been added to the product. This example will create a udp monitor with proper send and receive strings to query a DNS server. Much of the heavy lifting was pulled from an iApp template that ships with v11. I have tested this out on as far back as 10.2.1 base. Installation One method to get this into your config would be: In the code block below, click the icon in the top right corner '<>' view source. Ctrl-A, Ctrl-C to select and copy all. Then in tmsh, type 'edit /cli script dns.monitor' If your editor is configured to use vi (the default setting) Type :set paste, then c, then Shift + G. This will clear out the default skeleton. You will now be in insert mode. Press Shift + insert. This will paste in the script. Now save the script by pressing Esc, then typing :wq < enter > and follow the prompts. How to use this snippet: Example root@bigip(Active)(/Common)(tmos)# run cli script dns.monitor xxx Requires 5 arguments: := Name you want to give the monitor := A, NS, PTR, SOA, CNAME := What you are asking the DNS server (send string) := Why you expect for a "healthy" response (recv string) := The frequency you want the monitor to fire e.g. dns.monitor my_dns A www.example.com 192.0.2.55 5 root@bigip(Active)(/Common)(tmos)# run cli script dns.monitor my_dns A www.example.com 192.0.2.55 5 root@bigip(Active)(/Common)(tmos)# list ltm monitor udp my_dns ltm monitor udp my_dns { debug no defaults-from udp destination *:* interval 5 recv \\x00\\x01.*\\x04\\xc0\\x00\\x02\\x37 send \\x96\\x87\\x01\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x03www\\x07example\\x03com\\x00\\x00\\x01\\x00\\x01 time-until-up 0 timeout 16 } root@bigip(Active)(/Common)(tmos)# Code : create script dns.monitor { proc script::init {} { } proc script::run {} { if {$tmsh::argc == 6} { # create monitor set monitor_name [lindex $tmsh::argv 1] set monitor_type [string toupper [ lindex $tmsh::argv 2 ]] switch $monitor_type { A { set monitor_send [generate_monitor_send_string_a_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_a_record [lindex $tmsh::argv 4]] } NS { set monitor_send [generate_monitor_send_string_ns_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_ns_record [lindex $tmsh::argv 4] [lindex $tmsh::argv 3]] } PTR { set monitor_send [generate_monitor_send_string_ptr_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_ptr_record [lindex $tmsh::argv 4]] } SOA { set monitor_send [generate_monitor_send_string_soa_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_soa_record [lindex $tmsh::argv 4] [lindex $tmsh::argv 3]] } CNAME { set monitor_send [generate_monitor_send_string_cname_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_cname_record [lindex $tmsh::argv 4] [lindex $tmsh::argv 3]] } default { usage } } set monitor_interval [lindex $tmsh::argv 5] set monitor_timeout [ expr { [ expr {$monitor_interval * 3} ] + 1 } ] # doctor the send and receive strings so that they go through as intended set monitor_send [doctor_hex_string $monitor_send] set monitor_recv [doctor_hex_string $monitor_recv] tmsh_create "ltm monitor udp $monitor_name defaults-from udp interval $monitor_interval timeout $monitor_timeout send \"$monitor_send\" recv \"$monitor_recv\"" } else { usage } } proc script::help {} { tmsh::add_help "Create a DNS monitor \n\ " } proc script::tabc {} { } proc tmsh_create { arguments } { regsub -all {\[} $arguments "\\\[" arguments regsub -all {\]} $arguments "\\\]" arguments tmsh::create "$arguments" } proc number_to_hex_string { number } { if { $number < 0 } { set number [ expr {0 - $number} ] } set hex_string [format "\\x%02x" $number] return $hex_string } proc hostname_or_ip_address_to_dns_name { hostname } { set fields [split $hostname .] set dns_name "" foreach field $fields { set len [string length $field] set str_len [number_to_hex_string $len] append dns_name $str_len append dns_name $field } append dns_name "\\x00" return $dns_name } proc hostname_or_ip_address_to_reverse_dns_name { hostname } { set complete_hostname [format "arpa.in-addr.%s" $hostname] set fields [split $complete_hostname .] set dns_name "" foreach field $fields { # if [string is integer $field] { set len [string length $field] set str_len [number_to_hex_string $len] set dns_name [format "%s%s%s" $str_len $field $dns_name] # } } append dns_name "\\x00" return $dns_name } proc get_id { } { set value1 [expr { int(256 * rand()) }] set value2 [expr { int(256 * rand()) }] set id [number_to_hex_string $value1] append id [number_to_hex_string $value2] return $id } proc generate_dns_packet_payload { question } { set payload [get_id] append payload "\\x01\\x00" append payload "\\x00\\x01" append payload "\\x00\\x00" append payload "\\x00\\x00" append payload "\\x00\\x00" append payload $question return $payload } proc ip_addr_to_a_resp_addr { address } { set fields [split $address .] set num_fields 0 set field_encoding "" foreach field $fields { if {[string is integer $field]} { set num_fields [expr { $num_fields + 1 }] set part [number_to_hex_string $field] append field_encoding $part } } set dns_name [number_to_hex_string $num_fields] append dns_name $field_encoding return $dns_name } proc generate_send_payload { input_string record_type_code reverse } { if { $reverse == 0 } { set dns_name [hostname_or_ip_address_to_dns_name $input_string] } else { set dns_name [hostname_or_ip_address_to_reverse_dns_name $input_string] } set internet_address_qclass "\\x00\\x01" set question $dns_name append question $record_type_code append question $internet_address_qclass set send [generate_dns_packet_payload $question] return $send } proc generate_monitor_send_string_a_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x01" 0] return $send } proc generate_monitor_send_string_ns_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x02" 0] return $send } proc generate_monitor_send_string_ptr_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x0c" 1] return $send } proc generate_monitor_send_string_soa_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x06" 0] return $send } proc generate_monitor_send_string_cname_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x05" 0] return $send } proc generate_monitor_recv_string_a_record { input_string } { set a_resp_addr [ip_addr_to_a_resp_addr $input_string] set recv [format "\\x00\\x01.*%s" $a_resp_addr] return $recv } proc generate_monitor_recv_string_ns_record { input_string domain_name } { regsub -all {\.} $domain_name \. domain_name set re [format "^(.*)(\.%s)$" $domain_name] set found [regexp $re $input_string matched new_hostname rest] if { $found != 0 } { set recv_temp [hostname_or_ip_address_to_dns_name $new_hostname] set re {^(.*\\)(x00)$} set found [regexp -nocase $re $recv_temp matched recv rest] if { $found != 0 } { set recv [format "\\x00\\x02.*%sxc0" $recv] } else { set recv [format "\\x00\\x02.*%s" $recv_temp] } } else { set recv [hostname_or_ip_address_to_dns_name $input_string] set recv [format "\\x00\\x02.*%s" $recv] } return $recv } proc generate_monitor_recv_string_ptr_record { input_string } { set dns_name [hostname_or_ip_address_to_dns_name $input_string] set recv [format "\\x00\\x0c.*%s" $dns_name] return $recv } proc generate_monitor_recv_string_soa_record { input_string domain_name } { regsub -all {\.} $domain_name \. domain_name set re [format "^(.*)(\.%s)$" $domain_name] set found [regexp -nocase $re $input_string matched new_hostname rest] if { $found != 0 } { set recv_temp [hostname_or_ip_address_to_dns_name $new_hostname] set re {^(.*)(\\x00)$} set found [regexp -nocase $re $recv_temp matched recv rest] if { $found != 0 } { set recv [format "\\x00\\x06.*%s(\\xc0|\\x05)" $recv] } else { set recv [format "\\x00\\x06.*%s" $recv_temp] } } else { set recv [hostname_or_ip_address_to_dns_name $input_string] set recv [format "\\x00\\x06.*%s" $recv] } return $recv } proc generate_monitor_recv_string_cname_record { input_string hostname } { regsub -all {\.} $hostname \. hostname #puts "hostname = $hostname" set re {^([^\.]+)\.(.+)$} #puts "re = $re" set found [regexp -nocase $re $hostname matched first domain_name] #puts "found = $found" if { $found == 0 } { set domain_name $hostname } regsub -all {\.} $domain_name \. domain_name #puts "domain_name = $domain_name" #puts "input_string = $input_string" set re [format "^(.*)(\.%s)$" $domain_name] #puts "re = $re" set found [regexp -nocase $re $input_string matched new_hostname rest] if { $found != 0 } { set recv_temp [hostname_or_ip_address_to_dns_name $new_hostname] set re {^(.*\\)(x00)$} set found [regexp -nocase $re $recv_temp matched recv rest] if { $found != 0 } { set re {^(.*)(\\x00)$} set found [regexp -nocase $re $recv_temp matched recv rest] set recv [format "\\x00\\x05.*%s(\\xc0|\\x05)" $recv] } else { set recv [format "\\x00\\x05.*%s" $recv_temp] } } else { set recv [hostname_or_ip_address_to_dns_name $input_string] set recv [format "\\x00\\x05.*%s" $recv] } return $recv } proc doctor_hex_string { input } { set pieces [split $input \\ ] set output "" set first_time 1 foreach piece $pieces { if { $first_time == 1 } { set first_time 0 } else { append output {\\} } append output $piece } return $output } proc usage { } { puts "Requires 5 arguments: \n\ := Name you want to give the monitor \n\ := A, NS, PTR, SOA, CNAME \n\ := What you are asking the DNS server (send string) \n\ := Why you expect for a \"healthy\" response (recv string) \n\ := The frequency you want the monitor to fire \n\n\ e.g. [lindex $tmsh::argv 0] A www.example.com 192.0.2.55 5" exit } }522Views0likes0CommentsGenerate private key w/ CSR via iControl REST
Problem this snippet solves: Generate a private key w/ CSR How to use this snippet: To create a private key with a CSR via iControl REST: POST URL:https://10.1.1.165/mgmt/tm/sys/crypto/key Use the data below as your payload. For the name field, it must end in .key or you will get a false 404! Code : { "name":"www.testing.com.key", "commonName":"www.testing.com", "keySize":"4096", "keyType":"rsa-private", "options":[{"gen-csr":"www.testing.com"}], "organization":"Let It Snow Corp.", "ou":"Ice Engineering", "city":"Calhoun", "state":"AZ", "admin-email-address":"jerry@letit.snow", "email-address":"beth@letit.snow", "subject-alternative-name":"DNS:www.testing.com", "challenge-password":"myP4ssword" } Tested this on version: 13.01.9KViews3likes11CommentsFQDN nodes in non-default route domains
Problem this snippet solves: Currently there is no support for FQDN nodes in non-default route domains as per Article. With recent increase in cloud deployment, most of the time there is a requirement from F5 to load balance to servers or ELB in the cloud which has FQDN names as those are having dyanamic IP addresses. If you are using route domains in your BIGIP environment then this becomes a challenge. Below iRule script can be used in those scenerios to use F5 to send traffic to FQDN nodes in non-default route domains. How to use this snippet: If FQDN needs to be resolved by your internal DNS, create performance layer 4 VIP (dns_53) load balancing your DNS servers. DNS server can be used directly in the iRule itself, but it would be better to use a VIP to have redundancy. Create iRule and apply to the VIP Code : when CLIENT_ACCEPTED { set host [RESOLV::lookup @dns_53 "server.example.com"] set ip [getfield $host " " 1] node [lindex $ip 0]%<Rd> 443 } Tested this on version: 14.11.3KViews1like3CommentsBIG-IP Report
Problem this snippet solves: Overview This is a script which will generate a report of the BIG-IP LTM configuration on all your load balancers making it easy to find information and get a comprehensive overview of virtual servers and pools connected to them. This information is used to relay information to NOC and developers to give them insight in where things are located and to be able to plan patching and deploys. I also use it myself as a quick way get information or gather data used as a foundation for RFC's, ie get a list of all external virtual servers without compression profiles. The script has been running on 13 pairs of load balancers, indexing over 1200 virtual servers for several years now and the report is widely used across the company and by many companies and governments across the world. It's easy to setup and use and only requires auditor (read-only) permissions on your devices. Demo/Preview Interactive demo http://loadbalancing.se/bigipreportdemo/ Screen shots The main report: The device overview: Certificate details: How to use this snippet: Installation instructions BigipReport REST This is the only branch we're updating since middle of 2020 and it supports 12.x and upwards (maybe even 11.6). Downloads (two latest versions): https://loadbalancing.se/downloads/bigipreport-v5.7.11.zip https://loadbalancing.se/downloads/bigipreport-v5.7.10.zip Documentation, installation instructions and troubleshooting:https://loadbalancing.se/bigipreport-rest/ Docker support https://loadbalancing.se/2021/01/05/running-bigipreport-on-docker/ Kubernetes support https://loadbalancing.se/2021/04/16/bigipreport-on-kubernetes/ BIG-IP Report (Legacy) Older version of the report that only runs on Windows and is depending on a Powershell plugin originally written by Joe Pruitt (F5) BIG-IP Report (only download this if you have v10 devices): https://loadbalancing.se/downloads/bigipreport-5.4.0-beta.zip iControl Snapin https://loadbalancing.se/downloads/f5-icontrol.zip Documentation and Installation Instructions https://loadbalancing.se/bigip-report/ Upgrade instructions Protect the report using APM and active directory Written by DevCentral member Shann_P: https://loadbalancing.se/2018/04/08/protecting-bigip-report-behind-an-apm-by-shannon-poole/ Got issues/problems/feedback? Still have issues? Drop a comment below. We usually reply quite fast. Any bugs found, issues detected or ideas contributed makes the report better for everyone, so it's always appreciated. --- Join us on Discord: https://discord.gg/7JJvPMYahA Code : BigIP Report Tested this on version: 12, 13, 14, 15, 1613KViews20likes94CommentsBIG-IP Wide-IP to F5XC DNSLB converter
This is a conceptual sample script that converts BIG-IP Wide-IP records to F5XC DNSLB records. This bash script can be run using a cron job to check for configuration changes and synchronize them to F5XC. We used the F5XC API to post and update the configuration. You need to get an APIToken from your F5XC tenant and change the value on the POST commands on the script below. Note: Since this is not a full-blown converter script, it is limited to handling only a single Wide-IP pool member. You need to configure a GTM pool to include the IP addresses that need to be load balanced. Check the main article for more details. #!/bin/bash # Get list of wide IPs wideip_output=$(tmsh list gtm wideip all-properties one-line) # Get list of Pool pool_output=$(tmsh list gtm pool a one-line all-properties) # Declare associative arrays declare -A wideip_list declare -A current_wideip_info declare -A zone_array declare -A subdomain_info declare -A a_record_per_zone declare -A pool_list declare -A membersip_array # Unset variables function unset_arrays { unset current_wideip_info name subdomain domain type aliases description status failure_rcode last_resort_pool load_balancing_decision_log metadata minimal_response partition persist_cidr_ipv4 persist_cidr_ipv6 persistence pool_lb_mode pools pool_cname topology_edns0 ttl_persistence poolnames poolnames_array zone_array subdomain_info a_record_per_zone dnslb_name pool_list membersip_array } # Print wide IP details function print_wideip { for wideip in "${!wideip_list[@]}"; do echo "Wide IP: $wideip, Details: ${wideip_list[$wideip]}" done } # Create Zone function create_zone { curl -X POST -H "Authorization: APIToken XXXXX" -H "Accept: application/json" -H "Access-Control-Allow-Origin: *" -H "x-volterra-apigw-tenant: cag-waap2023" -H "Content-Type: application/json" -d "{\"metadata\":{\"name\":\"$zone\",\"namespace\":\"system\"},\"spec\":{\"primary\":{\"allow_http_lb_managed_records\":true},\"default_soa_parameters\":{},\"dnssec_mode\":{},\"rr_set_group\":[],\"soa_parameters\":{\"refresh\":3600,\"expire\":0,\"retry\":60,\"negative_ttl\":0,\"ttl\":0}}}" https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_zones } # Create DNSLB function create_dnslb { curl -X POST -H "Authorization: APIToken XXXXX" -H "Accept: application/json" -H "Access-Control-Allow-Origin: *" -H "x-volterra-apigw-tenant: cag-waap2023" -H "Content-Type: application/json" -d "{\"metadata\":{\"name\":\"$dnslbname\",\"namespace\":\"system\",\"labels\":{},\"annotations\":{},\"disable\":false},\"spec\":{\"record_type\":\"A\",\"rule_list\":{\"rules\":[{\"geo_location_set\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"geo-1\",\"kind\":\"geo_location_set\"},\"pool\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"$xcdnslbpoolname\",\"kind\":\"dns_lb_pool\"},\"score\":100}]},\"response_cache\":{\"disable\":{}}}}" https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_load_balancers } # Loop through each line of output while IFS= read -r line; do pool_name=$(awk '{print $4}' <<< "$line") dnslbpool_name=$(echo "$pool_name" | sed 's/[^a-zA-Z0-9]/-/g; s/.*/\L&/') pool_type=$(awk '{print $3}' <<< "$line") lbmode=$(grep -o 'load-balancing-mode [^ ]*' <<< "$line" | awk '{print $2}') # Convert load_balancing_mode to lowercase if it is "ROUND_ROBIN" if [[ "$lbmode" == "round-robin" ]]; then lbmode="ROUND_ROBIN" elif [[ "$lbmode" == "static-persistence" ]]; then lbmode="STATIC_PERSIST" elif [[ "$lbmode" == "global-availability" ]]; then lbmode="PRIORITY" elif [[ "$lbmode" == "ratio" ]]; then lbmode="RATIO_MEMBER" fi # Extract members block using awk #members=$(awk -F 'members {| }' '{print $2}' <<< "$line") members=$(echo "$line" | grep -o -P '(?<=members \{ ).*?(?=\} \})') membernames=$(echo "$members" | grep -oP '\S+(?=\s*{)') # Temporary array to hold member IP addresses declare -a temp_members_array temp_members_array=($(awk -F ':' '{print $2}' <<< "$membernames")) monitor=$(awk -F 'monitor ' '{print $2}' <<< "$line" | awk '{print $1}') ttl=$(awk '{print $2}' <<< "$(grep -o 'ttl [^ ]*' <<< "$line")") # Assign values to the associative array membersip_array["$dnslbpool_name"]="${temp_members_array[@]}" # Store extracted values in the array pool_list["$dnslbpool_name"]="pool_type: $pool_type, lbmode: $lbmode, monitor: $monitor, members: ${membersip_array["$dnslbpool_name"]}, ttl: $ttl" done <<< "$pool_output" # Loop through each pool in the pool_list for dnslbpool_name in "${!pool_list[@]}"; do # Extract only the TTL value from the string ttl=$(awk -F 'ttl: ' '{print $2}' <<< "${pool_list[$dnslbpool_name]}") lbmode=$(awk -F 'lbmode: ' '{print $2}' <<< "${pool_list[$dnslbpool_name]}" | awk -F ',' '{print $1}') members=$(awk -F 'members: ' '{print $2}' <<< "${pool_list[$dnslbpool_name]}" | awk -F ',' '{print $1}') pool_type=$(awk -F 'pool_type: ' '{print $2}' <<< "${pool_list[$dnslbpool_name]}" | awk -F ',' '{print $1}') # Check if pool_type is "a" if [[ "$pool_type" == "a" ]]; then # Initialize an empty string to store the JSON strings members_string="" # Loop through each record in the current zone for ip in ${membersip_array["$dnslbpool_name"]}; do # Create JSON string for each member and append to the existing string members_string+="{\"ip_endpoint\":\"$ip\",\"ratio\":10,\"priority\":1}," done # Remove the trailing comma from the JSON string members_string="${members_string%,}" # Create DNSLB Pools curl -X POST \ -H "Authorization: APIToken Rs0aGJm/lda/JmbE00c9lFXWw4I=" \ -H "Accept: application/json" \ -H "Access-Control-Allow-Origin: *" \ -H "x-volterra-apigw-tenant: cag-waap2023" \ -H "Content-Type: application/json" \ -d "{\"metadata\":{\"name\":\"$dnslbpool_name\",\"namespace\":\"system\"},\"spec\":{\"a_pool\":{\"members\":[$members_string],\"disable_health_check\":null,\"max_answers\":1},\"ttl\":\"$ttl\",\"load_balancing_mode\":\"$lbmode\"}}" \ "https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_lb_pools" fi done # Unset variables to free up memory unset pool_list membersip_array # Loop through each line of output while IFS= read -r line; do # Extracting specific details using awk and sed based on the current line name=$(echo "$line" | awk '{print $4}') dnslb_name=$(echo "$name" | sed 's/\./-/g') subdomain=$(echo "$name" | cut -d'.' -f1) domain=$(echo "$name" | sed 's/^[^.]*\.//') type=$(echo "$line" | awk '{print $3}') aliases=$(echo "$line" | grep -o 'aliases [^}]*' | awk '{print $2}') description=$(echo "$line" | grep -o 'description [^ ]*' | sed 's/description //') status=$(echo "$line" | awk '{print $12}') failure_rcode=$(echo "$line" | grep -o 'failure-rcode [^ ]*' | sed 's/failure-rcode //') last_resort_pool=$(echo "$line" | grep -o 'last-resort-pool [^ ]*' | sed 's/last-resort-pool //') load_balancing_decision_log=$(echo "$line" | grep -o 'load-balancing-decision-log-verbosity [^ ]*' | sed 's/load-balancing-decision-log-verbosity //') metadata=$(echo "$line" | grep -o 'metadata [^ ]*' | sed 's/metadata //') minimal_response=$(echo "$line" | grep -o 'minimal-response [^ ]*' | sed 's/minimal-response //') partition=$(echo "$line" | grep -o 'partition [^ ]*' | sed 's/partition //') persist_cidr_ipv4=$(echo "$line" | grep -o 'persist-cidr-ipv4 [^ ]*' | sed 's/persist-cidr-ipv4 //') persist_cidr_ipv6=$(echo "$line" | grep -o 'persist-cidr-ipv6 [^ ]*' | sed 's/persist-cidr-ipv6 //') persistence=$(echo "$line" | grep -o ' persistence [^ ]*' | sed 's/persistence //') pool_lb_mode=$(echo "$line" | grep -o 'pool-lb-mode [^ ]*' | sed 's/pool-lb-mode //') pools=$(echo "$line" | grep -o -P '(?<=pools \{ ).*?(?=\} \})') pool_cname=$(echo "$line" | grep -o 'pools-cname [^ ]*' | sed 's/pools-cname //') topology_edns0=$(echo "$line" | grep -o 'topology-prefer-edns0-client-subnet [^ ]*' | sed 's/topology-prefer-edns0-client-subnet //') ttl_persistence=$(echo "$line" | grep -o 'ttl-persistence [^ ]*' | sed 's/ttl-persistence //') # Use grep to find strings before "{" poolnames=$(echo "$pools" | grep -oP '\S+(?=\s*{)' | sed 's/[^a-zA-Z0-9]/-/g; s/.*/\L&/') # Convert matches to an array readarray -t poolnames_array <<< "$poolnames" # Store extracted values in the associative array current_wideip_info=([Type]="$type" [Subdomain]="$subdomain" [Domain]="$domain" [Status]="$status" [DNSLB]="$dnslb_name" [Pools]="${poolnames_array[@]}" [Pool_LB_Mode]="$pool_lb_mode") # Assign wideip_info to wideip_list wideip_list["$name"]="${current_wideip_info[@]}" # Add subdomains to zone_array if [ -n "${zone_array[$domain]}" ]; then zone_array["$domain"]="${zone_array[$domain]},$subdomain" else zone_array["$domain"]=$subdomain fi # Store subdomain information in subdomain_info array subdomain_info["$subdomain"]="${current_wideip_info[@]}" # Store subdomain type "a" and add it to the array for that zone if [ "$type" == "a" ]; then a_record_per_zone[$domain]="${a_record_per_zone[$domain]}${a_record_per_zone[$domain]:+,}$subdomain" fi done <<< "$wideip_output" for zone in "${!zone_array[@]}"; do create_zone done # Loop through each domain in a_record_per_zone and echo its A record subdomains for domain in "${!a_record_per_zone[@]}"; do echo "Domain: $domain" echo "A Record Subdomains: ${a_record_per_zone[$domain]}" echo "--------------------------" # Initialize an empty string to store the JSON strings a_records_string="" # Loop through each record in the current zone for record in ${a_record_per_zone[$domain]//,/ }; do # Create JSON string for each A record and append to the existing string xcdnslbpoolname=$(echo ${wideip_list[$record.$domain]} | awk '{for (i=6; i<=(NF-1); i++) {printf "%s", $i; if (i < NF-1) printf " "}}') #echo "${a_record_per_zone[$domain]}" #echo "xcdnslbpoolname: $xcdnslbpoolname" # Check if xcdnslbpoolname has multiple strings if [[ $xcdnslbpoolname == *" "* ]]; then echo "Multiple strings found in xcdnslbpoolname" # Split xcdnslbpoolname into an array based on space IFS=' ' read -ra pool_names <<< "$xcdnslbpoolname" # Initialize an empty string to store the JSON strings pools_string="" # Loop through each pool name in the array for pool_name in "${pool_names[@]}"; do # Create JSON string for each member and append to the existing string pools_string+="{\"geo_location_set\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"geo-1\",\"kind\":\"geo_location_set\"},\"pool\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"$pool_name\",\"kind\":\"dns_lb_pool\"},\"score\":100}," done # Remove the trailing comma from the JSON string pools_string="${pools_string%,}" else pools_string="{\"geo_location_set\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"geo-1\",\"kind\":\"geo_location_set\"},\"pool\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"$xcdnslbpoolname\",\"kind\":\"dns_lb_pool\"},\"score\":100}" fi dnslbname=$(echo "dnslb-$record-$domain" | sed 's/\./-/g') #create_dnslb curl -X POST -H "Authorization: APIToken XXXXX" -H "Accept: application/json" -H "Access-Control-Allow-Origin: *" -H "x-volterra-apigw-tenant: cag-waap2023" -H "Content-Type: application/json" -d "{\"metadata\":{\"name\":\"$dnslbname\",\"namespace\":\"system\",\"labels\":{},\"annotations\":{},\"disable\":false},\"spec\":{\"record_type\":\"A\",\"rule_list\":{\"rules\":[$pools_string]},\"response_cache\":{\"disable\":{}}}}" https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_load_balancers a_records_string+="{\"ttl\":3600,\"lb_record\": {\"name\":\"$record\",\"value\":{\"namespace\": \"system\",\"name\":\"$dnslbname\"}}}," done # Remove the trailing comma from the JSON string a_records_string="${a_records_string%,}" # Print the final JSON string echo "$a_records_string" #update zone record curl -X PUT -H "Authorization: APIToken XXXXX" -H "Accept: application/json" -H "Access-Control-Allow-Origin: *" -H "x-volterra-apigw-tenant: cag-waap2023" -H "Content-Type: application/json" -d "{\"metadata\":{\"name\":\"$domain\",\"namespace\":\"system\"},\"spec\":{\"primary\":{\"allow_http_lb_managed_records\":true,\"default_rr_set_group\":[$a_records_string],\"default_soa_parameters\":{},\"dnssec_mode\":{},\"rr_set_group\":[],\"soa_parameters\":{\"refresh\":3600,\"expire\":0,\"retry\":60,\"negative_ttl\":0,\"ttl\":0}}}}" https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_zones/$domain done unset_arrays76Views0likes0CommentsPacket Analysis with Scapy and tcpdump: Checking Compatibility with F5 SSL Orchestrator
In this guide I want to demonstrate how you can use Scapy (https://scapy.net/) and tcpdump for reproducing and troubleshooting layer 2 issues with F5 BIG-IP devices. Just in case you get into a finger-pointing situation... Starting situation This is a quite recent story from the trenches: My customer uses a Bypass Tap to forward or mirror data traffic to inline tools such as IDS/IPS, WAF or threat intelligence systems. This ByPass Tap offers a feature called Network Failsafe (also known as Fail-to-Wire). This is a fault tolerance feature that protects the flow of data in the event of a power outage and/or system failure. It allows traffic to be rerouted while the inline tools (IDS/IPS, WAF or threat intelligence systems) are shutting down, restarting, or unexpectedly losing power (see red line namedFallbackin the picture below). Since the ByPass Tap itself does not have support for SSL decryption and re-encryption, an F5 BIG-IP SSL Orchestrator shall be introduced as an inline tool in a Layer 2 inbound topology. Tools directly connected to the Bypass Tap will be connected to the SSL Orchestrator for better visibility. To check the status of the inline tools, the Bypass Tap sends health checks through the inline tools. What is sent on one interface must be seen on the other interface and vice versa. So if all is OK (health check is green), traffic will be forwarded to the SSL Orchestrator, decrypted and sent to the IDS/IPS and the TAP, and then re-encrypted and sent back to the Bypass Tap. If the Bypass Tap detects that the SSL Orchestrator is in a failure state, it will just forward the traffic to the switch. This is the traffic flow of the health checks: Target topology This results in the following topology: Problem description During commissioning of the new topology, it turned out that the health check packets are not forwarded through the vWire configured on the BIG-IP. A packet analysis with Wireshark revealed that the manufacturer uses ARP-like packets with opcode 512 (HEX 02 00). This opcode is not defined in the RFC that describes ARP (https://datatracker.ietf.org/doc/html/rfc826), the RFC only describes the opcodes Request (1 or HEX 00 01) and Reply (2 or HEX 00 02). NOTE:Don't get confused that you see ARP packets on port 1.1 and 1.2. They are not passing through, the Bypass Tap is just send those packets from both sides of the vWire, as explained above. The source MAC on port 1.1 and 1.2 are different. Since the Bypass Tap is located right behind the customer's edge firewall, lengthy and time-consuming tests on the live system are not an option, since it would result in a massive service interruption. Therefore, a BIG-IP i5800 (the same model as the customer's) was set up as SSL Orchestrator and a vWire configuration was build in my employers lab. The vWire configuration can be found in this guide (https://clouddocs.f5.com/sslo-deployment-guide/chapter2/page2.7.html). INFO:For those not familiar with vWire: "Virtual wire … creates a layer 2 bridge across the defined interfaces. Any traffic that does not match a topology listener will pass across this bridge." Lab Topology The following topology was used for the lab: I build a vWire configuration on the SSL Orchestrator, as in the customer's environment. A Linux system with Scapy installed was connected to Interface 1.1. With Scapy TCP, UDP and ARP packets can be crafted or sent like a replay from a Wireshark capture. Interface 1.3 was connected to another Linux system that should receive the ARP packets. All tcpdumps were captured on the F5 and analyzed on the admin system (not plotted). Validating vWire Configuration To check the functionality of the F5 and the vWire configuration, two tests were performed. A replay of the Healthcheck packets from the Bypass Tap and a test with RFC-compliant ARP requests. Use Scapy to resend the faulty packets First, I used Wireshark to extract a single packet from packet analysis we took in the customer environment and saved it to a pcap file. I replayed this pcap file to the F5 with Scapy. The sendp() function will work at layer 2, it requires the parametersrdpcap(location of the pcap file for replay) andiface(which interface it shall use for sending). webserverdude@tux480:~$ sudo scapy -H WARNING: IPython not available. Using standard Python shell instead. AutoCompletion, History are disabled. Welcome to Scapy (2.5.0) >>> sendp(rdpcap("/home/webserverdude/cusomter-case/bad-example.pcap"),iface="enp0s31f6") . Sent 1 packets. This test confirmed the behavior that was observed in the customer's environment. The F5 BIG-IP does not forward this packet. Use PING and Scapy to send RFC-compliant ARP packets To create RFC-compliant ARP requests, I first sent an ARP request (opcode 1) through the vWire via PING command. As expected, this was sent through the vWire. To ensure that this also works with Scapy, I also resent this packet with Scapy. >>> sendp(rdpcap("/home/webserverdude/cusomter-case/good-example.pcap"),iface="enp0s31f6") . Sent 1 packets. In the Wireshark analysis it can be seen that this packet is incoming on port 1.1 and then forwarded to port 1.3 through the vWire. Solving the issue with the help of the vendor It became evident that the BIG-IP was dropping ARP packets that failed to meet RFC compliance, rendering the Bypass Tap from this particular vendor seemingly incompatible with the BIG-IP. Following my analysis, the vendor was able to develop and provide a new firmware release addressing this issue. To verify that the issue was resolved in this firmware release, my customer's setup, the exact same model of the Bypass Tap and a BIG-IP i5800, were deployed in my lab, where the new firmware underwent thorough testing. With this approach I could test the functionality and compatibility of the systems under controlled conditions. In this Wireshark analysis it can be seen that the Healthcheck packets are incoming on port 1.1 and then forwarded to port 1.3 through the vWire (marked in green) and also the other way round, coming in on port 1.3 and then forwarded to port 1.1 (marked in pink). Also now you can see that the packet is a proper gratuitous ARP reply (https://wiki.wireshark.org/Gratuitous_ARP). Because the Healthcheck packets were not longer dropped by the BIG-IP, but were forwarded through the vWire the Bypass Tap subsequently marked the BIG-IP as healthy and available. The new firmware resolved the issue. Consequently, my customer could confidently proceed with this project, free from the constraints imposed by the compatibility issue.350Views2likes2CommentsF5 BIG-IP iRule based TCL DNS server
Short Description This code is for the purpose if there is a need to return a custom DNS reply not from the main DNS server but from the F5 Virtual Server. Problem solved by this Code Snippet The code is meant for small lab or dev environments as F5 has F5 DNS/GTM for replying to DNS requests with Intelligent Load Balancing, DNS Express Memory cache etc. How to use this Code Snippet Modify the irule code to configure the custom domain you want replies from the irule code and not the real DNS server. Add the irule to a DNS UDP LB Code Snippet Meta Information Version: 17.1 Coding Language: TCL Code You can find the code and further documentation in my GitHub repository: irule_dns/irule1 at main · Nikoolayy1/irule_dns (github.com)75Views0likes0Comments