Create DNS monitor without external dig or nslookup
Problem this snippet solves: NOTE: This was written in 10.x and early 11.x days. Since then, a native DNS monitor has been added to the product. This example will create a udp monitor with proper send and receive strings to query a DNS server. Much of the heavy lifting was pulled from an iApp template that ships with v11. I have tested this out on as far back as 10.2.1 base. Installation One method to get this into your config would be: In the code block below, click the icon in the top right corner '<>' view source. Ctrl-A, Ctrl-C to select and copy all. Then in tmsh, type 'edit /cli script dns.monitor' If your editor is configured to use vi (the default setting) Type :set paste, then c, then Shift + G. This will clear out the default skeleton. You will now be in insert mode. Press Shift + insert. This will paste in the script. Now save the script by pressing Esc, then typing :wq < enter > and follow the prompts. How to use this snippet: Example root@bigip(Active)(/Common)(tmos)# run cli script dns.monitor xxx Requires 5 arguments: := Name you want to give the monitor := A, NS, PTR, SOA, CNAME := What you are asking the DNS server (send string) := Why you expect for a "healthy" response (recv string) := The frequency you want the monitor to fire e.g. dns.monitor my_dns A www.example.com 192.0.2.55 5 root@bigip(Active)(/Common)(tmos)# run cli script dns.monitor my_dns A www.example.com 192.0.2.55 5 root@bigip(Active)(/Common)(tmos)# list ltm monitor udp my_dns ltm monitor udp my_dns { debug no defaults-from udp destination *:* interval 5 recv \\x00\\x01.*\\x04\\xc0\\x00\\x02\\x37 send \\x96\\x87\\x01\\x00\\x00\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x03www\\x07example\\x03com\\x00\\x00\\x01\\x00\\x01 time-until-up 0 timeout 16 } root@bigip(Active)(/Common)(tmos)# Code : create script dns.monitor { proc script::init {} { } proc script::run {} { if {$tmsh::argc == 6} { # create monitor set monitor_name [lindex $tmsh::argv 1] set monitor_type [string toupper [ lindex $tmsh::argv 2 ]] switch $monitor_type { A { set monitor_send [generate_monitor_send_string_a_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_a_record [lindex $tmsh::argv 4]] } NS { set monitor_send [generate_monitor_send_string_ns_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_ns_record [lindex $tmsh::argv 4] [lindex $tmsh::argv 3]] } PTR { set monitor_send [generate_monitor_send_string_ptr_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_ptr_record [lindex $tmsh::argv 4]] } SOA { set monitor_send [generate_monitor_send_string_soa_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_soa_record [lindex $tmsh::argv 4] [lindex $tmsh::argv 3]] } CNAME { set monitor_send [generate_monitor_send_string_cname_record [lindex $tmsh::argv 3]] set monitor_recv [generate_monitor_recv_string_cname_record [lindex $tmsh::argv 4] [lindex $tmsh::argv 3]] } default { usage } } set monitor_interval [lindex $tmsh::argv 5] set monitor_timeout [ expr { [ expr {$monitor_interval * 3} ] + 1 } ] # doctor the send and receive strings so that they go through as intended set monitor_send [doctor_hex_string $monitor_send] set monitor_recv [doctor_hex_string $monitor_recv] tmsh_create "ltm monitor udp $monitor_name defaults-from udp interval $monitor_interval timeout $monitor_timeout send \"$monitor_send\" recv \"$monitor_recv\"" } else { usage } } proc script::help {} { tmsh::add_help "Create a DNS monitor \n\ " } proc script::tabc {} { } proc tmsh_create { arguments } { regsub -all {\[} $arguments "\\\[" arguments regsub -all {\]} $arguments "\\\]" arguments tmsh::create "$arguments" } proc number_to_hex_string { number } { if { $number < 0 } { set number [ expr {0 - $number} ] } set hex_string [format "\\x%02x" $number] return $hex_string } proc hostname_or_ip_address_to_dns_name { hostname } { set fields [split $hostname .] set dns_name "" foreach field $fields { set len [string length $field] set str_len [number_to_hex_string $len] append dns_name $str_len append dns_name $field } append dns_name "\\x00" return $dns_name } proc hostname_or_ip_address_to_reverse_dns_name { hostname } { set complete_hostname [format "arpa.in-addr.%s" $hostname] set fields [split $complete_hostname .] set dns_name "" foreach field $fields { # if [string is integer $field] { set len [string length $field] set str_len [number_to_hex_string $len] set dns_name [format "%s%s%s" $str_len $field $dns_name] # } } append dns_name "\\x00" return $dns_name } proc get_id { } { set value1 [expr { int(256 * rand()) }] set value2 [expr { int(256 * rand()) }] set id [number_to_hex_string $value1] append id [number_to_hex_string $value2] return $id } proc generate_dns_packet_payload { question } { set payload [get_id] append payload "\\x01\\x00" append payload "\\x00\\x01" append payload "\\x00\\x00" append payload "\\x00\\x00" append payload "\\x00\\x00" append payload $question return $payload } proc ip_addr_to_a_resp_addr { address } { set fields [split $address .] set num_fields 0 set field_encoding "" foreach field $fields { if {[string is integer $field]} { set num_fields [expr { $num_fields + 1 }] set part [number_to_hex_string $field] append field_encoding $part } } set dns_name [number_to_hex_string $num_fields] append dns_name $field_encoding return $dns_name } proc generate_send_payload { input_string record_type_code reverse } { if { $reverse == 0 } { set dns_name [hostname_or_ip_address_to_dns_name $input_string] } else { set dns_name [hostname_or_ip_address_to_reverse_dns_name $input_string] } set internet_address_qclass "\\x00\\x01" set question $dns_name append question $record_type_code append question $internet_address_qclass set send [generate_dns_packet_payload $question] return $send } proc generate_monitor_send_string_a_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x01" 0] return $send } proc generate_monitor_send_string_ns_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x02" 0] return $send } proc generate_monitor_send_string_ptr_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x0c" 1] return $send } proc generate_monitor_send_string_soa_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x06" 0] return $send } proc generate_monitor_send_string_cname_record { input_string } { set send [generate_send_payload $input_string "\\x00\\x05" 0] return $send } proc generate_monitor_recv_string_a_record { input_string } { set a_resp_addr [ip_addr_to_a_resp_addr $input_string] set recv [format "\\x00\\x01.*%s" $a_resp_addr] return $recv } proc generate_monitor_recv_string_ns_record { input_string domain_name } { regsub -all {\.} $domain_name \. domain_name set re [format "^(.*)(\.%s)$" $domain_name] set found [regexp $re $input_string matched new_hostname rest] if { $found != 0 } { set recv_temp [hostname_or_ip_address_to_dns_name $new_hostname] set re {^(.*\\)(x00)$} set found [regexp -nocase $re $recv_temp matched recv rest] if { $found != 0 } { set recv [format "\\x00\\x02.*%sxc0" $recv] } else { set recv [format "\\x00\\x02.*%s" $recv_temp] } } else { set recv [hostname_or_ip_address_to_dns_name $input_string] set recv [format "\\x00\\x02.*%s" $recv] } return $recv } proc generate_monitor_recv_string_ptr_record { input_string } { set dns_name [hostname_or_ip_address_to_dns_name $input_string] set recv [format "\\x00\\x0c.*%s" $dns_name] return $recv } proc generate_monitor_recv_string_soa_record { input_string domain_name } { regsub -all {\.} $domain_name \. domain_name set re [format "^(.*)(\.%s)$" $domain_name] set found [regexp -nocase $re $input_string matched new_hostname rest] if { $found != 0 } { set recv_temp [hostname_or_ip_address_to_dns_name $new_hostname] set re {^(.*)(\\x00)$} set found [regexp -nocase $re $recv_temp matched recv rest] if { $found != 0 } { set recv [format "\\x00\\x06.*%s(\\xc0|\\x05)" $recv] } else { set recv [format "\\x00\\x06.*%s" $recv_temp] } } else { set recv [hostname_or_ip_address_to_dns_name $input_string] set recv [format "\\x00\\x06.*%s" $recv] } return $recv } proc generate_monitor_recv_string_cname_record { input_string hostname } { regsub -all {\.} $hostname \. hostname #puts "hostname = $hostname" set re {^([^\.]+)\.(.+)$} #puts "re = $re" set found [regexp -nocase $re $hostname matched first domain_name] #puts "found = $found" if { $found == 0 } { set domain_name $hostname } regsub -all {\.} $domain_name \. domain_name #puts "domain_name = $domain_name" #puts "input_string = $input_string" set re [format "^(.*)(\.%s)$" $domain_name] #puts "re = $re" set found [regexp -nocase $re $input_string matched new_hostname rest] if { $found != 0 } { set recv_temp [hostname_or_ip_address_to_dns_name $new_hostname] set re {^(.*\\)(x00)$} set found [regexp -nocase $re $recv_temp matched recv rest] if { $found != 0 } { set re {^(.*)(\\x00)$} set found [regexp -nocase $re $recv_temp matched recv rest] set recv [format "\\x00\\x05.*%s(\\xc0|\\x05)" $recv] } else { set recv [format "\\x00\\x05.*%s" $recv_temp] } } else { set recv [hostname_or_ip_address_to_dns_name $input_string] set recv [format "\\x00\\x05.*%s" $recv] } return $recv } proc doctor_hex_string { input } { set pieces [split $input \\ ] set output "" set first_time 1 foreach piece $pieces { if { $first_time == 1 } { set first_time 0 } else { append output {\\} } append output $piece } return $output } proc usage { } { puts "Requires 5 arguments: \n\ := Name you want to give the monitor \n\ := A, NS, PTR, SOA, CNAME \n\ := What you are asking the DNS server (send string) \n\ := Why you expect for a \"healthy\" response (recv string) \n\ := The frequency you want the monitor to fire \n\n\ e.g. [lindex $tmsh::argv 0] A www.example.com 192.0.2.55 5" exit } }Generate private key w/ CSR via iControl REST
Problem this snippet solves: Generate a private key w/ CSR How to use this snippet: To create a private key with a CSR via iControl REST: POST URL:https://10.1.1.165/mgmt/tm/sys/crypto/key Use the data below as your payload. For the name field, it must end in .key or you will get a false 404! Code : { "name":"www.testing.com.key", "commonName":"www.testing.com", "keySize":"4096", "keyType":"rsa-private", "options":[{"gen-csr":"www.testing.com"}], "organization":"Let It Snow Corp.", "ou":"Ice Engineering", "city":"Calhoun", "state":"AZ", "admin-email-address":"jerry@letit.snow", "email-address":"beth@letit.snow", "subject-alternative-name":"DNS:www.testing.com", "challenge-password":"myP4ssword" } Tested this on version: 13.0
FQDN nodes in non-default route domains
Problem this snippet solves: Currently there is no support for FQDN nodes in non-default route domains as per Article. With recent increase in cloud deployment, most of the time there is a requirement from F5 to load balance to servers or ELB in the cloud which has FQDN names as those are having dyanamic IP addresses. If you are using route domains in your BIGIP environment then this becomes a challenge. Below iRule script can be used in those scenerios to use F5 to send traffic to FQDN nodes in non-default route domains. How to use this snippet: If FQDN needs to be resolved by your internal DNS, create performance layer 4 VIP (dns_53) load balancing your DNS servers. DNS server can be used directly in the iRule itself, but it would be better to use a VIP to have redundancy. Create iRule and apply to the VIP Code : when CLIENT_ACCEPTED { set host [RESOLV::lookup @dns_53 "server.example.com"] set ip [getfield $host " " 1] node [lindex $ip 0]%<Rd> 443 } Tested this on version: 14.1BIG-IP Report
Problem this snippet solves: Overview This is a script which will generate a report of the BIG-IP LTM configuration on all your load balancers making it easy to find information and get a comprehensive overview of virtual servers and pools connected to them. This information is used to relay information to NOC and developers to give them insight in where things are located and to be able to plan patching and deploys. I also use it myself as a quick way get information or gather data used as a foundation for RFC's, ie get a list of all external virtual servers without compression profiles. The script has been running on 13 pairs of load balancers, indexing over 1200 virtual servers for several years now and the report is widely used across the company and by many companies and governments across the world. It's easy to setup and use and only requires auditor (read-only) permissions on your devices. Demo/Preview Interactive demo http://loadbalancing.se/bigipreportdemo/ Screen shots The main report: The device overview: Certificate details: How to use this snippet: Installation instructions BigipReport REST This is the only branch we're updating since middle of 2020 and it supports 12.x and upwards (maybe even 11.6). Downloads (two latest versions): https://loadbalancing.se/downloads/bigipreport-v5.7.11.zip https://loadbalancing.se/downloads/bigipreport-v5.7.10.zip Documentation, installation instructions and troubleshooting:https://loadbalancing.se/bigipreport-rest/ Docker support https://loadbalancing.se/2021/01/05/running-bigipreport-on-docker/ Kubernetes support https://loadbalancing.se/2021/04/16/bigipreport-on-kubernetes/ BIG-IP Report (Legacy) Older version of the report that only runs on Windows and is depending on a Powershell plugin originally written by Joe Pruitt (F5) BIG-IP Report (only download this if you have v10 devices): https://loadbalancing.se/downloads/bigipreport-5.4.0-beta.zip iControl Snapin https://loadbalancing.se/downloads/f5-icontrol.zip Documentation and Installation Instructions https://loadbalancing.se/bigip-report/ Upgrade instructions Protect the report using APM and active directory Written by DevCentral member Shann_P: https://loadbalancing.se/2018/04/08/protecting-bigip-report-behind-an-apm-by-shannon-poole/ Got issues/problems/feedback? Still have issues? Drop a comment below. We usually reply quite fast. Any bugs found, issues detected or ideas contributed makes the report better for everyone, so it's always appreciated. --- Join us on Discord: https://discord.gg/7JJvPMYahA Code : BigIP Report Tested this on version: 12, 13, 14, 15, 16
BIG-IP Wide-IP to F5XC DNSLB converter
This is a conceptual sample script that converts BIG-IP Wide-IP records to F5XC DNSLB records. This bash script can be run using a cron job to check for configuration changes and synchronize them to F5XC. We used the F5XC API to post and update the configuration. You need to get an APIToken from your F5XC tenant and change the value on the POST commands on the script below. Note: Since this is not a full-blown converter script, it is limited to handling only a single Wide-IP pool member. You need to configure a GTM pool to include the IP addresses that need to be load balanced. Check the main article for more details. #!/bin/bash # Get list of wide IPs wideip_output=$(tmsh list gtm wideip all-properties one-line) # Get list of Pool pool_output=$(tmsh list gtm pool a one-line all-properties) # Declare associative arrays declare -A wideip_list declare -A current_wideip_info declare -A zone_array declare -A subdomain_info declare -A a_record_per_zone declare -A pool_list declare -A membersip_array # Unset variables function unset_arrays { unset current_wideip_info name subdomain domain type aliases description status failure_rcode last_resort_pool load_balancing_decision_log metadata minimal_response partition persist_cidr_ipv4 persist_cidr_ipv6 persistence pool_lb_mode pools pool_cname topology_edns0 ttl_persistence poolnames poolnames_array zone_array subdomain_info a_record_per_zone dnslb_name pool_list membersip_array } # Print wide IP details function print_wideip { for wideip in "${!wideip_list[@]}"; do echo "Wide IP: $wideip, Details: ${wideip_list[$wideip]}" done } # Create Zone function create_zone { curl -X POST -H "Authorization: APIToken XXXXX" -H "Accept: application/json" -H "Access-Control-Allow-Origin: *" -H "x-volterra-apigw-tenant: cag-waap2023" -H "Content-Type: application/json" -d "{\"metadata\":{\"name\":\"$zone\",\"namespace\":\"system\"},\"spec\":{\"primary\":{\"allow_http_lb_managed_records\":true},\"default_soa_parameters\":{},\"dnssec_mode\":{},\"rr_set_group\":[],\"soa_parameters\":{\"refresh\":3600,\"expire\":0,\"retry\":60,\"negative_ttl\":0,\"ttl\":0}}}" https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_zones } # Create DNSLB function create_dnslb { curl -X POST -H "Authorization: APIToken XXXXX" -H "Accept: application/json" -H "Access-Control-Allow-Origin: *" -H "x-volterra-apigw-tenant: cag-waap2023" -H "Content-Type: application/json" -d "{\"metadata\":{\"name\":\"$dnslbname\",\"namespace\":\"system\",\"labels\":{},\"annotations\":{},\"disable\":false},\"spec\":{\"record_type\":\"A\",\"rule_list\":{\"rules\":[{\"geo_location_set\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"geo-1\",\"kind\":\"geo_location_set\"},\"pool\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"$xcdnslbpoolname\",\"kind\":\"dns_lb_pool\"},\"score\":100}]},\"response_cache\":{\"disable\":{}}}}" https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_load_balancers } # Loop through each line of output while IFS= read -r line; do pool_name=$(awk '{print $4}' <<< "$line") dnslbpool_name=$(echo "$pool_name" | sed 's/[^a-zA-Z0-9]/-/g; s/.*/\L&/') pool_type=$(awk '{print $3}' <<< "$line") lbmode=$(grep -o 'load-balancing-mode [^ ]*' <<< "$line" | awk '{print $2}') # Convert load_balancing_mode to lowercase if it is "ROUND_ROBIN" if [[ "$lbmode" == "round-robin" ]]; then lbmode="ROUND_ROBIN" elif [[ "$lbmode" == "static-persistence" ]]; then lbmode="STATIC_PERSIST" elif [[ "$lbmode" == "global-availability" ]]; then lbmode="PRIORITY" elif [[ "$lbmode" == "ratio" ]]; then lbmode="RATIO_MEMBER" fi # Extract members block using awk #members=$(awk -F 'members {| }' '{print $2}' <<< "$line") members=$(echo "$line" | grep -o -P '(?<=members \{ ).*?(?=\} \})') membernames=$(echo "$members" | grep -oP '\S+(?=\s*{)') # Temporary array to hold member IP addresses declare -a temp_members_array temp_members_array=($(awk -F ':' '{print $2}' <<< "$membernames")) monitor=$(awk -F 'monitor ' '{print $2}' <<< "$line" | awk '{print $1}') ttl=$(awk '{print $2}' <<< "$(grep -o 'ttl [^ ]*' <<< "$line")") # Assign values to the associative array membersip_array["$dnslbpool_name"]="${temp_members_array[@]}" # Store extracted values in the array pool_list["$dnslbpool_name"]="pool_type: $pool_type, lbmode: $lbmode, monitor: $monitor, members: ${membersip_array["$dnslbpool_name"]}, ttl: $ttl" done <<< "$pool_output" # Loop through each pool in the pool_list for dnslbpool_name in "${!pool_list[@]}"; do # Extract only the TTL value from the string ttl=$(awk -F 'ttl: ' '{print $2}' <<< "${pool_list[$dnslbpool_name]}") lbmode=$(awk -F 'lbmode: ' '{print $2}' <<< "${pool_list[$dnslbpool_name]}" | awk -F ',' '{print $1}') members=$(awk -F 'members: ' '{print $2}' <<< "${pool_list[$dnslbpool_name]}" | awk -F ',' '{print $1}') pool_type=$(awk -F 'pool_type: ' '{print $2}' <<< "${pool_list[$dnslbpool_name]}" | awk -F ',' '{print $1}') # Check if pool_type is "a" if [[ "$pool_type" == "a" ]]; then # Initialize an empty string to store the JSON strings members_string="" # Loop through each record in the current zone for ip in ${membersip_array["$dnslbpool_name"]}; do # Create JSON string for each member and append to the existing string members_string+="{\"ip_endpoint\":\"$ip\",\"ratio\":10,\"priority\":1}," done # Remove the trailing comma from the JSON string members_string="${members_string%,}" # Create DNSLB Pools curl -X POST \ -H "Authorization: APIToken Rs0aGJm/lda/JmbE00c9lFXWw4I=" \ -H "Accept: application/json" \ -H "Access-Control-Allow-Origin: *" \ -H "x-volterra-apigw-tenant: cag-waap2023" \ -H "Content-Type: application/json" \ -d "{\"metadata\":{\"name\":\"$dnslbpool_name\",\"namespace\":\"system\"},\"spec\":{\"a_pool\":{\"members\":[$members_string],\"disable_health_check\":null,\"max_answers\":1},\"ttl\":\"$ttl\",\"load_balancing_mode\":\"$lbmode\"}}" \ "https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_lb_pools" fi done # Unset variables to free up memory unset pool_list membersip_array # Loop through each line of output while IFS= read -r line; do # Extracting specific details using awk and sed based on the current line name=$(echo "$line" | awk '{print $4}') dnslb_name=$(echo "$name" | sed 's/\./-/g') subdomain=$(echo "$name" | cut -d'.' -f1) domain=$(echo "$name" | sed 's/^[^.]*\.//') type=$(echo "$line" | awk '{print $3}') aliases=$(echo "$line" | grep -o 'aliases [^}]*' | awk '{print $2}') description=$(echo "$line" | grep -o 'description [^ ]*' | sed 's/description //') status=$(echo "$line" | awk '{print $12}') failure_rcode=$(echo "$line" | grep -o 'failure-rcode [^ ]*' | sed 's/failure-rcode //') last_resort_pool=$(echo "$line" | grep -o 'last-resort-pool [^ ]*' | sed 's/last-resort-pool //') load_balancing_decision_log=$(echo "$line" | grep -o 'load-balancing-decision-log-verbosity [^ ]*' | sed 's/load-balancing-decision-log-verbosity //') metadata=$(echo "$line" | grep -o 'metadata [^ ]*' | sed 's/metadata //') minimal_response=$(echo "$line" | grep -o 'minimal-response [^ ]*' | sed 's/minimal-response //') partition=$(echo "$line" | grep -o 'partition [^ ]*' | sed 's/partition //') persist_cidr_ipv4=$(echo "$line" | grep -o 'persist-cidr-ipv4 [^ ]*' | sed 's/persist-cidr-ipv4 //') persist_cidr_ipv6=$(echo "$line" | grep -o 'persist-cidr-ipv6 [^ ]*' | sed 's/persist-cidr-ipv6 //') persistence=$(echo "$line" | grep -o ' persistence [^ ]*' | sed 's/persistence //') pool_lb_mode=$(echo "$line" | grep -o 'pool-lb-mode [^ ]*' | sed 's/pool-lb-mode //') pools=$(echo "$line" | grep -o -P '(?<=pools \{ ).*?(?=\} \})') pool_cname=$(echo "$line" | grep -o 'pools-cname [^ ]*' | sed 's/pools-cname //') topology_edns0=$(echo "$line" | grep -o 'topology-prefer-edns0-client-subnet [^ ]*' | sed 's/topology-prefer-edns0-client-subnet //') ttl_persistence=$(echo "$line" | grep -o 'ttl-persistence [^ ]*' | sed 's/ttl-persistence //') # Use grep to find strings before "{" poolnames=$(echo "$pools" | grep -oP '\S+(?=\s*{)' | sed 's/[^a-zA-Z0-9]/-/g; s/.*/\L&/') # Convert matches to an array readarray -t poolnames_array <<< "$poolnames" # Store extracted values in the associative array current_wideip_info=([Type]="$type" [Subdomain]="$subdomain" [Domain]="$domain" [Status]="$status" [DNSLB]="$dnslb_name" [Pools]="${poolnames_array[@]}" [Pool_LB_Mode]="$pool_lb_mode") # Assign wideip_info to wideip_list wideip_list["$name"]="${current_wideip_info[@]}" # Add subdomains to zone_array if [ -n "${zone_array[$domain]}" ]; then zone_array["$domain"]="${zone_array[$domain]},$subdomain" else zone_array["$domain"]=$subdomain fi # Store subdomain information in subdomain_info array subdomain_info["$subdomain"]="${current_wideip_info[@]}" # Store subdomain type "a" and add it to the array for that zone if [ "$type" == "a" ]; then a_record_per_zone[$domain]="${a_record_per_zone[$domain]}${a_record_per_zone[$domain]:+,}$subdomain" fi done <<< "$wideip_output" for zone in "${!zone_array[@]}"; do create_zone done # Loop through each domain in a_record_per_zone and echo its A record subdomains for domain in "${!a_record_per_zone[@]}"; do echo "Domain: $domain" echo "A Record Subdomains: ${a_record_per_zone[$domain]}" echo "--------------------------" # Initialize an empty string to store the JSON strings a_records_string="" # Loop through each record in the current zone for record in ${a_record_per_zone[$domain]//,/ }; do # Create JSON string for each A record and append to the existing string xcdnslbpoolname=$(echo ${wideip_list[$record.$domain]} | awk '{for (i=6; i<=(NF-1); i++) {printf "%s", $i; if (i < NF-1) printf " "}}') #echo "${a_record_per_zone[$domain]}" #echo "xcdnslbpoolname: $xcdnslbpoolname" # Check if xcdnslbpoolname has multiple strings if [[ $xcdnslbpoolname == *" "* ]]; then echo "Multiple strings found in xcdnslbpoolname" # Split xcdnslbpoolname into an array based on space IFS=' ' read -ra pool_names <<< "$xcdnslbpoolname" # Initialize an empty string to store the JSON strings pools_string="" # Loop through each pool name in the array for pool_name in "${pool_names[@]}"; do # Create JSON string for each member and append to the existing string pools_string+="{\"geo_location_set\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"geo-1\",\"kind\":\"geo_location_set\"},\"pool\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"$pool_name\",\"kind\":\"dns_lb_pool\"},\"score\":100}," done # Remove the trailing comma from the JSON string pools_string="${pools_string%,}" else pools_string="{\"geo_location_set\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"geo-1\",\"kind\":\"geo_location_set\"},\"pool\":{\"tenant\":\"cag-waap2023-gwjvytud\",\"namespace\":\"system\",\"name\":\"$xcdnslbpoolname\",\"kind\":\"dns_lb_pool\"},\"score\":100}" fi dnslbname=$(echo "dnslb-$record-$domain" | sed 's/\./-/g') #create_dnslb curl -X POST -H "Authorization: APIToken XXXXX" -H "Accept: application/json" -H "Access-Control-Allow-Origin: *" -H "x-volterra-apigw-tenant: cag-waap2023" -H "Content-Type: application/json" -d "{\"metadata\":{\"name\":\"$dnslbname\",\"namespace\":\"system\",\"labels\":{},\"annotations\":{},\"disable\":false},\"spec\":{\"record_type\":\"A\",\"rule_list\":{\"rules\":[$pools_string]},\"response_cache\":{\"disable\":{}}}}" https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_load_balancers a_records_string+="{\"ttl\":3600,\"lb_record\": {\"name\":\"$record\",\"value\":{\"namespace\": \"system\",\"name\":\"$dnslbname\"}}}," done # Remove the trailing comma from the JSON string a_records_string="${a_records_string%,}" # Print the final JSON string echo "$a_records_string" #update zone record curl -X PUT -H "Authorization: APIToken XXXXX" -H "Accept: application/json" -H "Access-Control-Allow-Origin: *" -H "x-volterra-apigw-tenant: cag-waap2023" -H "Content-Type: application/json" -d "{\"metadata\":{\"name\":\"$domain\",\"namespace\":\"system\"},\"spec\":{\"primary\":{\"allow_http_lb_managed_records\":true,\"default_rr_set_group\":[$a_records_string],\"default_soa_parameters\":{},\"dnssec_mode\":{},\"rr_set_group\":[],\"soa_parameters\":{\"refresh\":3600,\"expire\":0,\"retry\":60,\"negative_ttl\":0,\"ttl\":0}}}}" https://cag-waap2023.console.ves.volterra.io/api/config/dns/namespaces/system/dns_zones/$domain done unset_arraysPacket Analysis with Scapy and tcpdump: Checking Compatibility with F5 SSL Orchestrator
In this guide I want to demonstrate how you can use Scapy (https://scapy.net/) and tcpdump for reproducing and troubleshooting layer 2 issues with F5 BIG-IP devices. Just in case you get into a finger-pointing situation... Starting situation This is a quite recent story from the trenches: My customer uses a Bypass Tap to forward or mirror data traffic to inline tools such as IDS/IPS, WAF or threat intelligence systems. This ByPass Tap offers a feature called Network Failsafe (also known as Fail-to-Wire). This is a fault tolerance feature that protects the flow of data in the event of a power outage and/or system failure. It allows traffic to be rerouted while the inline tools (IDS/IPS, WAF or threat intelligence systems) are shutting down, restarting, or unexpectedly losing power (see red line namedFallbackin the picture below). Since the ByPass Tap itself does not have support for SSL decryption and re-encryption, an F5 BIG-IP SSL Orchestrator shall be introduced as an inline tool in a Layer 2 inbound topology. Tools directly connected to the Bypass Tap will be connected to the SSL Orchestrator for better visibility. To check the status of the inline tools, the Bypass Tap sends health checks through the inline tools. What is sent on one interface must be seen on the other interface and vice versa. So if all is OK (health check is green), traffic will be forwarded to the SSL Orchestrator, decrypted and sent to the IDS/IPS and the TAP, and then re-encrypted and sent back to the Bypass Tap. If the Bypass Tap detects that the SSL Orchestrator is in a failure state, it will just forward the traffic to the switch. This is the traffic flow of the health checks: Target topology This results in the following topology: Problem description During commissioning of the new topology, it turned out that the health check packets are not forwarded through the vWire configured on the BIG-IP. A packet analysis with Wireshark revealed that the manufacturer uses ARP-like packets with opcode 512 (HEX 02 00). This opcode is not defined in the RFC that describes ARP (https://datatracker.ietf.org/doc/html/rfc826), the RFC only describes the opcodes Request (1 or HEX 00 01) and Reply (2 or HEX 00 02). NOTE:Don't get confused that you see ARP packets on port 1.1 and 1.2. They are not passing through, the Bypass Tap is just send those packets from both sides of the vWire, as explained above. The source MAC on port 1.1 and 1.2 are different. Since the Bypass Tap is located right behind the customer's edge firewall, lengthy and time-consuming tests on the live system are not an option, since it would result in a massive service interruption. Therefore, a BIG-IP i5800 (the same model as the customer's) was set up as SSL Orchestrator and a vWire configuration was build in my employers lab. The vWire configuration can be found in this guide (https://clouddocs.f5.com/sslo-deployment-guide/chapter2/page2.7.html). INFO:For those not familiar with vWire: "Virtual wire … creates a layer 2 bridge across the defined interfaces. Any traffic that does not match a topology listener will pass across this bridge." Lab Topology The following topology was used for the lab: I build a vWire configuration on the SSL Orchestrator, as in the customer's environment. A Linux system with Scapy installed was connected to Interface 1.1. With Scapy TCP, UDP and ARP packets can be crafted or sent like a replay from a Wireshark capture. Interface 1.3 was connected to another Linux system that should receive the ARP packets. All tcpdumps were captured on the F5 and analyzed on the admin system (not plotted). Validating vWire Configuration To check the functionality of the F5 and the vWire configuration, two tests were performed. A replay of the Healthcheck packets from the Bypass Tap and a test with RFC-compliant ARP requests. Use Scapy to resend the faulty packets First, I used Wireshark to extract a single packet from packet analysis we took in the customer environment and saved it to a pcap file. I replayed this pcap file to the F5 with Scapy. The sendp() function will work at layer 2, it requires the parametersrdpcap(location of the pcap file for replay) andiface(which interface it shall use for sending). webserverdude@tux480:~$ sudo scapy -H WARNING: IPython not available. Using standard Python shell instead. AutoCompletion, History are disabled. Welcome to Scapy (2.5.0) >>> sendp(rdpcap("/home/webserverdude/cusomter-case/bad-example.pcap"),iface="enp0s31f6") . Sent 1 packets. This test confirmed the behavior that was observed in the customer's environment. The F5 BIG-IP does not forward this packet. Use PING and Scapy to send RFC-compliant ARP packets To create RFC-compliant ARP requests, I first sent an ARP request (opcode 1) through the vWire via PING command. As expected, this was sent through the vWire. To ensure that this also works with Scapy, I also resent this packet with Scapy. >>> sendp(rdpcap("/home/webserverdude/cusomter-case/good-example.pcap"),iface="enp0s31f6") . Sent 1 packets. In the Wireshark analysis it can be seen that this packet is incoming on port 1.1 and then forwarded to port 1.3 through the vWire. Solving the issue with the help of the vendor It became evident that the BIG-IP was dropping ARP packets that failed to meet RFC compliance, rendering the Bypass Tap from this particular vendor seemingly incompatible with the BIG-IP. Following my analysis, the vendor was able to develop and provide a new firmware release addressing this issue. To verify that the issue was resolved in this firmware release, my customer's setup, the exact same model of the Bypass Tap and a BIG-IP i5800, were deployed in my lab, where the new firmware underwent thorough testing. With this approach I could test the functionality and compatibility of the systems under controlled conditions. In this Wireshark analysis it can be seen that the Healthcheck packets are incoming on port 1.1 and then forwarded to port 1.3 through the vWire (marked in green) and also the other way round, coming in on port 1.3 and then forwarded to port 1.1 (marked in pink). Also now you can see that the packet is a proper gratuitous ARP reply (https://wiki.wireshark.org/Gratuitous_ARP). Because the Healthcheck packets were not longer dropped by the BIG-IP, but were forwarded through the vWire the Bypass Tap subsequently marked the BIG-IP as healthy and available. The new firmware resolved the issue. Consequently, my customer could confidently proceed with this project, free from the constraints imposed by the compatibility issue.F5 BIG-IP iRule based TCL DNS server
Short Description This code is for the purpose if there is a need to return a custom DNS reply not from the main DNS server but from the F5 Virtual Server. Problem solved by this Code Snippet The code is meant for small lab or dev environments as F5 has F5 DNS/GTM for replying to DNS requests with Intelligent Load Balancing, DNS Express Memory cache etc. How to use this Code Snippet Modify the irule code to configure the custom domain you want replies from the irule code and not the real DNS server. Add the irule to a DNS UDP LB Code Snippet Meta Information Version: 17.1 Coding Language: TCL Code You can find the code and further documentation in my GitHub repository: irule_dns/irule1 at main · Nikoolayy1/irule_dns (github.com)DNS Query Name Parsing iRule
Problem this snippet solves: This iRule will extract the DNS Query Name in the absence of a DNS profile being applied to a Virtual Server. How to use this snippet: # This is a shameless rip from an old Devcentral post DNS Hostname Parsing iRule that, to the best of my knowledge, never made it to a Code Share. To use this code, simply apply this to a UDP Virtual Server that processes DNS traffic. (No DNS Profile necessary). Code : when FLOW_INIT { #extract QNAME from QUESTION header #${i} is a sanity check so this logic won't spin on invalid QNAMEs set i 0 #initialize our position in the QNAME parsing and the text QNAME set offset 12 set length 1 set endlength 1 set name "" #/extract QNAME from QUESTION header while {${length} > 0 && ${i} < 10} { #length contains the first part length binary scan [string range [DATAGRAM::udp payload] ${offset} ${offset}]] c foo #make the length an unsigned integer set length [expr {${foo} & 0xff}] if {${length} > 0} { #grab a part and put it in our text QNAME section append name [string range [DATAGRAM::udp payload] [expr {${offset} + 1}] [expr {${offset} + ${length}}]] #Watch the DNS QNAME get built during the loop. Remove the following line for production use. log local0.info "BUILDING DNS NAME: [IP::client_addr] queried ${name} offset ${offset} length ${length}" #grab a part and put it in our text QNAME section set offset [expr {${offset} + ${length} +1}] #endlength contains the Last part length binary scan [string range [DATAGRAM::udp payload] ${offset} ${offset}]] c foo #make the length an unsigned integer set endlength [expr {${foo} & 0xff}] if { ${endlength} > 0} { #put a dot between parts like a normal DNS name append name "." } incr i } } #/extract QNAME from QUESTION header #Input the required action here, where "${name}" is the variable that is reviewed for decision making. #Sample action would be a pool statement. The below log statement should be removed for production use. log local0.info "FINAL DNS NAME: [IP::client_addr] queried ${name}" } Tested this on version: 12.1
GTM return LDNS IP to client
Problem this snippet solves: We do a lot of our load balancing based on topology rules, so it's often very useful to know where the DNS request is actually coming from rather than just the client's IP and the DNS servers they have configured. Especially if they're behind an ADSL router doing NAT or some other similar set up. This rule simply returns the IP address of the LDNS that eventually made the query to the GTM device in the response to a lookup for the WideIP using the rule, as well as logging the response and perceived location. Code : rule "DNS_debug" partition "Common" { when DNS_REQUEST { host [IP::client_addr] log local0.err "Debug address : [IP::client_addr] [whereis [IP::client_addr]]" } }Knowledge sharing: Velos and rSeries (F5OS) basic troubleshooting, logs and commands
This another part of my Knowledge sharing articles, where I will take a deeper look into Velos and rSeries investigation of issues, logs and command. 1. Velos HA controller and blade issues. As the Velos system is the one with two controllers in active/standby mode only with Velos it could be needed to check if there is an issue with the controller's HA. As the controller's HA order can be different for the system and the different partitions to check the HA for the system use the /var/log_controller/cc-confd file or for a partition HA issue look at the partition velos log at /var/F5/partition<ID>/log/velos.log . Also you can enable HA debug for the controllers with " system dbvars config debug confd ha-state-machine true ". Overview of HA: https://support.f5.com/csp/article/K19204400 Controller HA: https://support.f5.com/csp/article/K21130014 Partition HA: https://support.f5.com/csp/article/K58515297 2. Entering into F5OS objects. The rSeries and Velos tenants are like vCMP quests with VIPRION and sometimes if there are access issues with them it could be needed to open their console. For this the "virtctl" command can be used and as an example " /usr/share/omd/kubevirt/virtctl console<tenant_name>-<tenant_instance_ID> ". Also as velos uses blades and partitions it could be needed to ssh to a blade with " ssh slot<number> " or to enter a partition with " docker exec -it partition<ID>_cli su admin " as sometimes for example to see the GUI logs entering the GUI container for the partition could be needed but F5 support will for this in most cases and maybe this will be the way to enter the BIG-IP NEXT CLI. Overview of VELOS system architecture: https://support.f5.com/csp/article/K73364432 Overview of rSeries system architecture: https://support.f5.com/csp/article/K49918625 rSeries tanant access: https://support.f5.com/csp/article/K33373310 Velos blade and tenant access: https://support.f5.com/csp/article/K65442484 Velos partition access: https://support.f5.com/csp/article/K11206563 3. Usefull commands and logs. For Velos/rSeries as this is a system with a cluster the "show cluster" command is usefull to see any issues (look fo "cluster is NOT ready."). Also the velos.log for the controller and partitions is a great place to start and debug level can be enabled for it under " SYSTEM SETTINGS Log Settings " as this is also the place for rSeries logging to be set to debug. Also the /var/log/openshift.log is good be checked with velos if there are cluster issues or or ks3.log in rSeries. Also the confd logs are like mcpd logs, so they are really usefull for Velos or rSeries. Other nice commands are docker ps, oc get pod --all-namespaces -o wide, kubectl get pod --all-namespaces -o wide but the support will ask for them in most cases. Velos cluster status: https://support.f5.com/csp/article/K27427444 Velos debug: https://support.f5.com/csp/article/K51486849 Velos openshift example issue: https://support.f5.com/csp/article/K01030619 Monitoring Velos: https://clouddocs.f5.com/training/community/velos-training/html/monitoring_velos.html Monitoring rSeries: https://clouddocs.f5.com/training/community/rseries-training/html/monitoring_rseries.html 4. Velos and rSeries tcpdumps packet captures, file utility and qkview files. For Velos qkviews ca be created for controller or partition as they are seperate qkviews. Tcpdumps for client traffic are done a tcpdump utility from the F5OS (su - admin) and a tcpdump in the Linux kernel is just for the managment ip addresses of the appliance , controller (floating or local) , partition or tenant. The file utility allows for file transfers to remote servers or even downloading any log from the Velos/rSeries to your computer as this was not possible before with iSeries or Viprion. Also the file utility starts outbound session to the remote servers so this an extra security as no inbound sessions need to be allowed on the firewall/web proxy and it can be even triggered by API call and I may make a codeshare article for this. Velos tcpdump utility: https://support.f5.com/csp/article/K12313135 rSeries tcpdump utility: https://support.f5.com/csp/article/K80685750 Qkview Velos: https://support.f5.com/csp/article/K02521182 Qkview Velos CLI location: https://support.f5.com/csp/article/K79603072 Qkview rSeries: https://support.f5.com/csp/article/K04756153 SCP: https://support.f5.com/csp/article/K34776373 5. A final fast check could be to use ''kubectl get pods -o wide--all-namespaces'' (with Velos also ''oc get pods -o wide --all-namespaces'' should also work) to see that all pods are ok and running. Also ''docker ps'' or '' docker ps --format 'table {{.Names}}\t{{.RunningFor}}\t{{.Status}}' '' are usefull to see a container that could be going down and up and this can be correlated with issues seen with "show cluster" command. 6. The new F5OS has much better hardware diagnostics than the old devices, so no more the need to do EUD tests as all system hardware components and their health can be viewed from the GUI or CLI and also this is shown in F5 ihealth! https://techdocs.f5.com/en-us/velos-1-5-0/velos-systems-administration-configuration/title-system-settings.html Edit: For Velos and rSeries always keep the software up to date as for example I will give with the Velos 1.5.1 the cluster rebuild because of the openshift ssl cert being 1 year is much simpler or the F5 rSeries and the Cisco Nexus issues or the corrupt Qkview generation when the GUI not the CLI is used (the velos cluster rebuild with touch /var/omd/CLUSTER_REINSTALL can solve many issues but it will cause some timeout): http://cdn.f5.com/product/bugtracker/ID1135853.html https://my.f5.com/manage/s/article/K000092905 https://support.f5.com/csp/article/K79603072
