BIG-IP Configuration Conversion Scripts
Kirk Bauer, John Alam, and Pete White created a handful of perl and/or python scripts aimed at easing your migration from some of the “other guys” to BIG-IP.While they aren’t going to map every nook and cranny of the configurations to a BIG-IP feature, they will get you well along the way, taking out as much of the human error element as possible.Links to the codeshare articles below. Cisco ACE (perl) Cisco ACE via tmsh (perl) Cisco ACE (python) Cisco CSS (perl) Cisco CSS via tmsh (perl) Cisco CSM (perl) Citrix Netscaler (perl) Radware via tmsh (perl) Radware (python)
A Brief Introduction To External Application Verification Monitors
Background EAVs (External Application Verification) monitors are one of most useful and extensible features of the BIG-IP product line. They give the end user the ability to utilize the underlying Linux operating system to perform complex and thorough service checks. Given a service that does not have a monitor provided, a lot of users will assign the closest related monitor and consider the solution complete. There are more than a few cases where a TCP or UDP monitor will mark a service “up” even while the service is unresponsive. EAVs give us the ability to dive much deeper than merely performing a 3-way handshake and neglecting the other layers of the application or service. How EAVs Work An EAV monitor is an executable script located on the BIG-IP’s file system (usually under /usr/bin/monitors) that is executed at regular intervals by the bigd daemon and reports its status. One of the most common misconceptions (especially amongst those with *nix backgrounds) is that the exit status of the script dictates the fate of the pool member. The exit status has nothing to do with how bigd interprets the pool member’s health. Any output to stdout (standard output) from the script will mark the pool member “up”. This is a nuance that should receive special attention when architecting your next EAV. Analyze each line of your script and make sure nothing will inadvertently get directed to stdout during monitor execution. The most common example is when someone writes a script that echoes “up” when the checks execute correctly and “down” when they fail. The pool member will be enabled by the BIG-IP under both circumstances rendering a useless monitor. Bigd automatically provides two arguments to the EAV’s script upon execution: node IP address and node port number. The node IP address is provided with an IPv6 prefix that may need to be removed in order for the script to function correctly. You’ll notice we remove the “::ffff://” prefix with a sed substitution in the example below. Other arguments can be provided to the script when configured in the UI (or command line). The user-provided arguments will have offsets of $3, $4, etc. Without further ado, let’s take a look at a service-specific monitor that gives us a more complete view of the application’s health. An Example I have seen on more than one occasion where a DNS pool member has successfully passed the TCP monitor, but the DNS service was unresponsive. As a result, a more invasive inspection is required to make sure that the DNS service is in fact serving valid responses. Let’s take a look at an example: #!/bin/bash # $1 = node IP # $2 = node port # $3 = hostname to resolve [[ $# != 3 ]] && logger -p local0.error -t ${0##*/} -- "usage: ${0##*/} <node IP> <node port> <hostname to resolve>" && exit 1 node_ip=$(echo $1 | sed 's/::ffff://') dig +short @$node_ip $3 IN A &> /dev/null [[ $? == 0 ]] && echo “UP” We are using the dig (Domain Information Groper) command to query our DNS server for an A record. We use the exit status from dig to determine if the monitor will pass. Notice how the script will never output anything to stdout other than “UP” in the case of success. If there aren’t enough arguments for the script to proceed, we output the usage to /var/log/ltm and exit. This is a very simple 13 line script, but effective example. The Takeaways The command should be as lightweight and efficient as possible If the same result can be accomplished with a built-in monitor, use it EAV monitors don’t rely on the command’s exit status, only standard output Send all error and informational messages to logger instead of stdout or stderr (standard error) “UP” has no significance, it is just a series of character sent to stdout, the monitor would still pass if the script echoed “DOWN” Conclusion When I first discovered EAV monitors, it opened up a whole realm of possibilities that I could not accomplish with built in monitors. It gives you the ability to do more thorough checking as well as place logic in your monitors. While my example was a simple bash script, BIG-IP also ships with Perl and Python along with their standard libraries, which offer endless possibilities. In addition to using the built-in commands and libraries, it would be just as easy to write a monitor in a compiled language (C, C++, or whatever your flavor may be) and statically compile it before uploading it to the BIG-IP. If you are new to EAVs, I hope this gives you the tools to make your environments more robust and resilient. If you’re more of a seasoned veteran, we’ll have more fun examples in the near future.
Migration projects - how to avoid IP conflicts
Hi, Wonder if there is smarter/easier way to avoid IP conflicts during migrations - in the phase when production and new service should listen on the same IP. Scenario: All VIPs in 192.168.1.0/24 subnet All traffic to VIPs is coming via external router (no clients in 192.168.1.0/24 subnet) Production device IP: 192.168.1.254 Production VIP: 192.168.1.100 New device IP: 192.168.1.253 New VIP: 192.168.1.100 Traffic from any client except test station (192.168.10.100) should hit VIP at production device BIG-IP setup: Floating IP: 192.168.1.253 VIP: 192.168.1.100; ARP disabled External router setup: Route: From 192.168.10.100 to 192.168.1.100/32 gw 192.168.1.253 One important note: virtual-address object for VS has to be created in advance via tmsh. For example using tmsh load sys config from-terminal merge and similar config: ltm virtual-address 192.168.1.100 { address 192.168.1.100 arp disabled mask 255.255.255.255 traffic-group traffic-group-1 } or of course any other suitable way. Reason for that is simple - auto created virtual-address objects (created when VS is created) always has ARP enabled. After finishing testing all virtual-address objects can be updated with ARP enabled using simple bash script like below: !/bin/sh $1 contains source file with VIP to place in array $2 contains enable or disable to turn on and off ARP fro VIP if [ -z "$1" ] then bad arguments - quit echo "Syntax: vip_arp_enable-disable_from-file.sh " else mapfile -t myArray < $1 local count = 0 for vip in "${myArray[@]}" do echo "Vip is: $vip"; tmsh modify ltm virtual-address $vip arp $2; ((++count)); done echo "$count processed" fi With above script it's possible to both enable and disable ARP, file with list of virtual-addresses to be processed. Result: Traffic from any client (except sourced from 192.168.10.100) is just send to 192.168.1.100 based on MAC in ARP Reply send to 192.168.1.0/24 subnet by router. BIG-IP never responds to ARP Request for 192.168.1.100 (ARP disabled on this VIP) Traffic from sourced from 192.168.10.100 is send to 192.168.1.253 (next hop, using 192.168.1.253 MAC as target and 192.168.1.100 as target IP), then internally BIG-IP is able to route this packet to configured VS with 192.168.1.100. Tested and working, but maybe not optimal approach? What I am afraid is if ARP cache on router will not be issue - like when production traffic is routed MAC of production VIP is cached, then when test traffic is processed (to the same IP as production) this cached entry will be used - traffic will reach production instead of test VIP - never happened in my lab but it's not 100% confirmation it would not fail. Router simulated using other BIG IP with two VSs: Wildcard (Forwarding IP) accepting traffic to subnet 192.168.1.0/24 Wildcard (PerformanceL4): Source Address: 192.168.10.100/32 Destination Address/Mask: 192.168.1.0/24 All ports All protocols Address Translation and Port Translation: disabled Pool with Pool Member: 192.168.1.253 (Floating IP of other BIG-IP)
Migration projects - how to avoid IP conflicts
Hi, Wonder if there is smarter/easier way to avoid IP conflicts during migrations - in the phase when production and new service should listen on the same IP. Scenario: All VIPs in 192.168.1.0/24 subnet All traffic to VIPs is coming via external router (no clients in 192.168.1.0/24 subnet) Production device IP: 192.168.1.254 Production VIP: 192.168.1.100 New device IP: 192.168.1.253 New VIP: 192.168.1.100 Traffic from any client except test station (192.168.10.100) should hit VIP at production device BIG-IP setup: Floating IP: 192.168.1.253 VIP: 192.168.1.100; ARP disabled External router setup: Route: From 192.168.10.100 to 192.168.1.100/32 gw 192.168.1.253 One important note: virtual-address object for VS has to be created in advance via tmsh. For example using tmsh load sys config from-terminal merge and similar config: ltm virtual-address 192.168.1.100 { address 192.168.1.100 arp disabled mask 255.255.255.255 traffic-group traffic-group-1 } or of course any other suitable way. Reason for that is simple - auto created virtual-address objects (created when VS is created) always has ARP enabled. After finishing testing all virtual-address objects can be updated with ARP enabled using simple bash script like below: !/bin/sh $1 contains source file with VIP to place in array $2 contains enable or disable to turn on and off ARP fro VIP if [ -z "$1" ] then bad arguments - quit echo "Syntax: vip_arp_enable-disable_from-file.sh " else mapfile -t myArray < $1 local count = 0 for vip in "${myArray[@]}" do echo "Vip is: $vip"; tmsh modify ltm virtual-address $vip arp $2; ((++count)); done echo "$count processed" fi With above script it's possible to both enable and disable ARP, file with list of virtual-addresses to be processed. Result: Traffic from any client (except sourced from 192.168.10.100) is just send to 192.168.1.100 based on MAC in ARP Reply send to 192.168.1.0/24 subnet by router. BIG-IP never responds to ARP Request for 192.168.1.100 (ARP disabled on this VIP) Traffic from sourced from 192.168.10.100 is send to 192.168.1.253 (next hop, using 192.168.1.253 MAC as target and 192.168.1.100 as target IP), then internally BIG-IP is able to route this packet to configured VS with 192.168.1.100. Tested and working, but maybe not optimal approach? What I am afraid is if ARP cache on router will not be issue - like when production traffic is routed MAC of production VIP is cached, then when test traffic is processed (to the same IP as production) this cached entry will be used - traffic will reach production instead of test VIP - never happened in my lab but it's not 100% confirmation it would not fail. Router simulated using other BIG IP with two VSs: Wildcard (Forwarding IP) accepting traffic to subnet 192.168.1.0/24 Wildcard (PerformanceL4): Source Address: 192.168.10.100/32 Destination Address/Mask: 192.168.1.0/24 All ports All protocols Address Translation and Port Translation: disabled Pool with Pool Member: 192.168.1.253 (Floating IP of other BIG-IP)
Convert HTTP Class iRules for version 11.5.1 or later
Hello Folks, I had a customer who is using the old version of iRules, which contains HTTP Class selected commands, and now they want to upgrade to 11.5.1. The command is no longer supporting in that version and I am not able to convert it to compatible way to accept that iRule in 11.5.1 Sample iRule from the customer setup is as following. ltm rule /Common/Ems_staging_admins { when HTTP_REQUEST { if { [class match [IP::client_addr] equals "disable_asm_ip_ranges"] }{ HTTP::class select /Common/cls_EMS_staging_admin } else { HTTP::class select /Common/cls_EMS_staging } } } ltm rule /Common/efax_smtp_allow { when CLIENT_ACCEPTED { if { [class match [IP::client_addr] equals "allowed_ip_efax"] }{ log local0. "allowed to relay from [IP::client_addr]" } else { drop log local0. "Not allowed to relay from [IP::client_addr]" } } There are 2 iRules, and many more which needs to get edited. Can anyone shed some light on how can I modify the iRule in a way where it can be applied on 11.5.1 I have tried to follow the article, however I couldn't fix it. Thank you, DarshanHTTP Class to iRule Migration for 11.4.0 or later.
Hello Folks, I am working on a migration project for one of the customers who is running 11.2.1 HF8 currently and upgrading to 11.6.0. I have converted most of the HTTP Classes/iRule for 11.6.0 however there is one iRule is causing trouble. Current HTTP Class looks as following. paths { /geoportal(/.*)\? /arcgis(/.*)\? /directories(/.*)\? /mapviewer(/.*)\? /mapviewerar(/.*)\? /mapviewerbeta(/.*)\? /geoportalbeta(/.*)\? /proxy(/.*)\? /arcgis_js_api(/.*)\? /ogc(/.*)\? /tokenservice(/.*)\? /rest(/.*)\? /OntologyService(/.*)\? /services(/.*)\? } pool /Common/SDI_Pool_8085 Since the HTTP Path needs to be defined which matches the regex, I couldn't make further progress. Can anyone help me to configure an iRule to achieve the above? Any help will be highly appreciated. Thanks, DarshanDHCP Relay Virtual Server
BIG-IP LTM version 11.1 introduces the DHCP Relay Virtual Server. Previously, it was possible to forward the requests with a set of extensive iRules that probed deeply into the ways of binary, but with the new virtual server style, it is trivial. How DHCP Works DHCP is defined in RFC 2131 and RFC 2132 for clients and servers, as well as RFC 1542 for relay agents. The basic (successful) operation of a DHCP transaction between client and server is shown below. A client issues a broadcast in the DHCP Discover, one or more DHCP servers respond with an offer, the client responds with the binding IP address, and the server acknowledges. A DHCP Relay comes into play when a network grows beyond a handful of subnets and centralized control is desired. Because a DHCP Discover is a broadcast packet, it would never reach a centralized server as the packet would never cross the broadcast domain into another segment. So the job of a relay is to take that broadcast and package it as a unicast request and send on to the defined dhcp servers. Consider the test lab below: I have two dhcp servers configured on one side of a BIG-IP LTM VE, and a client configured for dhcp on the other. With no configuration on the LTM, the LTM receives the broadcast, but does nothing with it: 09:37:11.596823 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:0c:29:99:0c:30, length: 300 09:37:14.689826 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:0c:29:99:0c:30, length: 300 09:37:20.522498 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:0c:29:99:0c:30, length: 300 09:37:27.609915 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:0c:29:99:0c:30, length: 300 09:37:42.846379 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:0c:29:99:0c:30, length: 300 Creating the Configuration The configuration is very simple. Create a pool of your dhcp servers, assigning IP and port as appropriate (port 67 for IPv4, port 547 for IPv6). The LB algorithm doesn’t matter, as all servers will receive the request. The virtual server configuration is equally simple. Name it, select the type as DHCP Relay, and then choose the IPv4 or IPv6 destination. Also, define the vlans this virtual should listen on. In my case, net106 where my dhcp client resides. Now, a dhcp discover from my client is forwarded as expected to my dhcp servers: 08:57:30.176648 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:0c:29:99:0c:30, length: 300 08:57:30.176766 IP 192.168.106.5.bootps > 192.168.40.102.bootps: BOOTP/DHCP, Request from 00:0c:29:99:0c:30, length: 300 08:57:30.176771 IP 192.168.106.5.bootps > 192.168.40.103.bootps: BOOTP/DHCP, Request from 00:0c:29:99:0c:30, length: 300 Caveats In a chained configuration where there are multiple BIG-IP LTM’s between client and server, it will be necessary to preserve the source of the originating relay agent (the self IP of the first BIG-IP LTM receiving the broadcast). This is accomplished with a no-translate snat address: ltm snat dhcp-no-translate { origins { 192.168.106.5/32 { } } translation /Common/192.168.106.5 } as well as a now-unicast dhcp relay on the second BIG-IP LTM as shown in the diagram below For dhcp lease renewal, which is unicast, a forwarding virtual server should be configured (0.0.0.0:67/0.0.0.0) and a no-translate snat should be in place as well. Note that this is an optional parameter. If it is unsuccessful, the client will revert to broadcast. A request for enhancement that would include the no-translate and dhcp renewal configuration as part of the dhcp relay virtual server type selection has been submitted for consideration for future versions.
Migrating older F5 BIG-IP has 3 partitions & Route Domains (RD) to new F5 BIG-IP with 2 partion & RD
Hi everyone, I have an old appliance with 3 partitions and each partition has its own route domain. I want to terminate one of the partitions in the new appliance due to a change in the design. Is there any way to remove the partition before moving it to the new appliances? Rather than migrating the configuration to the new appliance and starting deleting the configuration for a specific partition.Writing to and rotating custom log files
Sometimes I need to log information from iRules to debug something. So I add a simple log statement, like this: when HTTP_REQUEST { if { [HTTP::uri] equals "/secure" } { log local0. "[IP::remote_addr] attempted to access /secure" } } This is fine, but it clutters up the /var/log/ltm log file. Ideally I want to log this information into a separate log file. To accomplish this, I first change the log statement to incorporate a custom string - I chose the string "##": when HTTP_REQUEST { if { [HTTP::uri] equals "/secure" } { log local0. "##[IP::remote_addr] attempted to access /secure" } } Now I have to customize syslog to catch this string, and send it somewhere other than /var/log/ltm. I do this by customizing syslog with an include statement: tmsh modify sys syslog include '" filter f_local0 { facility(local0) and not match(\": ##\"); }; filter f_local0_customlog { facility(local0) and match(\": ##\"); }; destination d_customlog { file(\"/var/log/customlog\" create_dirs(yes)); }; log { source(local); filter(f_local0_customlog); destination(d_customlog); }; "' save the configuration change: tmsh save / sys config and restarting the syslog-ng service: tmsh restart sys service syslog-ng The included "f_local0" filter overrides the built-in "f_local0" syslog-ng filter, since the include statement will be the last one to load. The "not match" statement is regex which will prevent any statement containing a “##” string from being written to the /var/log/ltm log. The next filter,"f_local0_customlog", catches the "##" log statement and the remaining include statements handle the job of sending them to a new destination which is a file I chose to name "/var/log/customlog". You may be asking yourself why I chose to match the string ": ##" instead of just "##". It turns out that specifying just "##" also catches AUDIT log entries which (in my configuration) are written every time an iRule with the string "##" is modified. But only the log statement from the actual iRule itself will contain the ": ##" string. This slight tweak keeps those two entries separated from each other. So now I have a way to force my iRule logging statements to a custom log file. This is great, but how do I incorporate this custom log file into the log rotation scheme like most other log files? The answer is with a logrotate include statement: tmsh modify sys log-rotate syslog-include '" /var/log/customlog { compress missingok notifempty }"' and save the configuration change: tmsh save / sys config Logrotate is kicked off by cron, and the change should get picked up the next time it is scheduled to run. And that's it. I now have a way to force iRule log statements to a custom log file which is rotated just like every other log file. It’s important to note that you must save the configuration with "tmsh save / sys config" whenever you execute an include statement. If you don't, your changes will be lost then next time your configuration is loaded. That's why I think this solution is so great - it's visible in the bigip_sys.conf file -not like customizing configuration files directly. And it's portable.
