BIG-IP Configuration Conversion Scripts
Kirk Bauer, John Alam, and Pete White created a handful of perl and/or python scripts aimed at easing your migration from some of the “other guys” to BIG-IP.While they aren’t going to map every nook and cranny of the configurations to a BIG-IP feature, they will get you well along the way, taking out as much of the human error element as possible.Links to the codeshare articles below. Cisco ACE (perl) Cisco ACE via tmsh (perl) Cisco ACE (python) Cisco CSS (perl) Cisco CSS via tmsh (perl) Cisco CSM (perl) Citrix Netscaler (perl) Radware via tmsh (perl) Radware (python)
BIG-IQ Central Management API automation and programmability - BULK Discovery, Import and Re Import - Perl
Summary In conjunction with the announcement of BIG-IQ 5.0 we are excited to bring the field greater flexibility when centrally managing BIG-IP devices utilizing well defined workflows and the BIG-IQ iControl REST API. We understand that automation and programmability is becoming more the norm these days as the network is evolving into a software defined application aware environment. This article is an addendum to “BIG-IQ Central Management API automation and programmability – Python” and will demonstrate bulk device trust, discovery to populate resolver groups and import bigip configuration of many BIG-IP device as defined in a csv configuration file. This automation can be run as a standalone utilities that will run directly in the BIG-IQ shell for adding devices to inventory in a sequential yet automated fashion. API Reference Trust: https://'bigiq-ip'/mgmt/cm/global/tasks/device-trust Discovery: https://'bigiq-ip'/mgmt/cm/global/tasks/device-discovey Import: ADC - https://'bigiq-ip'/mgmt/cm/adc-core/tasks/declare-mgmt-authority Import: Firewall - https://'bigiq-ip'/mgmt/cm/firewall/tasks/declare-mgmt-authority Let’s get started – When using the BIG-IQ it is suggested to make a directory called scripts under /shared and securely copy this distribution into /shared/scripts/. Contents: ../config/bulk_discovery.csv ../src/perl/bulkDiscovery.pl ../src/perl/bulkReImport.pl Everything is predicated by a main loop which will invoke each task by calling supporting perl subroutines self-contained in the script. All rest calls, using curl (https://curl.haxx.se/), made are highlighted below. Establishment of device trust is completed in the main loop while the process of discovery and import configurations are completed in subroutine blocks within the script. #====================================================== # Main loop # Process trust, discovery, and imports #====================================================== for $bigip (@bigips) { my %postBodyHash = ("address"=>$bigiq, "userName"=>$user, "password"=>$pw,"clusterName"=>"", "useBigiqSync"=>"false", "name"=>"trust_$mip"); my $postBody = encode_json(\%postBodyHash); my $trustCmd = "curl -s -k -u $bigiqCreds -H \"$contType\" -X POST -d \'$postBody\' https://localhost/mgmt/cm/global/tasks/device-trust"; if (discoverModules($bigiq, $machineId)) { if (importModules($bigiq)) { } } # upgrade the framework if nessasary if (handleFrameworkUpdade ($trustTask, $bigip)) { } } end of all devices #====================================================== # Discover specified modules. #====================================================== sub discoverModules { my %postBodyHash = ("moduleList" => \@moduleList, "status" => "STARTED"); # POST a new discovery task $postBodyHash{"deviceReference"}{"link"} = "cm/system/machineid-resolver/$machineId"; my $newDiscoverTaskCmd = "curl -s -k -u $bigiqCreds -H \"$contType\" -X POST -d \'$postBodyJson\' \"https://localhost/mgmt/cm/global/tasks/device-discovery\""; } end of discoverModules #====================================================== # A subroutine for importing individual module. #====================================================== sub importModule { # POST a new import task $postBodyHash{"deviceReference"}{"link"} = "cm/system/machineid-resolver/$machineId"; my $postBody = encode_json(\%postBodyHash); my $newImportTaskCmd = "curl -s -k -u $bigiqCreds -H \"$contType\" -X POST -d \'$postBodyJson\' \"$dmaUrl\""; # if we ecounter conflicts, we mark them to use BigIQ, patch the task back to started, and poll again if (($currentStep eq "PENDING_CONFLICTS") or ($currentStep eq "PENDING_CHILD_CONFLICTS")) if (resolveConflicts($mip, $module, $currentStep, $importSelfLink, @conflicts)) } # end of importModule #========================================================== # sub routine calling importModule to colocate all modules #========================================================== sub importModules { $ltmSuccess = importModule($mip, "ltm", "https://localhost/mgmt/cm/adc-core/tasks/declare-mgmt-authority", %postBodyHash); $asmSuccess = importModule($mip, "asm", "https://localhost/mgmt/cm/asm/tasks/declare-mgmt-authority", %postBodyHash); } And last but not least Re Import of BIGIP configuration objects for greater than one BIGIP device. This script can be run periodically based on Linux cron to ensure your device configurations managed by BIGIQ are up to date. On occasion other Element Management Systems could modify BIGIP object base and BIGIQ should be aware of these changes. If you refer to the below main loop, the discovery and import call's are the same. So two things actually happen that differs from inital bulk discovery and import. 1. Trust establishment is removed as it already contains mutaul certificate trust. 2. We test if the discovery and import tasks exists, if they do we can just PATCH discovery and import tasks to enforce a re import. That's about it. Refer to the code snippet below. #====================================================== # Main loop # Process Re Discovery, and Imports #====================================================== for $bigip (@bigips) { ## get the device properties my $deviceCmd = "curl -s -k -u $bigiqCreds -H \"$contType\" -X GET https://localhost/mgmt/shared/resolver/device-groups/cm-bigip-allBigIpDevices/devices"; ## call disc routine using ip and machine id. if (discoverModules($bigiq, $machineId)) { ## call import routine using up and machine id. if (importModules($bigiq, $machineId)) } } } # end for devices Just to re iterate the above the discovery and import routines used for Re Import just PATCH the existing task created during inital discovery and import. Here are the PATCH requests. #====================================================== # Discover specified modules. #====================================================== sub discoverModules { ## get the discovery task based on the machineId my $findDiscoverTaskCmd = "curl -s -k -u $bigiqCreds -H \"$contType\" -X GET \"https://localhost/mgmt/cm/global/tasks/device-discovery?\\\$filter=deviceReference/link+eq+\'*$machineId*\'+and+status+eq+\'FINISHED\'\""; ## If it exists PATCH the task if (defined $discoveryTask->{"items"}[0]) { # PATCH the existing discovery task my $discoveryTaskSelfLink = $discoveryTask->{"items"}[0]->{"selfLink"}; $postBodyJson = encode_json(\%postBodyHash); my $discoverCmd = "curl -s -k -u $bigiqCreds -H \"$contType\" -X PATCH -d \'$postBodyJson\' $discoveryTaskSelfLink"; } #====================================================== # A subroutine for importing individual module. #====================================================== sub importModule { ## get import task based on the machineid my $findImportTaskCmd = "curl -s -k -u $bigiqCreds -H \"$contType\" -X GET \"$dmaUrl?\\\$filter=deviceReference/link+eq+\'*$machineId*\'\""; ## If exists PATCH the task if (defined $findImportTask->{"items"}[0]) { # PATCH the existing import task $importTaskLink = $findImportTask->{"items"}[0]->{"selfLink"}; my $importCmd = "curl -s -k -u $bigiqCreds -H \"$contType\" -X PATCH -d \'$postBodyJson\' $importTaskLink"; } #======================================== # sub routine for calling importModule to collocate all modules. #======================================== sub importModules { $ltmSuccess = importModule($mip, $machineId, "ltm", "https://localhost/mgmt/cm/adc-core/tasks/declare-mgmt-authority", %postBodyHash); $asmSuccess = importModule($mip, machineId, "asm", "https://localhost/mgmt/cm/asm/tasks/declare-mgmt-authority", %postBodyHash); } If you are interested in this code for collaboration or solution, seach on key words "bigiq" "api" "python" or "perl" in code share section on dev central or here is the reference link: https://devcentral.f5.com/s/articles/big-iq-big-ip-rest-api-bulk-device-discovery-perl-972 We will also create a repository on github for easy accessability. Please visit us soon and often for periodic updates.
Adding or deleting a pool member
Problem this snippet solves: The script you need for this task must:. Identify the Enterprise Manager through which you will access the necessary BIG-IP devices. Authenticate your client to the Enterprise Manager. Retrieve a list of the BIG-IP devices managed by Enterprise Manager. For each managed BIG-IP device, retrieve a list of pools and pool members. For each managed BIG-IP device, specify the operation you want to perform (add/delete). How to use this snippet: To add/delete pool members: Create a script similar to the code sample shown below. Use either the add or delete parameter to produce the required outcome. From a command prompt, run your script. When the code finishes running, the specified pool member is enabled or disabled in each pool and on each device on which it resides. Code : #!/usr/bin/perl use SOAP::Lite; use UNIVERSAL 'isa'; $ENV{'PERL_LWP_SSL_VERIFY_HOSTNAME'} = 0; # Description: Adds/removes a pool member to/from all the pools on all BIG-IP devices managed by EM. # Usage: test-icontrolproxy-pool-member-operation.pl host uid pwd pool_member add|remove #---------------------------------------------------------------------------- # Validate Arguments #---------------------------------------------------------------------------- my $sHost = $ARGV[0]; # EM Host my $sUID = $ARGV[1]; # EM User (administrator role) my $sPWD = $ARGV[2]; # EM Password my $sPoolMember = $ARGV[3]; # BIG-IP Pool Member my $sOperation = $ARGV[4]; # BIG-IP Pool Member operation: add/remove my $proxy_uri = sprintf("https://%s:443/iControl/iControlPortal.cgi", $sHost); my $soap = SOAP::Lite->proxy($proxy_uri); sub SOAP::Transport::HTTP::Client::get_basic_credentials { return $sUID => $sPWD; } sub checkResponse { my ($resp) = (@_); die "$resp->faultcode $resp->faultstring\n" if $resp->fault; if (@{$resp->result}) { print "\tItem: $_\n" foreach (@{$resp->result}); } else { printf "\tResult: %s\n", $resp->result; } } sub getPoolList() { $soapResponse = $soap->uri("urn:iControl:LocalLB/Pool")->get_list(); &checkResponse($soapResponse); my @pool_list = @{$soapResponse->result}; return @pool_list; } sub getMemberLists() { my (@pool_list) = (@_); # Get the list of pool members for all the pools. $soapResponse = $soap->uri("urn:iControl:LocalLB/Pool")->get_member ( SOAP::Data->name(pool_names => [@pool_list]) ); &checkResponse($soapResponse); @member_lists = @{$soapResponse->result}; return @member_lists; } sub findPoolsFromMember() { my ($node_addr_port) = (@_); my ($node_addr, $node_port) = split(/:/, $node_addr_port, 2); my @pool_match_list; my @pool_list = &getPoolList(); my @member_lists = &getMemberLists(@pool_list); for $i (0 .. $#pool_list) { $pool = @pool_list[$i]; foreach $member (@{@member_lists[$i]}) { $addr = $member->{"address"}; $port = $member->{"port"}; if ( ($node_addr eq $addr) && ($node_port eq $port) ) { push @pool_match_list, $pool; } } } return @pool_match_list; } #---------------------------------------------------------------------------- # Adds or removes a pool member to/from all pools. #---------------------------------------------------------------------------- sub performPoolMemberOperation() { my ($node_addr_port, $operation) = (@_); my ($node_addr, $node_port) = split(/:/, $node_addr_port, 2); my $member = { address => $node_addr, port => $node_port }; # Validate operation. if (($operation ne "add") and ($operation ne "remove")) { print "Invalid operation: $operation..."; return; } my @pool_list; # Depending on the operation, get the appropraite pool list. if ($operation eq "add") { @pool_list = &getPoolList(); } else { @pool_list = &findPoolsFromMember($node_addr_port); } # memberList is the 1st dimension of the array, we need one for each pool push @memberList, $member; # memberLists is the 2nd dimension. push pool members for each pool here. for $i (0 .. $#pool_list) { push @memberLists, [@memberList]; } # Perform pool member operation. if ($operation eq "add") { $resp = $soap->uri("urn:iControl:LocalLB/Pool")->add_member( SOAP::Data->name(pool_names => [@pool_list]), SOAP::Data->name(members => [@memberLists])); } else { $resp = $soap->uri("urn:iControl:LocalLB/Pool")->remove_member( SOAP::Data->name(pool_names => [@pool_list]), SOAP::Data->name(members => [@memberLists])); } &checkResponse($soapResponse); print "Pool member: $node_addr_port. Operation: $operation. Affected pools: "; foreach $pool (@pool_list) { print "$pool, "; } print "\n"; } #---------------------------------------------------------------------------- # Main application entry point. #---------------------------------------------------------------------------- # EM: get devices. print "\nGet devices...\n"; my $resp = $soap->uri('urn:iControl:Management/EM')->get_devices(); my $device_list = $resp->result; # EM: generate a context ID. print "\nGenerate context ID...\n"; $resp = $soap->uri("urn:iControl:Management/EM")->get_context_id(); my $context_id = $resp->result; # Append context ID to SOAP URI. $proxy_uri = sprintf("%s?context_id=%s", $proxy_uri, $context_id); $soap = SOAP::Lite->proxy($proxy_uri); # Iterate through the device list. foreach (@{$device_list}) { # Get current device address. my $device = $_; # EM: set device context (to proxy to). print "\nSet device context...\n"; $resp = $soap->uri("urn:iControl:Management/EM")->set_device_context(SOAP::Data->name("ip_address" => $device)); &checkResponse($resp); # Perform pool member operation. &performPoolMemberOperation($sPoolMember, $sOperation); }
Perl Ltm Config To Xml/Excel (version 3.1)
Problem this snippet solves: This code is an improvement of "Perl Ltm Config To Xml (version 2)" published on previous DevCentral (https://devcentral.f5.com/s/articles/perl-ltm-config-to-xml-version-2-1009) queries the components of a virtual servers configuration and prints it out in an XML format. Is recursive through partitions and prints the following data: VS_name VS_IP: IP and port RD (route domain) VS_state: status (ex. green) and state (ex. enabled) Profiles: name, type, context. If SSL profile aditional info: certificate, chain, key, cipher list Persistence iRules: order and name Pool_name LB-method: method, minimum active members Member: name, IP, port, ratio, priority, status and state Monitors: name, interval, timeout. If HTTP/HTTPS monitors additional info: Send and Receive strings SNAT_name SNAT_members Has a new option "-s" or "-simplified" which outputs multiple values like the profile list or pool members into a unique XML field. Purpose of simplified mode is to be able to have one Virtual Server per row on Excel, with lists showing on a single cell. This code has been tested on BIG-IP version 12.1 and 14.1. How to use this snippet: Default mode: ./LTM-to-XML.pl bigip uid pwd > result.xml Example for simplified mode and removal of partition information: ./LTM-to-XML.pl -s 127.0.0.1 uid pwd | sed "s/\/.*app\///g" | sed "s/\/Common\///g" > result.xml The XML can be dragged and dropped on MS Excel and opened as an XML table. Code : (see ZIP file) Tested this on version: 12.1, 14.1
BIG-IP Configuration Visualizer
Problem this snippet solves: This is a script utilizing the Graphviz package that will create graphs of the relationships of configuration objects of a virtual server. How to use this snippet: Requirements Perl SOAP::Lite Net::Netmask Graphviz Note: There are other required packages, but the remainder are default in Ubuntu 10.10 perl distribution. This may not be the case with your configuration. Install dependent libraries via apt-get - depending on your distribution you may have all of these as packages: libssl-dev libcrypt-ssleay-perl libio-socket-ssl-perl libgraph-writer-graphviz-perl libsoap-lite-perl libnet-netmask-perl For any which aren't available, open a CPAN shell (perl -MCPAN -e shell) and install them with commands such as: install SOAP::Lite install Net::Netmask Grab the archive below, extract the perl script file, and make it executable Minimal command-line invocation is: ./f5-gl.pl -t bigip-host-name.your.domain -u userid -p password The output by default goes into a subdirectory img, but you might want to add the option -d output-directory to keep output files separate from successive runs, particularly if you will run the utility against different BIG-IP systems for different operational environments. You can also use these: -c partition -v vs to run for only a particular VS (the partition must also be given) -e to change the output file type - default is JPG, but there are the options of SVG or PNG Run it! Put it in /etc/crontab to run every night, perhaps, and output the files to a directory served by Apache? The files are predictably named, so you could link directly to them from other tools. If you are averse to command-line parameters (perhaps you don't want the password to appear in a process listing) you can instead update these variables in the script: my $ltm_host my $user_id my $user_password
Perl Ltm Config To Xml (version 2)
Problem this snippet solves: This example is based on "Perl Ltm Config To Xml" by Joe Pruitt: queries the components of a virtual servers configuration and prints it out in an XML format for further processing. Now with a few more data retrieved (SNAT, object status) and does recursively on partitions, this way listing configuration generated by iApps on separate partitions. How to use this snippet: ./LTM-to-XML.pl bigip user pass Code : #!/usr/bin/perl #---------------------------------------------------------------------------- # The contents of this file are subject to the iControl Public License # Version 4.5 (the "License"); you may not use this file except in # compliance with the License. You may obtain a copy of the License at # http://www.f5.com/. # # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. # # The Original Code is iControl Code and related documentation # distributed by F5. # # The Initial Developer of the Original Code is F5 Networks, # Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 1996-2003 F5 Networks, # Inc. All Rights Reserved. iControl (TM) is a registered trademark of F5 Networks, Inc. # # Alternatively, the contents of this file may be used under the terms # of the GNU General Public License (the "GPL"), in which case the # provisions of GPL are applicable instead of those above. If you wish # to allow use of your version of this file only under the terms of the # GPL and not to allow others to use your version of this file under the # License, indicate your decision by deleting the provisions above and # replace them with the notice and other provisions required by the GPL. # If you do not delete the provisions above, a recipient may use your # version of this file under either the License or the GPL. #---------------------------------------------------------------------------- #use SOAP::Lite + trace => qw(method debug); use SOAP::Lite; #---------------------------------------------------------------------------- # Validate Arguments #---------------------------------------------------------------------------- my $BIGIP = $ARGV[0]; my $User = $ARGV[1]; my $Pass = $ARGV[2]; sub usage() { die ("Usage: LtmConfigToXml.pl host uid pwd\n"); } if ( ($BIGIP eq "") or ($User eq "") or ($Pass eq "") ) { usage(); } #---------------------------------------------------------------------------- # Transport Information #---------------------------------------------------------------------------- sub SOAP::Transport::HTTP::Client::get_basic_credentials { return "$User" => "$Pass"; } $urnMap = { "{urn:iControl}LocalLB.LBMethod" => 1, "{urn:iControl}LocalLB.MonitorRuleType" => 1, "{urn:iControl}LocalLB.ProfileContextType" => 1, "{urn:iControl}LocalLB.ProfileType" => 1, "{urn:iControl}LocalLB.VirtualServer.VirtualServerType" => 1, "{urn:iControl}LocalLB.AvailabilityStatus" => 1, "{urn:iControl}LocalLB.MonitorStatus" => 1, "{urn:iControl}LocalLB.EnabledStatus" => 1, "{urn:iControl}Common.EnabledState" => 1, "{urn:iControl}Common.VLANFilterList" => 1, }; sub SOAP::Deserializer::typecast { my ($self, $value, $name, $attrs, $children, $type) = @_; my $retval = undef; if ( 1 == $urnMap->{$type} ) { $retval = $value; } return $retval; } sub GetInterface() { my ($module, $name) = @_; $interface = SOAP::Lite -> uri("urn:iControl:$module/$name") -> readable(1) -> proxy("https://$BIGIP/iControl/iControlPortal.cgi"); eval { $interface->transport->http_request->header ( 'Authorization' => 'Basic ' . MIME::Base64::encode("$User:$Pass", '') ); }; return $interface; } sub FixEmptyEntries() { my @list1 = @_; my @list2; my $valid_item = ""; for $i (0 .. $#list1) { $item = @list1[$i]; if ( $item ne "" ) { $valid_item = $item; break; } } for $i (0 .. $#list1) { $item = @list1[$i]; if ( $item ne "" ) { push @list2, $item; } else { push @list2, $valid_item; } } return @list2; } # Status conversion my $MONITOR_STATUS_MAP = { "MONITOR_STATUS_UNCHECKED" => "UNCHECKED", "MONITOR_STATUS_CHECKING" => "CHECKING", "MONITOR_STATUS_UP" => "UP", "MONITOR_STATUS_DOWN" => "DOWN", "MONITOR_STATUS_FORCED_DOWN" => "FORCED_DOWN", "MONITOR_STATUS_MAINT" => "MAINT", "MONITOR_STATUS_ADDRESS_DOWN" => "ADDRESS_DOWN", }; my $ENABLED_STATUS_MAP = { "ENABLED_STATUS_NONE" => "NONE", "ENABLED_STATUS_ENABLED" => "ENABLED", "ENABLED_STATUS_DISABLED" => "DISABLED", "ENABLED_STATUS_DISABLED_BY_PARENT" => "DISABLED_BY_PARENT", }; # Get config sub GetConfigXML() { # Enable Recursive Partition/Folder lookup and set /Common as active folder $SystemSession = &GetInterface("System", "Session"); $soapResponse = $SystemSession->set_recursive_query_state( SOAP::Data->name(state => "STATE_ENABLED") ); &checkResponse($soapResponse); $soapResponse = $SystemSession->set_active_folder( SOAP::Data->name(folder => "/") ); # Get data $LocalLBVirtualServer = &GetInterface("LocalLB", "VirtualServer"); $LocalLBPool = &GetInterface("LocalLB", "Pool"); $LocalLBPoolMember = &GetInterface("LocalLB", "PoolMember"); $LocalLBSNATPool = &GetInterface("LocalLB", "SNATPool"); # VS List $soapResponse = $LocalLBVirtualServer->get_list(); &checkResponse($soapResponse); @vs_list = @{$soapResponse->result}; # Destination $soapResponse = $LocalLBVirtualServer->get_destination_v2( SOAP::Data->name (virtual_servers => [@vs_list]) ); &checkResponse($soapResponse); @destination_list = @{$soapResponse->result}; # Type $soapResponse = $LocalLBVirtualServer->get_type( SOAP::Data->name (virtual_servers => [@vs_list]) ); &checkResponse($soapResponse); @vstype_list = @{$soapResponse->result}; # iRules $soapResponse = $LocalLBVirtualServer->get_rule( SOAP::Data->name (virtual_servers => [@vs_list]) ); &checkResponse($soapResponse); @rule_listA = @{$soapResponse->result}; # Profiles $soapResponse = $LocalLBVirtualServer->get_profile( SOAP::Data->name(virtual_servers => [@vs_list]) ); &checkResponse($soapResponse); @profile_listA = @{$soapResponse->result}; $soapResponse = $LocalLBVirtualServer->get_persistence_profile( SOAP::Data->name(virtual_servers => [@vs_list]) ); &checkResponse($soapResponse); @persistenceprofile_listA = @{$soapResponse->result}; # SNAT $soapResponse = $LocalLBVirtualServer->get_source_address_translation_snat_pool( SOAP::Data->name(virtual_servers => [@vs_list]) ); &checkResponse($soapResponse); @snat_list = @{$soapResponse->result}; # ensure we don't pass empty entries to the following methods. @snat_list2 = &FixEmptyEntries(@snat_list); # SNAT Pool Members $soapResponse = $LocalLBSNATPool->get_member_v2( SOAP::Data->name(snat_pools => [@snat_list2]) ); &checkResponse($soapResponse); @snatmember_listA = @{$soapResponse->result}; # State $soapResponse = $LocalLBVirtualServer->get_enabled_state( SOAP::Data->name(virtual_servers => [@vs_list]) ); &checkResponse($soapResponse); @state_list = @{$soapResponse->result}; # Pools $soapResponse = $LocalLBVirtualServer->get_default_pool_name( SOAP::Data->name (virtual_servers => [@vs_list]) ); &checkResponse($soapResponse); @pool_list = @{$soapResponse->result}; # ensure we don't pass empty entries to the following methods. @pool_list2 = &FixEmptyEntries(@pool_list); # Pool Members $soapResponse = $LocalLBPool->get_member_v2( SOAP::Data->name(pool_names => [@pool_list2]) ); &checkResponse($soapResponse); @member_listA = @{$soapResponse->result}; # Member/Monitor Status $soapResponse = $LocalLBPoolMember->get_monitor_status( SOAP::Data->name(pool_names => [@pool_list2]) ); &checkResponse($soapResponse); @monitor_statusA = @{$soapResponse->result}; $soapResponse = $LocalLBPoolMember->get_object_status( SOAP::Data->name(pool_names => [@pool_list2]) ); &checkResponse($soapResponse); @object_statusA = @{$soapResponse->result}; # LB Method $soapResponse = $LocalLBPool->get_lb_method( SOAP::Data->name(pool_names => [@pool_list2]) ); &checkResponse($soapResponse); @lbmethod_list = @{$soapResponse->result}; $soapResponse = $LocalLBPool->get_monitor_association( SOAP::Data->name(pool_names => [@pool_list2]) ); &checkResponse($soapResponse); @monitor_list = @{$soapResponse->result}; # print XML print "\n"; print " \n"; print " \n"; for $i (0 .. $#vs_list) { # vip $vip = @vs_list[$i]; $pool = @pool_list[$i]; $destination = @destination_list[$i]; $vstype = @vstype_list[$i]; $snat = @snat_list[$i]; $state = @state_list[$i]; @rule_list = @{$rule_listA[$i]}; @profile_list = @{$profile_listA[$i]}; @persistenceprofile_list = @{$persistenceprofile_listA[$i]}; # pool @member_list = @{$member_listA[$i]}; @monitor_status = @{$monitor_statusA[$i]}; @object_status = @{$object_statusA[$i]}; $monitorassociations = @monitor_list[$i]; $monitor_rule = $monitorassociations->{"monitor_rule"}; $lb_method = $lbmethod_list[$i]; # SNAT @snatmember_list = @{$snatmember_listA[$i]}; $daddr = $destination->{"address"}; $dport = $destination->{"port"}; print " \n"; for $j (0 .. $#rule_list) { $rule = @rule_list[$j]; $name = $rule->{"rule_name"}; $priority = $rule->{"priority"}; print " \n"; } for $j (0 .. $#profile_list) { $profile = @profile_list[$j]; $name = $profile->{"profile_name"}; $type = $profile->{"profile_type"}; $context = $profile->{"profile_context"}; $default = $pprofile->{"default_profile"}; print " \n"; } for $j (0 .. $#persistenceprofile_list) { $pprofile = @persistenceprofile_list[$j]; $name = $pprofile->{"profile_name"}; $default = $pprofile->{"default_profile"}; print " \n"; } if ( $pool ne "" ) { print " \n"; for $j (0 .. $#member_list) { $member = @member_list[$j]; $address = $member->{"address"}; $port = $member->{"port"}; $monitorS = @monitor_status[$j]->{"monitor_status"}; $objectS = @object_status[$j]->{"object_status"}; $enabledS = $objectS->{"enabled_status"}; $status = "$MONITOR_STATUS_MAP->{$monitorS}-$ENABLED_STATUS_MAP->{$enabledS}"; print " \n"; } $type = $monitor_rule->{"type"}; @templates = @{$monitor_rule->{"monitor_templates"}}; print " \n"; for $j (0 .. $#templates) { $template = @templates[$j]; print " \n"; } print " \n"; print " \n"; } if ( $snat ne "" ) { print " \n"; for $j (0 .. $#snatmember_list) { $snatmember = @snatmember_list[$j]; print " \n"; } print " \n"; } print " \n"; } print " \n"; print " \n"; # Set /Common as active folder $soapResponse = $SystemSession->set_active_folder( SOAP::Data->name(folder => "/") ); &checkResponse($soapResponse); } &GetConfigXML(); #---------------------------------------------------------------------------- # checkResponse #---------------------------------------------------------------------------- sub checkResponse() { my ($soapResponse) = (@_); if ( $soapResponse->fault ) { print $soapResponse->faultcode, " ", $soapResponse->faultstring, "\n"; exit(); } } Tested this on version: 12.0BIG-IP Backup Scripts
Problem this snippet solves: A set of scripts that can manage F5 configuration backups, runs on a scheduler, currently geared towards a Windows Server for remote backups. Requires SSH and passwordless key-based authentication. All files backed up are date stamped with the current day and can be used to perform a b config load and restore the appliances to a different state. There are 2 scripts as follows: f5confbkp.pl (runs on the BIG-IP). Uses b config save to save a copy under /var/tmp/f5configs f5getconf.pl (runs on a Windows server, modify the script if you want to run it on a UNIX system). Uses the key pair generated above to remote scp the files off of the F5's to a directory structure, in the scripts example that is e:\f5backups.) Key Exchange Setup Note: Only required for backup from a remote system. Generate a set of keys without a passphrase. (Reference ssh-keygen for details on generating keys). Take the id_dsa.pub key and put that into the /root/.ssh/authorized_keys file on the BIG-IP's you want to back up. Permissions on the authorized_keys file must be set to 644. Test your ssh connections from the remote system to ensure the account you are using on the remote backup server can ssh into the F5's you are backing up without getting prompted, as this will cause the script to fail otherwise. Cron job setup Setup the cronjob on the BIG-IP's. A job similar to the following will run the f5confbkp.pl script every day at 1 30 1 * /usr/bin/perl /root/scripts/f5confbkp.pl #!/usr/bin/perl # ------------------------------------ # # Name: f5getconf.pl # Purpose: Using SSH key based authentication gets the most current date stamped ucs file # Compatible: v4X and v9X versions of BigIP appliances, Windows 2k/2k3 or any Unix system # Requirements: On Windows cygwin (including ssh service installed) # SSH Key gen with no password for passwordless authentication # Requires that you connect and accept the key finger print before running # as a cron or via the Windows scheduler, otherwise it will stick trying to # connect via SCP. # Also requires an F5hosts.txt with the following entries: # bigipname (Case sensitive) IP Address. Example: # noc-f5-somehostname 10.0.1.1 # noc-f5-somehostnameb 10.0.1.2 # # Version: 2.0 # Author: Wes North # Currently geared for: Windows 2k/2k3 # ------------------------------------ # #Get LocalTime use Net::Telnet; #to test the socket before trying to SCP use Sys::Hostname; @T=localtime(); $year = $T[5] + 1900; $month = $T[4] + 1; $day = $T[3]; if ($T[2] > 12) { $hour = $T[2] - 12; $time = "$hour$T[1]PM"; } else { $hour = $T[2]; $time = "$hour$T[1]AM"; } if ($month < 10) { $month = "0$month"; } if ($day < 10) { $day = "0$day"; } $date = "$year$month$day"; #returns yyyymmdd $bakfile = "${hostname}-${date}.ucs"; $destdrive = qq!E:\\!; $destdir = qq!${destdrive}f5backups!; if (!-d $destdir) { `mkdir $destdir`; } $errdir = qq!$destdir\\errors!; if (!-d $errdir) { `mkdir $errdir`; } $errfile = "$errdir/debug.log"; open (ERROR, ">>$errfile") || die "Cannot open $errfile because $!\n"; #Get the current backup file for the F5 running config $cathosts = qq!cmd \/c type f5hosts.txt!; @f5list = split '\n', `$cathosts`; foreach $f5 (@f5list) { next if ($f5 =~ /^\#/); $errnum = ""; @sepline = split ' ', $f5; $hostname = $sepline[0]; $ip = $sepline[1]; $connection=Net::Telnet->new(Timeout => 2, Host => "$ip", port=>22, Errmode => sub {&error;}); if ($errnum ne "") { print ERROR "$month\/$day\/$year: $ip is not accessible $!\n"; next; } $bkfile = qq!${hostname}-${date}.ucs!; $putdir = qq!${destdir}\\$hostname!; if (!-d $putdir) { `mkdir $putdir`; } $getdir = qq!/var/tmp/f5configs!; $getfile = qq!${getdir}/${bkfile}!; #Replace the directory path and private key file reference with your own. $scpcmd = qq!cmd \/c scp -i "/cygdrive/c/Documents\ and\ Settings//.ssh/id_dsa" \ root\@${ip}\:${getfile} /cygdrive//f5backups/${hostname}/${bkfile}!; $Error = 0; $Error += system($scpcmd); if ($Error > 0) { print ERROR "$month\/$day\/$year: $scp command failed $!\n"; } else { print ERROR "$month\/$day\/$year: scp command for $hostname at $ip was successful\n"; } } close ERROR; #Cheesy and can be done differently, I didn't have a whole lot of time, so I settled for something like this. sub error { $errnum = 1; return $errnum; #print "Connection Failed!\n"; } exit 0; Code : #!/usr/bin/perl # ------------------------------------ # # Name: f5confbkp.pl # Purpose: without the enterprise management appliance, this can be run as a cron to backup configs from F5 appliances # Compatible: v4X and v9X versions of BigIP appliances # Requirements: Enough space to store ucs files, and really nothing else, it uses the builtin perl binaries # # Version: 2.0 # Author: Wes North # Currently geared for: Any version that supports b config save # ------------------------------------ # #Get LocalTime # use Sys::Hostname; #Locatime section, cheesy but have to set specific month and hour values @T=localtime(); $year = $T[5] + 1900; $month = $T[4] + 1; $day = $T[3]; if ($T[2] > 12) { $hour = $T[2] - 12; $time = "$hour$T[1]PM"; } else { $hour = $T[2]; $time = "$hour$T[1]AM"; } if ($month < 10) { $month = "0$month"; } if ($day < 10) { $day = "0$day"; } $date = "$year$month$day"; #returns yyyymmdd $fqdn = hostname(); #returns node.domain.suffix @n = split '\.', $fqdn; $hostname = $n[0]; #$bakfile = "${hostname}-${date}_${time}.ucs"; $bakfile = "${hostname}-${date}.ucs"; $errdir = "/var/tmp/f5configs"; if (!-d $errdir) { `mkdir $errdir`; } $errfile = "$errdir/$date-Error.log"; open (ERROR, ">>$errfile") || die "Cannot open $errfile because $!\n"; #Create a backup file for the F5 running config $bcmd = "b config save"; $path = "/var/tmp"; $f5cmd = "${bcmd} ${path}/f5configs/${bakfile} >> /dev/null 2>&1 &"; if (-f "$path/f5configs/$bakfile") { print ERROR "$month\/$day\/$year: $bakfile already exists exiting $0\n"; exit 1; } else { $possiblefile = "$path/f5configs/$hostname-*"; $ls = `ls $path/f5configs | grep .ucs`; if ($ls ne "" ) { @files = split '\n', $ls; foreach (@files) { $rm = qq!rm -rf "$path/f5configs/$_"!; `$rm`; } } $Error = 0; $Error += system($f5cmd); if ($Error != 0) { print ERROR "$month\/$day\/$year: $f5cmd command failed for $hostname\n"; exit 1; } else { print ERROR "$month\/$day\/$year: $f5cmd command completed successfully\n"; } } close ERROR; exit 0;Hosting Sorry, Error, or Maintenance Pages on BIG-IP LTM with iRules
Note Whereas you can still customize your BIG-IP with perl-scripts and sorry pages in this way, F5 has made this far easier with iFiles, introduced in 11.1. This tech tip on iFiles should be the recommended path for sorry pages going forward. I’ve posted on this before (Host that Sorry Page on your BIG-IP!) but it’s been a while and there have been a few updates. Besides, narrowing the application to only sorry pages is a bit myopic—I’m sure my BIG-IP is offended that I treated it so callously. Anyway, I got an inquiry a week or so ago about the images in tables not being picked up by the script. The images in the table were referenced as such: #<table background="genericofflinebackground.gif" align="center" width="1024" height="768" > I reached out to the author, Kirk Bauer, and he gave me some pointers as where to look. There’s a function in the perl script that parses the html to look for items of interest: sub start { my ($self, $tag, $attr, $attrseq, $origtext) = @_; # print out original text if ($tag eq 'img') { if ($attr->{'src'}) { $attr->{'src'} = &handle_object($tag, 'src', $attr->{'src'}); } } Modifying the if ($tag..) conditional to match the table wasn’t that hard at all: sub start { my ($self, $tag, $attr, $attrseq, $origtext) = @_; # print out original text if ($tag eq 'img') { if ($attr->{'src'}) { $attr->{'src'} = &handle_object($tag, 'src', $attr->{'src'}); } } if ($tag eq 'table') { if ($attr->{'background'}) { $attr->{'background'} = &handle_object($tag, 'background', $attr->{'background'}); } } That solved problem number one. The second problem with the script was that it wasn’t asking about partition preference, rather it just dumped the iRule and datagroups into the last partition defined in bigip.conf. This was strange, as the code to handle partitions was in place: my @partitions; open (CONF, "/config/bigip.conf") or die "Could not read /config/bigip.conf: $!\n"; while (my $line = <CONF>) { if ($line =~ /^partition (.+) {/) { push @partitions, $1; } } The problem is that the regex is trying to match “partition <my partition> {“ and that is (at least in 10.2.1 HF3) no longer in the bigip.conf file. It has been moved to bigip_sys.conf. Updating the code as shown below solved the issue and now the user is asked for the appropriate partition and the iRule and datagroup gets deployed as expected. my @partitions; open (CONF, "/config/bigip_sys.conf") or die "Could not read /config/bigip_sys.conf: $!\n"; while (my $line = <CONF>) { if ($line =~ /^partition (.+) {/) { push @partitions, $1; } } For the full script, head to the iRules wiki entry LTM Maintenance Page Generator and grab version 2.2. Related Articles Host that Sorry Page on your BIG-IP! about i-rule sorry page configuration - DevCentral - F5 DevCentral ... DevCentral Wiki: Automatic_maintenance_page___ Sorry_page_with_images DevCentral Wiki: Sorry Page I Rule Generator_ Perl DevCentral Wiki: LTM Maintenance Page Generator DevCentral Wiki: CodeShare Sorry Page when Severs are down - DevCentral - F5 DevCentral ... Site Dwon Page form https virtual server - DevCentral - F5 ...Getting Started with iControl Code Samples
Problem this snippet solves: This repository on github is for the full scripts highlighted in the Getting Started with iControl article series. Developed against BIG-IP TMOS version 12.0, but should work in most versions 11.6+. How to use this snippet: Choose your language and iControl portal of choice to start investigating iControl in your environment. Target audience: beginners. Code : https://github.com/f5devcentral/iControl-GettingStarted Tested this on version: 12.0Forcing a reload of External Data Groups within an iRule
This article shows how to use iControl to force an iRule to reload external configuration data when that data has changed. Background One of the more common uses of iRules is to extract part of the request (client-ip, host, uri, payload, ...) and use that as input to a policy making decision. Most often, the data you are comparing your input to (a list of blacklisted ip addresses, a list of valid hosts, uri to server mappings, etc) is too long to store directly within an iRule. Data Groups That's where Data Groups come in. Data Groups (or classes) are simply lists that can be references at the global scope from within an iRule to make the iRule itself more manageable. You can use the matchclass and findclass helper commands to lookup values in Data Groups or you could use the TCL list commands to loop through the list manually. There are four types of data groups: Address, String, Integer, and External. Address Data Groups are used to store a list of IP Addresses (and optionally subnet masks). These are often used when comparing an incoming client address against a list of valid or invalid addresses. String Data Groups store a list newline delimited strings. These are multi-purpose as you can use them for straight lookups (valid requested uri or hostname) or for building mapping (this uri gets routed to that pool). Given a token delimiter, the findclass command can be used to look for the first token in the string and return the second token. Integer Data Groups store a list of integers. A common use for this type of data group is to compare an ID passed in as part of the URI to a valid value. While this could be accomplished with a string data group, integer data groups require less resources and comparisons are faster. External Data Groups are lists that are stored externally the data base (on the file system). These can be of any of the other three types (Address, String, and Integer). A common use for this type of Data Group is when you are storing thousands of values and those values change often. It is much easier to manage an external file via a text editor, or a remote API like iControl. The Problem The iRules engine utilizes a caching mechanism for external Data Groups. The problem is that when a change is made to an External Data Group, The iRules engine has no way of knowning that the changes have actually been made. Even a configuration reload will not do it as a reload is just an overwrite on the current configuration and since the iRule didnt' change the internal caching doesn't get invalidated and the next time the iRule is processed it still reads the cached values in the Data Group. The Solution Fear not, there is a fairly simple solution. If you invalidate the iRule itself by changing it, the cache will be invalidated and the External Data Group will be reloaded. How do you do this? Well, we've found that the easiest way is to add a space to the end of the iRule. This will not change the behaviour of the iRule but will cause it's checksum to change. Just add a space to the end, save the iRule, then delete the space, and then save the iRule again! This can be accomplished via the GUI, CLI, or iControl. The GUI and CLI are fiarly intuitive, so I'll show you how to use iControl to do this. More than likely for large External Data Groups, you are using iControl already to remotely add/remove items from the lists. If that is the case, it is a very simple addition to your application to add this logic. If you are not using iControl to manage your External Data Group, then you can plop this perl script right on your BIG-IP and use it from the command line. Either way, we've got you covered. Here's an iControl application written in Perl that will invalidate an iRule to force reloading of Data Groups. #!/usr/bin/perl #---------------------------------------------------------------------------- # The contents of this file are subject to the "END USER LICENSE AGREEMENT FOR F5 # Software Development Kit for iControl"; you may not use this file except in # compliance with the License. The License is included in the iControl # Software Development Kit. # # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. # # The Original Code is iControl Code and related documentation # distributed by F5. # # The Initial Developer of the Original Code is F5 Networks, # Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 1996-2006 F5 Networks, # Inc. All Rights Reserved. iControl (TM) is a registered trademark of F5 Networks, Inc. # # Alternatively, the contents of this file may be used under the terms # of the GNU General Public License (the "GPL"), in which case the # provisions of GPL are applicable instead of those above. If you wish # to allow use of your version of this file only under the terms of the # GPL and not to allow others to use your version of this file under the # License, indicate your decision by deleting the provisions above and # replace them with the notice and other provisions required by the GPL. # If you do not delete the provisions above, a recipient may use your # version of this file under either the License or the GPL. #---------------------------------------------------------------------------- #use SOAP::Lite + trace => qw(method debug); use SOAP::Lite; #---------------------------------------------------------------------------- # Variables #---------------------------------------------------------------------------- my $DEBUG = 0; my $sHost = "localhost"; my $sPort = 80; my $sUID = ""; my $sPWD = ""; my $sProtocol = "http"; my $sRuleName = "MSM_reputation"; #---------------------------------------------------------------------------- # Transport Information #---------------------------------------------------------------------------- sub SOAP::Transport::HTTP::Client::get_basic_credentials { return "$sUID" => "$sPWD"; } #---------------------------------------------------------------------------- # iControl interface declarations #---------------------------------------------------------------------------- $LocalLBRule = SOAP::Lite -> uri('urn:iControl:LocalLB/Rule') -> readable(1) -> proxy("$sProtocol://$sHost:$sPort/iControl/iControlPortal.cgi"); #---------------------------------------------------------------------------- # Main logic #---------------------------------------------------------------------------- &invalidateRule(); #---------------------------------------------------------------------------- # sub invalidateRule # # This iRule will query the contents of an iRule, save it with an added space # and then re-save it with the original contents. #---------------------------------------------------------------------------- sub invalidateRule() { if ( $DEBUG ) { print "Querying rule definition for rule '$sRuleName'\n"; } # Query the rule definition $soapResponse = $LocalLBRule->query_rule(SOAP::Data->name(rule_names => [$sRuleName])); &checkResponse($soapResponse); @RuleDefinitionList = @{$soapResponse->result}; # Extract the first element in the array $RuleDefinition = @RuleDefinitionList[0]; # save the original and temporary changed values $rule_definition = $RuleDefinition->{"rule_definition"}; $temp_rule_definition = "$rule_definition "; # update RuleDefinition with the temporary rule details if ( $DEBUG ) { print "modifying with temporary changes...\n"; } $RuleDefinition->{"rule_definition"} = $temp_rule_definition; $soapResponse = $LocalLBRule->modify_rule(SOAP::Data->name(rules => [$RuleDefinition])); &checkResponse($soapResponse); # update RuleDefinition with the original rule details. if ( $DEBUG ) { print "modifying with reverted changes...\n"; } $RuleDefinition->{"rule_definition"} = $rule_definition; $soapResponse = $LocalLBRule->modify_rule(SOAP::Data->name(rules => [$RuleDefinition])); &checkResponse($soapResponse); if ( $DEBUG ) { print "rule initialized...\n"; } } #---------------------------------------------------------------------------- # sub checkResponse # Exit on SOAP Fault. #---------------------------------------------------------------------------- sub checkResponse() { my ($soapResponse) = (@_); if ( $soapResponse->fault ) { print $soapResponse->faultcode, " ", $soapResponse->faultstring, "\n"; exit(); } }
