TLS server_name extension based routing without clientssl profile
Problem this snippet solves: Some configuration requires to not decrypt SSL traffic on F5 appliances to select pool based on HTTP Host header. I found a useful irule and this code keeps the structure and most of binary commands of it. I'm not sure if the first author was Kevin Stewart or Colin Walker. thanks both of them to have provided such code. I worked to understand it reading TLS 1.2 RFC 5246 and TLS 1.3 draft-23 and provided some enhancements and following description with irule variables references. According to TLS 1.3 draft-23, this code will still be valid with next TLS version. the following network diagram shows one use cases where this code will help. This diagram show how this code works based on the tls_servername_routing_dg Datagroup values and detected server name and TLS versions detected in the CLIENT_HELLO packet. For performances reasons, only the first TCP data packet is analyzed. Versions : 1.1 : Updated to support TLS version detection and SSL offload feature. (05/03/2018) 1.2 : Updated to support TLS Handshake Failure Messages instead of reject. (09/03/2018) 1.3 : Updated to support node forwarding, logs only for debug (disabled with static variable), and changed the Datagroup name to tls_servername_routing_dg . (16/03/2018) 1.4 : Added 16K handshake length limit defined in RFC 1.2 in variable payload. (13/04/2018) 1.5 : Added supported version extension recursion, to bypass unknown TLS version if a known and allowed version is in the list. This correct an issue with Google chrome which include not documented TLS version on top of the list. (30/04/2018) How to use this snippet: create a virtual server with following configuration: type : Standard SSL Profile (client) : Only if you want to enable SSL offload for some pools irule : code bellow create all objects used in following datagroup (virtual servers, pools) create a data-group named tls_servername_routing_dg. if you want to forward to pool, add the value pool NameOfPool if you want to forward to pool and enable SSL Offload (ClientSSL profile must be enabled on virtual server), add the value pool NameOfPool ssl_offload if you want to forward to virtual server, add the value virtual NameOfVirtual if you want to forward to an IP address, add the value node IPOfServer , backend server will not be translated if you want to reject the connection with RFC compliant handshake_failure message, add the value handshake_failure if you want to reject the connection, add the value reject if you want to drop the connection, add the value drop The default value keyword is search if there is no TLS server name extension or if TLS server name extension is not found in the data group. here is an example: ltm data-group internal tls_servername_routing_dg { records { app1.company.com { data "virtual vs_app1.company.com" } app2.company.com { data "pool p_app2" } app3.company.com { data "pool p_app3 ssl_offload" } app4.company.com { reject } default { data "handshake_failure" } } type string } Code : when RULE_INIT { set static::sni_routing_debug 0 } when CLIENT_ACCEPTED { if { [PROFILE::exists clientssl] } { # We have a clientssl profile attached to this VIP but we need # to find an SNI record in the client handshake. To do so, we'll # disable SSL processing and collect the initial TCP payload. set ssldisable "SSL::disable" set sslenable "SSL::enable" eval $ssldisable } TCP::collect set default_pool [LB::server pool] set tls_servername "" set tls_handshake_prefered_version "0000" } when CLIENT_DATA { # Store TCP Payload up to 2^14 + 5 bytes (Handshake length is up to 2^14) set payload [TCP::payload 16389] set payloadlen [TCP::payload length] # - Record layer content-type (1 byte) --> variable tls_record_content_type # Handshake value is 22 (required for CLIENT_HELLO packet) # - SSLv3 / TLS version. (2 byte) --> variable tls_version # SSLv3 value is 0x0300 (doesn't support SNI, not valid in first condition) # TLS_1.0 value is 0x0301 # TLS_1.1 value is 0x0302, 0x0301 in CLIENT_HELLO handskake packet for backward compatibility (not specified in RFC, that's why the value 0x0302 is allowed in condition) # TLS_1.2 value is 0x0303, 0x0301 in CLIENT_HELLO handskake packet for backward compatibility (not specified in RFC, that's why the value 0x0303 is allowed in condition) # TLS_1.3 value is 0x0304, 0x0301 in CLIENT_HELLO handskake packet for backward compatibility (explicitly specified in RFC) # TLS_1.3 drafts values are 0x7FXX (XX is the hexadecimal encoded draft version), 0x0301 in CLIENT_HELLO handskake packet for backward compatibility (explicitly specified in RFC) # - Record layer content length (2 bytes) : must match payload length --> variable tls_recordlen # - TLS Hanshake protocol (length defined by Record layer content length value) # - Handshake action (1 byte) : CLIENT_HELLO = 1 --> variable tls_handshake_action # - handshake length (3 bytes) # - SSL / TLS handshake version (2 byte) # In TLS 1.3 CLIENT_HELLO handskake packet, TLS hanshake version is sent whith 0303 (TLS 1.2) version for backward compatibility. a new TLS extension add version negociation. # - hanshake random (32 bytes) # - handshake sessionID length (1 byte) --> variable tls_handshake_sessidlen # - handshake sessionID (length defined by sessionID length value, max 32-bit) # - CipherSuites length (2 bytes) --> variable tls_ciphlen # - CipherSuites (length defined by CipherSuites length value) # - Compression length (2 bytes) --> variable tls_complen # - Compression methods (length defined by Compression length value) # - Extensions # - Extension length (2 bytes) --> variable tls_extension_length # - list of Extensions records (length defined by extension length value) # - extension record type (2 bytes) : server_name = 0, supported_versions = 43--> variable tls_extension_type # - extension record length (2 bytes) --> variable tls_extension_record_length # - extension data (length defined by extension record length value) # # TLS server_name extension data format: # - SNI record length (2 bytes) # - SNI record data (length defined by SNI record length value) # - SNI record type (1 byte) # - SNI record value length (2 bytes) # - SNI record value (length defined by SNI record value length value) --> variable tls_servername # # TLS supported_version extension data format (added in TLS 1.3): # - supported version length (1 bytes) --> variable tls_supported_versions_length # - List of supported versions (2 bytes per version) --> variable tls_supported_versions # If valid TLS 1.X CLIENT_HELLO handshake packet if { [binary scan $payload cH4Scx3H4x32c tls_record_content_type tls_version tls_recordlen tls_handshake_action tls_handshake_version tls_handshake_sessidlen] == 6 && \ ($tls_record_content_type == 22) && \ ([string match {030[1-3]} $tls_version]) && \ ($tls_handshake_action == 1) && \ ($payloadlen == $tls_recordlen+5)} { # store in a variable the handshake version set tls_handshake_prefered_version $tls_handshake_version # skip past the session id set record_offset [expr {44 + $tls_handshake_sessidlen}] # skip past the cipher list binary scan $payload @${record_offset}S tls_ciphlen set record_offset [expr {$record_offset + 2 + $tls_ciphlen}] # skip past the compression list binary scan $payload @${record_offset}c tls_complen set record_offset [expr {$record_offset + 1 + $tls_complen}] # check for the existence of ssl extensions if { ($payloadlen > $record_offset) } { # skip to the start of the first extension binary scan $payload @${record_offset}S tls_extension_length set record_offset [expr {$record_offset + 2}] # Check if extension length + offset equals payload length if {$record_offset + $tls_extension_length == $payloadlen} { # for each extension while { $record_offset < $payloadlen } { binary scan $payload @${record_offset}SS tls_extension_type tls_extension_record_length if { $tls_extension_type == 0 } { # if it's a servername extension read the servername # SNI record value start after extension type (2 bytes), extension record length (2 bytes), record type (2 bytes), record type (1 byte), record value length (2 bytes) = 9 bytes binary scan $payload @[expr {$record_offset + 9}]A[expr {$tls_extension_record_length - 5}] tls_servername set record_offset [expr {$record_offset + $tls_extension_record_length + 4}] } elseif { $tls_extension_type == 43 } { # if it's a supported_version extension (starting with TLS 1.3), extract supported version in a list binary scan $payload @[expr {${record_offset} + 4}]cS[expr {($tls_extension_record_length -1)/2}] tls_supported_versions_length tls_supported_versions set tls_handshake_prefered_version [list] foreach version $tls_supported_versions { lappend tls_handshake_prefered_version [format %04X [expr { $version & 0xffff }] ] } if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : prefered version list : $tls_handshake_prefered_version"} set record_offset [expr {$record_offset + $tls_extension_record_length + 4}] } else { # skip over other extensions set record_offset [expr {$record_offset + $tls_extension_record_length + 4}] } } } } } elseif { [binary scan $payload cH4 ssl_record_content_type ssl_version] == 2 && \ ($tls_record_content_type == 22) && \ ($tls_version == 0300)} { # SSLv3 detected set tls_handshake_prefered_version "0300" } elseif { [binary scan $payload H2x1H2 ssl_version handshake_protocol_message] == 2 && \ ($ssl_version == 80) && \ ($handshake_protocol_message == 01)} { # SSLv2 detected set tls_handshake_prefered_version "0200" } unset -nocomplain payload payloadlen tls_record_content_type tls_recordlen tls_handshake_action tls_handshake_sessidlen record_offset tls_ciphlen tls_complen tls_extension_length tls_extension_type tls_extension_record_length tls_supported_versions_length tls_supported_versions foreach version $tls_handshake_prefered_version { switch -glob -- $version { "0200" { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : SSLv2 ; connection is rejected"} reject return } "0300" - "0301" { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : SSL/TLS ; connection is rejected (0x$version)"} # Handshake Failure packet format: # # - Record layer content-type (1 byte) --> variable tls_record_content_type # Alert value is 21 (required for Handshake Failure packet) # - SSLv3 / TLS version. (2 bytes) --> from variable tls_version # - Record layer content length (2 bytes) : value is 2 for Alert message # - TLS Message (length defined by Record layer content length value) # - Level (1 byte) : value is 2 (fatal) # - Description (1 bytes) : value is 40 (Handshake Failure) TCP::respond [binary format cH4Scc 21 $tls_version 2 2 40] after 10 TCP::close #drop #reject return } "030[2-9]" - "7F[0-9A-F][0-9A-F]" { # TLS version allowed, do nothing break } "0000" { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : No SSL/TLS protocol detected ; connection is rejected (0x$version)"} reject return } default { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : Unknown CLIENT_HELLO TLS handshake prefered version : 0x$version"} } } } if { $tls_servername equals "" || ([set sni_dg_value [class match -value [string tolower $tls_servername] equals tls_servername_routing_dg]] equals "")} { set sni_dg_value [class match -value "default" equals tls_servername_routing_dg] } switch [lindex $sni_dg_value 0] { "virtual" { if {[catch {virtual [lindex $sni_dg_value 1]}]} { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : TLS server_name value = ${tls_servername} ; TLS prefered version = 0x${tls_handshake_prefered_version} ; Virtual server [lindex $sni_dg_value 1] doesn't exist"} } else { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : TLS server_name value = ${tls_servername} ; TLS prefered version = 0x${tls_handshake_prefered_version} ; forwarded to Virtual server [lindex $sni_dg_value 1]"} } } "pool" { if {[catch {pool [lindex $sni_dg_value 1]}]} { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : TLS server_name value = ${tls_servername} ; TLS prefered version = 0x${tls_handshake_prefered_version} ; Pool [lindex $sni_dg_value 1] doesn't exist"} } else { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : TLS server_name value = ${tls_servername} ; TLS prefered version = 0x${tls_handshake_prefered_version} ; forwarded to Pool [lindex $sni_dg_value 1]"} } if {[lindex $sni_dg_value 2] equals "ssl_offload" && [info exists sslenable]} { eval $sslenable } } "node" { if {[catch {node [lindex $sni_dg_value 1]}]} { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : TLS server_name value = ${tls_servername} ; TLS prefered version = 0x${tls_handshake_prefered_version} ; Invalid Node value [lindex $sni_dg_value 1]"} } else { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : TLS server_name value = ${tls_servername} ; TLS prefered version = 0x${tls_handshake_prefered_version} ; forwarded to Node [lindex $sni_dg_value 1]"} } } "handshake_failure" { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : TLS server_name value = ${tls_servername} ; TLS prefered version = 0x${tls_handshake_prefered_version} ; connection is rejected (with Handshake Failure message)"} TCP::respond [binary format cH4Scc 21 $tls_handshake_prefered_version 2 2 40] after 10 TCP::close return } "reject" { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : TLS server_name value = ${tls_servername} ; TLS prefered version = 0x${tls_handshake_prefered_version} ; connection is rejected"} reject return } "drop" { if {$static::sni_routing_debug} {log local0. "[IP::remote_addr] : TLS server_name value = ${tls_servername} ; TLS prefered version = 0x${tls_handshake_prefered_version} ; connection is dropped"} drop return } } TCP::release }
Radware config translation
Problem this snippet solves: This is a simple Python script that translates Radware text configuration file and outputs as config snippet and certificate files. $ ./rad2f5 Usage: rad2f5 <filename> [partition] Example: ./rad2f5 radwareconfig.txt MyPartition Using partition /MyPartition/ IP routes:46 Non-floating IP interfaces:26 Floating IP interfaces:19 Monitors:349 Virtual Servers:430 Pools:152 Certificates: 31 SSL profiles: 17 WARNING! Found L7 Regular Expression at "appdirector l7 farm-selection method-table setCreate l7Rule-Redirect" Layer 7 rules:12 Layer 7 policies:4 !! Copy all certificate files to BIG-IP /var/tmp and use load_certs.sh to load !! # Configuration #--------------------------------------# ltm policy /MyPartition/l7rule-Redirect { controls { forwarding } requires { http } rules { Redirect { actions { 0 { forward select pool SORRYPOOL } } ordinal 1 } } net route /MyPartition/10.20.30.40_32 { network 10.20.30.40/32 gateway 10.1.1.1 } How to use this snippet: This includes translation of routes and non-floating self-IPs, monitors, pools, virtual servers, certificates, SSL profiles and some layer 7 rules. To install, either download the file attached here, extract and run it or use pip: pip install rad2f5 To load the configuration to the f5 device, output to a file eg ./rad2f5 filename>newcfg.txt , remove the statistics header from the file with a text editor, upload the file into /var/tmp on the BIG-IP and test loading with tmsh load sys config merge file /var/tmp/<filename> verify . Fix any issues and load with tmsh load sys config merge file /var/tmp/<filename> The script will also output the certificates and keys which should be uploaded to the BIG-IP /var/tmp directory and run file check_certs.sh and load_certs.sh to check and load the certs and keys. Works with Python v2 and v3. If you want something to be added then message me with details of the text and i will try to add it. Feel free to add or change anything Code : #!/usr/bin/env python # v1.1 14/3/2018 Deal with lack of text in monitors ( line 157 ) # v2 3/1/2019 Updated after changes suggested by Avinash Piare # v3 6/11/2019 Updated to deal with \\r\n at the end of the line # v3.1 6/11/2019 Minor changes to handle VS names with spaces # 3.4 18/4/2020 Fixed minor issues # Peter White 6/11/2019 # usage ./rad2f5 [partition] import sys import re import textwrap import ipaddress print ("Running with Python v" + sys.version) # Set debug to true to output the workings of the script ie the arrays etc debug = False def parse_options(options): # Function to split options and return a dict containing option and value # Example: -pl 27 -v 123 -pa 172.28.1.234 # Return: { 'pl':'27', 'v':'123', 'pa':'172.28.1.234' } output = dict() # Deal with non-quotes eg -b 12345 for r in re.finditer(r'-(\w{1,3}) (\S+)',options): output[r.group(1)] = r.group(2) # Deal with quotes eg -u "User Name" for v in re.finditer(r'-(\w{1,3}) (".+")',options): output[v.group(1)] = v.group(2) return output def splitvars(vars): # Split a string on | and then =, return a dict of ABC=XYZ # eg HDR=User-Agent|TKN=Googlebot-Video/1.0|TMATCH=EQ| # Return is { 'HDR':'User-Agent','TKN': 'Googlebot-Video/1.0', 'TMATCH': 'EQ' } vlist = vars.split("|") o = {} for v in vlist: if v == '': continue w = v.split("=") if len(w) > 1: o[w[0]] = w[1] else: o[w[0]] = '' return o def array_add(a,d): # Function to safely add to an array #global a if not len(a): a = [ d ] else: a.append( d ) return a def setup_pools(name): global pools if not name in pools: pools[name] = { 'destinations': [], 'member-disabled': [], 'member-ids': [], 'priority-group': [] } else: if not 'destinations' in pools[name]: pools[name]['destinations'] = [] if not 'member-disabled' in pools[name]: pools[name]['member-disabled'] = [] if not 'member-ids' in pools[name]: pools[name]['member-ids'] = [] if not 'priority-group' in pools[name]: pools[name]['priority-group'] = [] def getVsFromPool(poolname): # This retrieves the names of virtual servers that use a specific pool global farm2vs if poolname in farm2vs: return farm2vs[poolname] else: return False def is_ipv6(ip): if ':' in ip: return True else: return False # Check there is an argument given if not len(sys.argv) > 1: exit("Usage: rad2f5 [partition]") if len(sys.argv) > 2: # Second command-line variable is the partition partition = "/" + sys.argv[2] + "/" print ("Using partition " + partition) else: partition = "/Common/" # Check input file fh = open(sys.argv[1],"r") if not fh: exit("Cannot open file " + argv[1]) rawfile = fh.read() # v2 # Remove any instances of slash at the end of line # eg health-monitoring check create\ # HC_DNS -id 5 -m DNS -p 53 -a \ #HOST=ns.domain.com|ADDR=1.1.1.1| -d 2.2.2.2 #file = re.sub('^security certificate table\\\\\n','',rawfile) file = re.sub('\\\\\n','',rawfile) file = re.sub('\\\\\r\n','',file) ################ LACP trunks ########################## # # # ############################################################# #net linkaggr trunks set T-1 -lap G-13,G-14 -lam Manual -lat Fast -law 3 -laa 32767 trunks = {} for r in re.finditer(r'net linkaggr trunks set (\S+) (.+)',file): name = r.group(1) options = parse_options(r.group(2)) if 'lap' in options: # Manage interfaces ints = options['lap'].split(',') trunks[name] = { 'interfaces': ints } print ("LACP trunks:" + str(len(trunks))) if debug: print('DEBUG LACP trunks: ' + str(trunks)) ################ IP routes ########################## # # # ############################################################# routes = {} for r in re.finditer(r'net route table create (\S+) (\S+) (\S+) (\S+)',file): if r.group(1) == '0.0.0.0': name = 'default' else: name = r.group(1) + "_" + r.group(2) routes[name] = { 'network': r.group(1), 'mask': r.group(2), 'gateway': r.group(3), 'interface': r.group(4)} print ("IP routes:" + str(len(routes))) if debug: print('DEBUG routes: ' + str(routes)) ################ Non-floating self-IPs ############### # # # # ############################################################# selfIpNonFloating = {} for s in re.finditer(r'net ip-interface create (\S+) (\S+) (.+)',file): output = { 'interface': s.group(2) } opts = parse_options(s.group(3)) if 'pl' in opts: output['mask'] = opts['pl'] if 'v' in opts: output['vlan'] = opts['v'] if 'pa' in opts: output['peerAddress'] = opts['pa'] selfIpNonFloating[s.group(1)] = output print ("Non-floating IP interfaces:" + str(len(selfIpNonFloating))) if debug: print("DEBUG non-floating IPs: " + str(selfIpNonFloating)) ################ Floating self-IPs ################### # # # ############################################################# selfIpFloating = {} for s in re.finditer(r'redundancy vrrp virtual-routers create (\S+) (\S+) (\S+) (.+)',file): output = { 'version': s.group(1),'interface': s.group(3), 'ipAddresses': []} opts = parse_options(s.group(4)) #if 'as' in opts: # Enabled - assume all are enabled if 'pip' in opts: output['peerIpAddress'] = opts['pip'] selfIpFloating[s.group(2)] = output # Retrieve IP addresses for Floating self-IPs # for s in re.finditer(r'redundancy vrrp associated-ip create (\S+) (\S+) (\S+) (\S+)',file): selfIpFloating[s.group(2)]['ipAddresses'] = array_add(selfIpFloating[s.group(2)]['ipAddresses'],s.group(4)) print ("Floating IP interfaces:" + str(len(selfIpFloating))) if debug: print("DEBUG Floating IPs: " + str(selfIpFloating)) ################ Monitors ########################### # # -m XYZ is the monitor type may or may not be present ( not present for icmp type ) # -id is the monitor ID, -p is the port, -d is the destination # ############################################################# monitors = {} for m in re.finditer(r'health-monitoring check create (\S+) (.+)',file): output = { 'name': m.group(1) } opts = parse_options(m.group(2)) if 'id' in opts: id = opts['id'] else: print ("No ID for monitor " + m.group(1)) continue if 'm' in opts: output['type'] = opts['m'] else: output['type'] = 'icmp' if 'a' in opts: output['text'] = opts['a'] else: # Added in v2 output['text'] = '' if 'p' in opts: output['port'] = opts['p'] else: output['port'] = '' monitors[id] = output print ("Monitors:" + str(len(monitors))) if debug: print("DEBUG Monitors:" + str(monitors)) ################ Virtual Servers ##################### # # 0 = name, 1= IP, 2=protocol, 3=port, 4=source 5=options # -ta = type, -ht = http policy, -fn = pool name, -sl = ssl profile name, -ipt = translation? # -po = policy name # ############################################################# farm2vs = {} virtuals = {} snatPools = {} for v in re.finditer(r'appdirector l4-policy table create (\S+) (\S+) (\S+) (\S+) (\S+) (.*)',file): # Puke if VS has quotes if v.group(1).startswith('"'): name = v.group(1).strip('"').replace(' ','_') + v.group(2).strip('"').replace(' ','_') address,protocol,port = v.group(3),v.group(4),v.group(5) source = v.group(6).split(' ')[0] options = ' '.join(v.group(6).split(' ')[1:]) else: name,address,protocol,port,source,options = v.group(1),v.group(2),v.group(3),v.group(4),v.group(5),v.group(6) opts = parse_options(options) # Get rid of ICMP virtual servers if protocol == 'ICMP': continue print ("Name " + name + " has quotes, address: " + address + " protocol: " + protocol) if port == 'Any': port = '0' output = {'source': source, 'destination': address + ":" + port, 'protocol': protocol, 'port': port } if 'fn' in opts: output['pool'] = opts['fn'] if not opts['fn'] in farm2vs: farm2vs[opts['fn']] = [] farm2vs[opts['fn']].append(name) output['port'] = port if 'po' in opts: output['policy'] = opts['po'] if 'ipt' in opts: output['snat'] = 'snatpool_' + opts['ipt'] if not 'snatpool_' + opts['ipt'] in snatPools: #Create snatpool snatPools['snatpool_' + opts['ipt']] = { 'members': [opts['ipt']] } # Set the correct profiles profiles = [] # Layer 4 if protocol == "TCP" or protocol == "UDP": profiles.append(protocol.lower()) else: profiles.append('ipother') # http if port == '80' or port == '8080': profiles.append('http') profiles.append('oneconnect') # ftp if port == '21': profiles.append('ftp') # RADIUS if port == '1812' or port == '1813': profiles.append('radius') # SSL if 'sl' in opts: profiles.append(opts['sl']) profiles.append('http') profiles.append('oneconnect') output['profiles'] = profiles # virtuals[name] = output print ("Virtual Servers:" + str(len(virtuals))) if debug: print("DEBUG Virtual Servers:" + str(virtuals)) #print("DEBUG Farm to VS Mapping:" + str(farm2vs)) ################ Pools ############################### # # Pools config options are distributed across multiple tables # This sets the global config such as load balancing algorithm # -dm This sets the distribution method. cyclic = Round Robin, Fewest Number of Users = least conn, Weighted Cyclic = ratio, Response Time = fastest # -as this is the admin state # -cm is the checking method ie monitor # -at is the activation time ############################################################# pools = {} for p in re.finditer(r'appdirector farm table setCreate (\S+) (.*)',file): name = p.group(1) setup_pools(name) opts = parse_options(p.group(2)) output = {} # Admin state output['poolDisabled'] = False if 'as' in opts: if opts['as'] == 'Disabled': output['poolDisabled'] = True # Deal with distribution methods method = 'round-robin' if 'dm' in opts: if opts['dm'] == '"Fewest Number of Users"': method = 'least-conn' elif opts['dm'] == 'Hashing': method = 'hash' else: method = 'round-robin' output['lbMethod'] = method if 'at' in opts: output['slowRamp'] = opts['at'] pools[name] = output # This sets the pool members # 0=name, 1=node, 2=node address, 3=node port, 4=? -id =id -sd ? -as admin state eg Disable, -om operation mode eg Backup ( fallback server ), -rt backup server address 0.0.0.0 for p in re.finditer(r'appdirector farm server table create (\S+) (\S+) (\S+) (\S+) (\S+) (.*)',file): name,node,node_address,node_port,node_hostname = p.group(1),p.group(2),p.group(3),p.group(4),p.group(5) opts = parse_options(p.group(6)) setup_pools(name) if node_port == 'None': node_port = '0' pools[name]['destinations'] = array_add(pools[name]['destinations'],node_address + ":" + node_port) # Manage monitor if node_port == '80' or node_port == '8080': monitor = 'http' elif node_port == '443': monitor = 'https' elif node_port == '53': monitor = 'dns' elif node_port == '21': monitor = 'ftp' elif node_port == '0': monitor = 'gateway-icmp' else: monitor = 'tcp' pools[name]['monitor'] = monitor # Retrieve pool member ID if 'id' in opts: pools[name]['member-ids'] = array_add(pools[name]['member-ids'],opts['id']) else: print ("ID not found for pool " + name) # Check if member is disabled if 'as' in opts and opts['as'] == 'Disabled': # This pool member is disabled pools[name]['member-disabled'] = array_add(pools[name]['member-disabled'],True) else: pools[name]['member-disabled'] = array_add(pools[name]['member-disabled'],False) # Check if member is backup ie Priority Groups if 'om' in opts and opts['om'] == 'Backup': pools[name]['priority-group'] = array_add(pools[name]['priority-group'],20) else: pools[name]['priority-group'] = array_add(pools[name]['priority-group'],0) print ("Pools:" + str(len(pools)) ) if debug: print("DEBUG pools: " + str(pools)) ################ SSL certificates ################### # # #Name: certificate_name \ #Type: certificate \ #-----BEGIN CERTIFICATE----- \ #MIIEcyFCA7agAwIAAgISESFWs9QGF # ############################################################# certs = {} for r in re.finditer(r'Name: (\S+) Type: (\S+) (-----BEGIN CERTIFICATE-----.+?-----END CERTIFICATE-----)',file): name,type,text = r.group(1),r.group(2),r.group(3) certs[name] = { 'type': type,'text': text.replace(' \\\r\n','') } # Print out to file or something print ("SSL Certificates: " + str(len(certs))) if debug: print("DEBUG certs: " + str(certs)) ################ SSL Keys ################### # # #Name: New_Root_Cert Type: key Passphrase: -----BEGIN RSA PRIVATE KEY----- Proc-Type: 4,ENCRYPTED DEK-Info: [key] [key] -----END RSA PRIVATE KEY----- \ # ############################################################# keys = {} # Keys with passphrase for r in re.finditer(r'Name: (\S+) Type: key Passphrase: (\S+) (-----BEGIN RSA PRIVATE KEY-----.+?-----END RSA PRIVATE KEY-----)',file): name,passphrase,text = r.group(1),r.group(2),r.group(3) keys[name] = { 'passphrase': passphrase,'text': text.replace(' \\\r\n','') } # Keys without passphrase for r in re.finditer(r'Name: (\S+) Type: key (-----BEGIN RSA PRIVATE KEY-----.+?-----END RSA PRIVATE KEY-----)',file): name,text = r.group(1),r.group(2) keys[name] = { 'text': text.replace(' \\\r\n','') } print ("SSL Keys: " + str(len(keys))) if debug: print("DEBUG SSL keys: " + str(keys)) ################ SSL profiles ######################## # # -c is cert, -u is ciphers, -t is chain cert, -fv - versions ############################################################## sslProfiles = {} for s in re.finditer(r'^appdirector l4-policy ssl-policy create (\S+) (.+)',file.replace('\\\r\n',''),re.MULTILINE): name = s.group(1) output = {} opts = parse_options(s.group(2)) if 'c' in opts: # Certificate output['certificate'] = opts['c'] if 't' in opts: # Chain cert output['chaincert'] = opts['t'] if 'fv' in opts: # TLS Version output['version'] = opts['fv'] if 'u' in opts: # User-defined Ciphers output['cipher'] = opts['u'] if 'lp' in opts: # ? output['lp'] = opts['lp'] if 'pa' in opts: # ? output['pa'] = opts['pa'] if 'cb' in opts: # ? output['cb'] = opts['cb'] if 'i' in opts: # Backend SSL Cipher -> Values: Low, Medium, High, User Defined output['i'] = opts['i'] if 'bs' in opts: # Backend SSL in use ie serverSSL output['serverssl'] = opts['bs'] sslProfiles[name] = output print ("SSL profiles: " + str(len(sslProfiles))) if debug: print("DEBUG SSL Profiles: " + str(sslProfiles)) ################ SNAT pools ########################## # # Note that this only works for addresses in the same /24 subnet ############################################################# for s in re.finditer(r'appdirector nat client address-range create (\S+) (\S+)',file): name = s.group(1) if sys.version.startswith('2'): start = ipaddress.ip_address(unicode(name,'utf_8')) end = ipaddress.ip_address(unicode(s.group(2),'utf_8')) else: start = ipaddress.ip_address(name) end = ipaddress.ip_address(s.group(2)) current = start ipAddresses = [] while current <= end: ipAddresses.append(str(current)) current += 1 snatPools['snatpool_' + name] = { 'members': ipAddresses } print ("SNAT Pools:" + str(len(snatPools))) if debug: print("DEBUG SNAT Pools: " + str(snatPools)) #################################################################### ################ Layer 7 functions ########################## # # #################################################################### l7rules = { } for r in re.finditer(r'appdirector l7 farm-selection method-table setCreate (\S+) (.+)',file): name = r.group(1) l7rules[name] = { 'match': [], 'action': [] } opts = parse_options(r.group(2)) if 'cm' in opts and opts['cm'] == '"Header Field"' and 'ma' in opts: # This is a rule to insert a header params = splitvars(opts['ma']) if 'HDR' in params and 'TKN' in params: if params['TKN'] == "$Client_IP": params['TKN'] = '[IP::client_addr]' l7rules[name]['action'].append("http-header\ninsert\nname " + params['HDR'] + "\nvalue " + params['TKN']) if 'cm' in opts and opts['cm'] == 'URL' and 'ma' in opts: # This does a match on Host and URL params = splitvars(opts['ma']) if 'HN' in params: l7rules[name]['match'].append("http-host\nhost\nvalues { " + params['HN'] + " }") if 'P' in params: l7rules[name]['match'].append("http-uri\npath\nvalues { " + params['P'] + " }") if 'cm' in opts and opts['cm'] == '"Regular Expression"' and 'ma' in opts: params = splitvars(opts['ma']) if 'EXP' in params and params['EXP'] == '.': # This is a regex which matches everything print ("REGEX: " + name) else: print ("WARNING! Found L7 Regular Expression at \"appdirector l7 farm-selection method-table setCreate " + name + "\". Manually set the match in output config.") # Note that there can be multiple entries ie multiple rules per policy l7policies = {} for p in re.finditer(r'appdirector l7 farm-selection policy-table setCreate (\S+) (\d+) (.+)',file): name = p.group(1) precedence = p.group(2) opts = parse_options(p.group(3)) if 'fn' in opts: farm = opts['fn'] else: farm = '' if 'm1' in opts: rule = opts['m1'] if 'pa' in opts: # Retain HTTP Persistency (PRSST)-If the argument is ON (or undefined), AppDirector maintains HTTP 1.1 # HTTP Redirect To (RDR)-Performs HTTP redirection to the specified name or IP address. # HTTPS Redirect To (RDRS)-AppDirector redirects the HTTP request to the specified name or IP address and modifies the request to a HTTPS request. # Redirection Code (RDRC)-Defines the code to be used for redirection. # RDRC=PRMN stand for Permanent I assume , as in HTTP 301 # SIP Redirect To (RDRSIP)-Performs SIP redirection to the specified name or IP address. params = splitvars(opts['pa']) if 'RDR' in params: url = 'http://' + params['RDR'] elif 'RDRS' in params: url = 'https://' + params['RDRS'] else: url = '' if not name in l7policies: l7policies[name] = [] if rule in l7rules: if farm != '': l7rules[rule]['action'].append("forward\nselect\npool " + farm) elif url != '': l7rules[rule]['action'].append("http-redirect\nhost " + url) l7policies[name].append({ 'precedence': precedence, 'farm': farm, 'rule': rule }) print ("Layer 7 rules:" + str(len(l7rules))) print ("Layer 7 policies:" + str(len(l7policies))) if debug: print("DEBUG L7 rules: " + str(l7rules)) print("DEBUG L7 policies: " + str(l7policies)) # # We have retrieved the required configuration from the input file # Now start outputting the config ################# output SSL certificates import script ###################### # # ######################################################################## if len(certs): print ("-- Creating SSL certs and keys --") with open("load_certs.sh",'w') as loadScript: loadScript.write("#!/bin/bash\n# Script to load SSL certs and keys from /var/tmp\n") ##### Manage Certificates ######## for cert in certs: # Write certs to load_certs.sh if certs[cert]['type'] == 'certificate' or certs[cert]['type'] == 'interm': loadScript.write("tmsh install sys crypto cert " + partition + cert + ".crt from-local-file /var/tmp/" + cert + ".crt\n") # Create the certificate files with open(cert + ".crt","w") as certFile: for m in re.finditer(r'(-----BEGIN CERTIFICATE-----)\s?(.+)\s?(-----END CERTIFICATE-----)',certs[cert]['text']): certFile.write(m.group(1) + "\n") certFile.write(textwrap.fill(m.group(2),64) + "\n") certFile.write(m.group(3) + "\n") print ("Created SSL certificate file " + cert + ".crt") ##### Manage Keys ################# for key in keys: # Write keys to load_certs.sh loadScript.write("tmsh install sys crypto key " + partition + key + ".key") if 'passphrase' in keys[key]: loadScript.write(" passphrase " + keys[key]['passphrase']) loadScript.write(" from-local-file /var/tmp/" + key + ".key\n") # Create the key files with open(key + ".key","w") as keyFile: for n in re.finditer(r'(-----BEGIN RSA PRIVATE KEY-----)(.+)(-----END RSA PRIVATE KEY-----)',keys[key]['text']): keyFile.write(n.group(1) + "\n") if 'Proc-Type:' in n.group(2) and 'DEK-Info:' in n.group(2): # File is encrypted, separate the first lines for o in re.finditer(r'(Proc-Type: \S+) (DEK-Info: \S+) (.+)',n.group(2)): keyFile.write(o.group(1) + "\n") keyFile.write(o.group(2) + "\n\n") keyFile.write(textwrap.fill(o.group(3),64) + "\n") else: # File is not encrypted, output as it is keyFile.write(textwrap.fill(n.group(2),64) + "\n") keyFile.write(n.group(3) + "\n") print ("Created SSL key file " + key + ".key") print ("-- Finished creating SSL certs and keys --") print ("!! Copy all .crt and .key SSL files to BIG-IP /var/tmp and use load_certs.sh to load !!" ) ####################################################################### ################# output SSL certificates checking script ###################### # # ######################################################################## with open("check_certs.sh",'w') as checkScript: checkScript.write("#!/bin/bash\n# Script to check SSL certs and keys\n") checkScript.write("\n# Check SSL Certs\n") for cert in certs: if certs[cert]['type'] == 'certificate' or certs[cert]['type'] == 'interm': checkScript.write("openssl x509 -in " + cert + ".crt -noout || echo \"Error with file " + cert + ".crt\"\n") checkScript.write("\n# Check SSL Keys\n") for key in keys: checkScript.write("openssl rsa -in " + key + ".key -check || echo \"Error with file " + key + ".key\"\n") print ("!! Run the check_certs.sh script to check the certificates are valid !!" ) ####################################################################### # # # # # # # # print ("\n\n\n\n\n\n# Configuration \n#--------------------------------------#") ################# output policy config ##################### # # ############################################################# #print ("L7 rules: " + str(l7rules)) for i in l7policies.keys(): output = "ltm policy " + partition + i + " {\n\tcontrols { forwarding }\n\trequires { http }\n\t" output += "rules {\n\t" ordinal = 1 for j in l7policies[i]: output += "\t" + j['rule'] + " { \n" if len(l7rules[j['rule']]['match']): # Deal with conditions output += "\t\t\tconditions { \n" l = 0 for k in l7rules[j['rule']]['match']: output += "\t\t\t\t" + str(l) + " { \n\t\t\t\t\t" + k.replace('\n','\n\t\t\t\t\t') l += 1 output += "\n\t\t\t\t }\n" output += "\n\t\t\t }\n" if len(l7rules[j['rule']]['action']): # Deal with actions output += "\t\t\tactions { \n" m = 0 for n in l7rules[j['rule']]['action']: output += "\t\t\t\t" + str(m) + " { \n\t\t\t\t\t" + n.replace('\n','\n\t\t\t\t\t') m += 1 output += "\n\t\t\t\t }\n" output += "\n\t\t\t }\n" output += "\t\tordinal " + str(ordinal) output += "\n\t\t}\n\t" ordinal += 1 output += "\n\t}\n}\n" print (output) ################# output LACP trunk config ############## # # ############################################################# for trunk in trunks.keys(): output = "net trunk " + partition + trunk + " {\n" if 'interfaces' in trunks[trunk]: output += "\tinterfaces {\n" for int in trunks[trunk]['interfaces']: output += "\t\t" + int + "\n" output += "\t}\n" output += "}\n" print (output) ################# output vlan config ############## # # ############################################################# for ip in selfIpNonFloating.keys(): output = "" if 'vlan' in selfIpNonFloating[ip]: vlanName = "VLAN-" + selfIpNonFloating[ip]['vlan'] output += "net vlan " + partition + vlanName + " {\n" if 'interface' in selfIpNonFloating[ip]: output += "\tinterfaces {\n" output += "\t\t" + selfIpNonFloating[ip]['interface'] + " {\n" output += "\t\t\ttagged\n\t\t}\n" output += "\t}\n" output += "\ttag " + selfIpNonFloating[ip]['vlan'] + "\n}\n" print(output) ################# output non-floating self-ip config ####### # # ############################################################# for ip in selfIpNonFloating.keys(): output = "" if 'mask' in selfIpNonFloating[ip]: mask = "/" + selfIpNonFloating[ip]['mask'] else: if is_ipv6(ip): mask = "/64" else: mask = "/32" if 'vlan' in selfIpNonFloating[ip]: vlanName = "VLAN-" + selfIpNonFloating[ip]['vlan'] else: continue output += "net self " + partition + "selfip_" + ip + " {\n\taddress " + ip + mask + "\n\t" output += "allow-service none\n\t" output += "traffic-group traffic-group-local-only\n" output += "\tvlan " + vlanName + "\n" output += "}\n" print (output) ################# output network route config ############## # # ############################################################# for route in routes.keys(): network,mask,gateway = routes[route]['network'],routes[route]['mask'],routes[route]['gateway'] output = "net route " + partition + route + " {\n\tnetwork " + network + "/" + mask + "\n\t" output += "gateway " + gateway + "\n}" print (output) ################# output SSL profiles config ########################## # # ######################################################################## for s in sslProfiles: #print (str(sslProfiles[s])) output = "ltm profile client-ssl " + partition + s + " {\n" output += "\tcert-key-chain { " + s + " { cert " output += sslProfiles[s]['certificate'] + ".crt " output += " key " output += sslProfiles[s]['certificate'] + ".key " if 'chaincert' in sslProfiles[s]: output += "chain " + sslProfiles[s]['chaincert'] output += " }\n\tdefaults-from /Common/clientssl\n}\n" print (output) if 'serverssl' in sslProfiles[s]: print("WARNING: VSs using Client SSL profile " + s + " should have Server SSL profile assigned") ################# output monitor config #################### # # ############################################################# for m in monitors.keys(): if monitors[m]['type'] == '"TCP Port"': output = "ltm monitor tcp " + partition + monitors[m]['name'] + " {\n\tdefaults-from /Common/tcp\n}\n" if monitors[m]['type'] == '"UDP Port"': output = "ltm monitor udp " + partition + monitors[m]['name'] + " {\n\tdefaults-from /Common/udp\n}\n" if monitors[m]['type'] == 'HTTP': ''' https://webhelp.radware.com/AppDirector/v214/214Traffic%20Management%20and%20Application%20Acceleration.03.001.htm https://webhelp.radware.com/AppDirector/v214/HM_Checks_Table.htm Arguments: Host name - HOST=10.10.10.53 path - PATH=/hc.htm HTTP Header HTTP method - MTD=G (G/H/P) send HTTP standard request or proxy request - PRX=N use of no-cache - NOCACHE=N text for search within a HTTP header and body, and an indication whether the text appears Username and Password for basic authentication NTLM authentication option - AUTH=B Up to four valid HTTP return codes - C1=200 -a PATH=/hc.htm|C1=200|MEXIST=Y|MTD=G|PRX=N|NOCACHE=N|AUTH=B| ''' output = "ltm monitor http " + partition + monitors[m]['name'] + " {\n\tdefaults-from /Common/http\n" vars = splitvars(monitors[m]['text']) output += "\tsend \"" if 'MTD' in vars: if vars['MTD'] == 'G': output += "GET " elif vars['MTD'] == 'P': output += "POST " elif vars['MTD'] == 'H': output += "HEAD " else: output += "GET " if 'PATH' in vars: output += vars['PATH'] + ' HTTP/1.0\\r\\n\\r\\n"\n' if 'C1' in vars: output += "\trecv \"^HTTP/1\.. " + vars['C1'] + "\"\n" output += "}\n" if monitors[m]['type'] == 'HTTPS': output = "ltm monitor https " + partition + monitors[m]['name'] + " {\n\tdefaults-from /Common/https\n" vars = splitvars(monitors[m]['text']) output += "\tsend \"" if 'MTD' in vars: if vars['MTD'] == 'G': output += "GET " elif vars['MTD'] == 'P': output += "POST " elif vars['MTD'] == 'H': output += "HEAD " else: output += "GET " if 'PATH' in vars: output += vars['PATH'] + ' HTTP/1.0\\r\\n\\r\\n"\n' output += "}\n" if monitors[m]['type'] == 'LDAP': ''' The Health Monitoring module enhances the health checks for LDAP servers by allowing performing searches in the LDAP server. Before Health Monitoring performs the search, it issues a Bind request command to the LDAP server. After performing the search, it closes the connection with the Unbind command. A successful search receives an answer from the server that includes a "searchResultEntry" message. An unsuccessful search receives an answer that only includes only a "searchResultDone" message. Arguments: Username A user with privileges to search the LDAP server. Password The password of the user. Base Object The location in the directory from which the LDAP search begins. Attribute Name The attribute to look for. For example, CN - Common Name. Search Value The value to search for. Search scope baseObject, singleLevel, wholeSubtree Search Deref Aliases neverDerefAliases, dereflnSearching, derefFindingBaseObj, derefAlways USER=cn=healthcheck,dc=domain|PASS=1234|BASE=dc=domain|ATTR=cn|VAL=healthcheck|SCP=1|DEREF=3| ''' output = "ltm monitor ldap " + partition + monitors[m]['name'] + " {\n\tdefaults-from /Common/ldap\n" vars = splitvars(monitors[m]['text']) if 'BASE' in vars: output += "\tbase \"" + vars['BASE'] + "\"\n" if 'USER' in vars: output += "\tusername \"" + vars['USER'] + "\"\n" if 'PASS' in vars: output += "\tpassword \"" + vars['PASS'] + "\"\n" output += "}\n" if monitors[m]['type'] == 'icmp': output = "ltm monitor gateway-icmp " + partition + monitors[m]['name'] + " {\n\tdefaults-from /Common/gateway-icmp\n}\n" if monitors[m]['type'] == 'DNS': output = "ltm monitor dns " + partition + monitors[m]['name'] + " {\n\tdefaults-from /Common/dns\n" vars = splitvars(monitors[m]['text']) if 'HOST' in vars: output += "\tqname \"" + vars['HOST'] + "\"\n" if 'ADDR' in vars: output += "\trecv \"" + vars['ADDR'] + "\"\n" output += "}\n" if monitors[m]['type'] == '"Radius Accounting"': output = "ltm monitor radius-accounting " + partition + monitors[m]['name'] + " {\n\tdefaults-from /Common/radius-accounting\n" vars = splitvars(monitors[m]['text']) if 'USER' in vars: output += "\tusername \"" + vars['USER'] + "\"\n" #if 'PASS' in vars: if 'SECRET' in vars: output += "\tsecret \"" + vars['SECRET'] + "\"\n" output += "}\n" if monitors[m]['type'] == '"Radius Authentication"': output = "ltm monitor radius " + partition + monitors[m]['name'] + " {\n\tdefaults-from /Common/radius\n" vars = splitvars(monitors[m]['text']) if 'USER' in vars: output += "\tusername \"" + vars['USER'] + "\"\n" #if 'PASS' in vars: if 'SECRET' in vars: output += "\tsecret \"" + vars['SECRET'] + "\"\n" output += "}\n" print (output) ################# output SNAT pool config ###################### # # ############################################################# for s in snatPools.keys(): output = "ltm snatpool " + partition + s + " {\n\t" output += "members {\n" #run through pool range and add members as individual IP addresses below each other for member in snatPools[s]['members']: output += "\t " + member + "\n" output += "\t}" output += "\n}" print (output) ################# output pool config ###################### # # ############################################################# for p in pools.keys(): output = "ltm pool " + partition + p + " {\n\t" if 'monitor' in pools[p]: output += "monitor min 1 of { " + pools[p]['monitor'] + " }\n\t" if 'lbMethod' in pools[p]: output += "load-balancing-mode " + pools[p]['lbMethod'] + "\n\t" output += "members {\n" if 'destinations' in pools[p] and len(pools[p]['destinations']): for i,v in enumerate(pools[p]['destinations']): d = re.sub(':\d+$','',v) output += "\t\t" + v + " {\n\t\t\taddress " + d + "\n" if pools[p]['member-disabled'][i]: output += "\t\t\tstate down\n" output += "\t\t\tpriority-group " + str(pools[p]['priority-group'][i]) + "\n" output += "\t\t}\n" output += "\t}\n}" print (output) ################# output virtual server config ###################### # # ###################################################################### for v in virtuals: output = "ltm virtual " + partition + v + " {\n" output += "\tdestination " + virtuals[v]['destination'] + "\n" output += "\tip-protocol " + virtuals[v]['protocol'].lower() + "\n" if 'pool' in virtuals[v] and virtuals[v]['pool']: output += "\tpool " + virtuals[v]['pool'] + "\n" if 'profiles' in virtuals[v] and len(virtuals[v]['profiles']): output += "\tprofiles {\n" for p in virtuals[v]['profiles']: output += "\t\t" + p + " { }\n" output += "\t}\n" if 'snat' in virtuals[v]: output += "\tsource-address-translation {\n\t" if virtuals[v]['snat'] == 'automap': output += "\ttype automap\n\t" else: output += "\ttype snat\n\t" output += "\tpool " + virtuals[v]['snat'] + "\n\t" output += "}\n" if 'policy' in virtuals[v]: output += "\tpolicies {\n\t" output += "\t" + virtuals[v]['policy'] + " { }\n\t}\n" output += "}" print (output) Tested this on version: 12.0AppDynamics EUM JavaScript Injection for Selective URLs
Problem this snippet solves: This code allows for the adrum.js agent to be dynamically injected into an application to support collecting EUM metrics. With this code, you can select from any number of HTTP Request headers and/or POST parameters. Additional EUM features in use: Custom GeoIP mappings based on a DataGroup User Data for Browser Snapshots, currently APM SID and Username How to use this snippet: This sample iRule assumes that the TCL variables $username and $sid have been previously created to support the UserData injection. If you don't have this information or APM available then these lines should be commented out or changed to meet your needs. Import the ltm profiles and iRule via tmsh load sys config merge <from-terminal> On your virtual, attache the Head-HTML profile as well as the iRule. Code : LTM Profiles Required: ltm html-rule tag-raise-event HTMLEvent-HeadStart-HTMLRule { description "Match an HTML tag for " match { tag-name head } } ltm profile html Head-HTML { app-service none content-detection disabled content-selection { text/html text/xhtml } defaults-from html description none rules { HTMLEvent-HeadStart-HTMLRule } } iRule to be attached to the Virtual: ltm rule SND-AppDynamics-HTML-Rule { when CLIENT_ACCEPTED { set APPDYNAMICS_EUM_DEBUG 0; } when HTTP_REQUEST { set APPDYNAMICS_EUM_APP_ID "AD-xxx-xxx-xxx-xxx" if { $APPDYNAMICS_EUM_DEBUG > 0 } {set uri [HTTP::uri]} if {[HTTP::uri] starts_with "/my/HomePage.do"} { # set enableEum 1 if { [HTTP::header Content-Type] eq "application/x-www-form-urlencoded" } { # HTTP::collect [HTTP::header Content-Length] # Only collect the first 200 bytes which should be enough to capture eventSource HTTP::collect 200 } else { if {[HTTP::uri] starts_with "/my/defaultPage"} { set enableEum 1 } } } elseif {[HTTP::uri] starts_with "/app1"} { set enableEum 1 } elseif {[HTTP::uri] starts_with "/app2"} { set enableEum 1 } when HTTP_REQUEST_DATA { set namevals [split [HTTP::payload] "&"] for {set i 0} {$i < [llength $namevals]} {incr i} { set params [split [lindex $namevals $i] "="] # log local0. " [lindex $params 0] : [lindex $params 1]" if {[lindex $params 0] equals "eventSource"} { switch -glob [lindex $params 1] { "PostParm1" - "PostParm2" - "PostParm3" { set enableEum 1 if { $APPDYNAMICS_EUM_DEBUG > 0 } { log local0. "FOUND HTTP DATA [lindex $params 1] TO ENABLE APPDYNAMICS EUM"; } } } break; } unset params } unset namevals } when HTTP_RESPONSE { if {[info exists enableEum]} { if { ($enableEum == 1) && ([HTTP::header "Content-Type"] starts_with "text/html") } { HTML::enable set enableEum 0 } else { HTML::disable } } else { HTML::disable } } when HTML_TAG_MATCHED { if { $APPDYNAMICS_EUM_DEBUG > 0 } { log local0. "uri = $uri element = [HTML::tag name] attribute id = [HTML::tag attribute id]"} switch [HTML::tag name] { "head" { if { $APPDYNAMICS_EUM_DEBUG > 0 } { log local0. "$uri info username [info exists username] info sid [info exists sid]"} if {([info exists username]) && ([info exists sid])} { # Send custom GeoIP Data to with the adrum beacon if we are on the private network set geoData [split [class match -value [IP::client_addr] equals Branch-Address-DataGroup ] "/"] if { $geoData != "" } { HTML::tag append [subst { window\["adrum-app-key"\] = "$APPDYNAMICS_EUM_APP_ID"; window\["adrum-start-time"\] = new Date().getTime(); window\["adrum-geo-resolver-url"\] = "thisdoesnotmatter"; window\["adrum-config"\] = { geo: { "country": "United States", "region": "[lindex $geoData 0]", "city": "[lindex $geoData 1]", "localIP": "[IP::client_addr]" } }; if (ADRUM) { ADRUM.command ("addUserData", "User", "$username"); } { ADRUM.command ("addUserData", "LastMRH_Session", "$sid"); } } ] } else { HTML::tag append [subst { window\["adrum-app-key"\] = "$APPDYNAMICS_EUM_APP_ID"; window\["adrum-start-time"\] = new Date().getTime(); if (ADRUM) { ADRUM.command ("addUserData", "User", "$username"); } { ADRUM.command ("addUserData", "LastMRH_Session", "$sid"); } } ] } unset geoData } } } } } Tested this on version: 11.5Use F5 LTM as HTTP Proxy
Problem this snippet solves: LTM product can be used as a HTTP Proxy for servers and PC. This code explains minimum requirements to configure proxy feature without SWG module (configurations from Explicit Forward Proxy documentation without documentation ) and without explicit proxy iApp. How to use this snippet: All these commands must be run in bash shell. Create HTTP PROXY VIRTUAL SERVER Configure variables used in next commands Variable HTTPBaseName is used to create : Resolver object : RESOLVER_${HTTPBaseName} HTTP profile : http_${HTTPBaseName} virtual server : VS_${HTTPBaseName} HTTPBaseName="HTTP_FORWARD_PROXY" VS_IP="192.168.2.80" VS_PORT="8080" create DNS resolver with your DNS server (1.1.1.1 is for demo using cloudflare) tmsh create net dns-resolver RESOLVER_${HTTPBaseName} { forward-zones replace-all-with { . { nameservers replace-all-with { 1.1.1.1:domain { } } } } route-domain 0 } create HTTP profile type explicit, using DNS resolver. The parameter default-connect-handling allow enables HTTPS connections without SSL inspection tmsh create ltm profile http http_${HTTPBaseName} { defaults-from http-explicit explicit-proxy { default-connect-handling allow dns-resolver RESOLVER_${HTTPBaseName} } proxy-type explicit } create HTTP proxy Virtual server tmsh create ltm virtual VS_${HTTPBaseName} { destination ${VS_IP}:${VS_PORT} ip-protocol tcp mask 255.255.255.255 profiles replace-all-with { http_${HTTPBaseName} { } tcp } source 0.0.0.0/0 source-address-translation { type automap } translate-address enabled translate-port enabled} ENABLE SSL FORWARD PROXY This section is not required to forward HTTPS requests but only to enable SSL inspection on HTTPS requests. Note : Following configuration requires SSL, Forward Proxy License. Configure variables used in next commands Variable SSLBaseName is used to create : certificate / key pair : ${SSLBaseName} Client SSL profile : clientssl_${SSLBaseName} Server SSL profile : serverssl_${SSLBaseName} virtual server : VS_${SSLBaseName} SSLBaseName="SSL_FORWARD_PROXY" dirname="/var/tmp" CASubject="/C=FR/O=DEMO\ COMPANY/CN=SSL\ FORWARD\ PROXY\ CA" Create self-signed certificate for CA purpose (not available in WebUI) Self-signed certificates created in WebUI doesn't have CA capability required for SSL FORWARD PROXY. openssl genrsa -out ${dirname}/${SSLBaseName}.key 4094 openssl req -sha512 -new -x509 -days 3650 -key ${dirname}/${SSLBaseName}.key -out ${dirname}/${SSLBaseName}.crt -subj "${CASubject}" Import certificates in TMOS tmsh install sys crypto key ${SSLBaseName}.key from-local-file ${dirname}/${SSLBaseName}.key; tmsh install sys crypto cert ${SSLBaseName}.crt from-local-file ${dirname}/${SSLBaseName}.crt; After CA Certificate is imported, browse in WebUI, retrieve it and import it in client browsers trusted CA Create SSL profiles for SSL FORWARD PROXY tmsh create ltm profile client-ssl clientssl_${SSLBaseName} { cert-lookup-by-ipaddr-port disabled defaults-from clientssl mode enabled proxy-ca-cert ${SSLBaseName}.crt proxy-ca-key ${SSLBaseName}.key ssl-forward-proxy enabled } tmsh create ltm profile server-ssl serverssl_${SSLBaseName} { defaults-from serverssl ssl-forward-proxy enabled } create SSL FORWARD PROXY Virtual server tmsh create ltm virtual VS_${SSLBaseName} { destination 0.0.0.0:https ip-protocol tcp profiles replace-all-with { clientssl_${SSLBaseName} { context clientside } serverssl_${SSLBaseName} { context serverside } http { } tcp { } } source 0.0.0.0/0 translate-address disabled translate-port disabled vlans replace-all-with { http-tunnel } vlans-enabled } Change HTTP EXPLICIT PROXY Default Connect Handling to Deny tmsh modify ltm profile http http_${HTTPBaseName} explicit-proxy { default-connect-handling deny } Note : These commands were tested in both 12.1 and 13.1 versions. Code : No Code
BIGIP LTM Automated Pool Monitor Flap Troubleshooting Script in Bash
Problem this snippet solves: A bash script is mainly for collecting data when F5 BIG-IP LTM pool member monitor flaps in a period of time and help determine the Root Cause of BIGIP monitor health check failure; Script will monitor the LTM logs, if new pool member down message event occurs, script will perform following functions: 1. Turn on LTM bigd debug ; 2. Start to tcpdump capture to capture relevant traffics; 3. Turn off bigd debug and terminate tcpdump process when timer elapse (timer is configurable) 4. Generate qkview (optinal) 5. Tar ball full logs files under /var/log/ directory (optinal) Script has been tested on v11.x Code : #!/usr/bin/bash ##########identify the log file that script is monitoring filename="/var/log/ltm" ##########identify the period of time that debug and tcpdump are running, please change it according to the needs; timer=60 ##########IP address of pool member flaps poolMemberIP="10.10.10.229" ##########self IP address of LTM is usd to send LTM Health Monitor traffics ltmSelfip="10.10.10.248" ##########pool member service port number poolMemberPort="443" ##########TMOS command to turn on bigd debug turnonBigdDebug="tmsh modify sys db bigd.debug value enable" ##########TMOS command to turn off bigd debug turnoffBigdDebug="tmsh modify sys db bigd.debug value disable" ##########BASH command to tar BIGIP log files tarLogs="tar -czpf /var/tmp/logfiles.tar.gz /var/log/*" ####### function file check: following code will check if /var/log/ltm exist on the system, ####### if it exists, script will be running and perform subsequent functions if [ -f $filename ] then echo "/var/log/ltm exists and program is running to collect data when BG-IP pool member flaps" else ####### if it does not exist, programe will be terminated and log following message echo "no /var/log/ltm file found and program is terminated" exit 0 fi ####### function file check ends ###### write timestap to /var/log/ltm for tracking purpose echo "$(date) monitoring the log" >> $filename ###### start to monitor the /var/log/ltm for new events tail -f -n 0 $filename | while read -r line do ###### counter for pool down message appears hit=$(echo "$line" | grep -c "$poolMemberIP:$poolMemberPort monitor status down") #echo $hit ###### if [ "$hit" == "1" ]; then ###### diplay the pool down log event in file /var/log/ltm echo $line ###### show timestamp of debug is on echo "$(date) Turning on system bigddebug" ###### turn on bigd debug echo $($turnonBigdDebug) ###### turn on tcpdump capture echo $(tcpdump -ni 0.0:nnn -s0 -w /var/tmp/Monitor.pcap port $poolMemberPort and \(host $poolMemberIP and host $ltmSelfip\)) & ###### running timer sleep $timer ###### show timestamp of debug is off echo "$(date) Truning off system bigddebug" ###### turn off bigd debug echo $($turnoffBigdDebug) ###### terminate tcpdump process echo $(killall tcpdump) ###### generate qkview, it's an optional function, enable it by remove "#" sign #echo $(qkview) ###### tar log files, it's an optional function, enable it by remove "#" sign #echo $($tarLogs) break #else #echo "Monitor in progress" fi done ###### show message that programe is end echo "$(date) exiting from programe" ###### exit from the program exit 0 Tested this on version: 11.6Knowledge sharing: Ways to trigger and schedule scripts on the F5 BIG-IP devices.
Problem this snippet solves: Available script and rest-api options for f5 automatic configurations. How to use this snippet: Code : I think that it is interesting to share how on F5 different scripts can be run at different times and states. 1.You can use the cron job like on any linux device to run a script. As I have used this to restart the tomcat and httpd each night with "bigstart restart " or "tmsh restart /sys service " (https://support.f5.com/csp/article/K89999342), because of a bug till I upgade the devices (https://support.f5.com/csp/article/K25554628 ). https://support.f5.com/csp/article/K03108954 2.Newer versions of F5 also have anacron tool that can add some randomness to the timframe when a script is run and many F5 default scripts use this and not the crontab: https://support.f5.com/csp/article/K33730915 3.You can even trigger scripts on the F5 device if the state changes from active to standby or from standby to active by adding the scripts under /config/failover/ . For example if you have a bug for a critical process that causes a failover ( you can use the command show /sys ha-status all-properties to check for this https://support.f5.com/csp/article/K20060182 ) but the device does not reboot or fix the process you can run a script to when the device becomes standby to restart the process. https://support.f5.com/csp/article/K6008 4.You afcource can run scripts at the F5 start time (startup/bootup): https://support.f5.com/csp/article/K11948 5.The final thing thing I can think of is to run a script at the backround that monitors the log and for example when there is a specific message in /var/log/ltm to trigger a tcpdump (in some cases better than creating a rotating tcpdum to catch an issue as per https://support.f5.com/csp/article/K65251607 ). The script can be a bash script with "tail -f" command that is run on the backround or better use the F5 intergrated "icall" feature. Bash: https://www.thegeekstuff.com/2010/12/5-ways-to-execute-linux-command/ Icall: https://devcentral.f5.com/s/articles/what-is-icall-27404 https://devcentral.f5.com/s/articles/run-tcpdump-on-event 5.You can use utility "logger -p" to generate manually log messages in the F5 device's log for testing of your scripts as this is used also for SNMP custom alarm traps tests (for more about SNMP https://support.f5.com/csp/article/K3727 ) https://support.f5.com/csp/article/K86480148 6.You can also trigger scripts from an BIG-IQ device bt you still can't schedule them when to run: https://clouddocs.f5.com/training/community/big-iq-cloud-edition/html/class5/module1/lab6.html 7.Of course the final option is to use ansible or python SDK that uses the F5 rest-api to execute commands on the F5 devices. https://f5-sdk.readthedocs.io/en/latest/ 8. You can even use TCP expect and bash for automations using SSH connection but this is really old way to do things: https://devcentral.f5.com/s/articles/f5-automation-tcl-amp-bash-921 https://f5-sdk.readthedocs.io/en/latest/userguide/ltm_pools_members_code_example.html 9.F5 is well integrated with Ansible and it is better than REST-API Python SDK or TCL for me as even the declarative AS3 interface is supported: https://clouddocs.f5.com/products/orchestration/ansible/devel/ https://clouddocs.f5.com/products/orchestration/ansible/devel/ https://www.f5.com/partners/technology-alliances/ansible Imperative: https://support.f5.com/csp/article/K42420223 https://clouddocs.f5.com/products/orchestration/ansible/devel/usage/playbook_tutorial.html Declaritive: https://www.f5.com/company/blog/f5-as3-and-red-hat-ansible-automation https://clouddocs.f5.com/training/fas-ansible-workshop-101/3.0-as3-intro.html 10.For some automations without rest-api better use the F5 native cli scripts than bash with tmsh commands: https://clouddocs.f5.com/cli/tmsh-reference/v14/modules/cli/cli_script.html https://clouddocs.f5.com/api/tmsh/script__run.html Tested this on version: No Version Found
Quick and dirty shell script to find unused certificates
Problem this snippet solves: This has been edited quite a bit since I first posted so it's probably not as quick and dirty as it was before. This in response to a question regarding removing unused certificateshttps://devcentral.f5.com/questions/how-to-find-the-unused-ssl-certificates-63166 The following bash script will output any installed certificate names to a file, then iterate over each line. If the certificate is not referenced in bigip.conf in either the /config/ or within a partition folder, then it can be reasonably assumed it is not in use and can be safely deleted. The script will give you the option to delete any certs that are not in use and save a UCS archive (just in case) If there are any keys associated with the certificate, this will be deleted too. As the moment, the script will not look for keys without an equivalent cert, e.g. my-cert.key and my-cert.crt. So you many still end up with rouge keys. I'll look to get this updated eventually. There is an array called ignoreCerts ignoreCerts=("f5-irule.crt" "ca-bundle.crt") Here you can add certificates you may want to ignore. For example, f5-irule.crt is used to sign F5 provided iRules and bigip.conf does not reference it. Add any additional certs to this array to ensure they are not deleted Script can be downloaded directly from GitLab using the link below https://gitlab.com/stratalabs/f5-devcental/snippets/1863498/raw?inline=false How to use this snippet: paste into vi chmod +x myScript.sh ./myScript.sh Code : #!/bin/sh function buildInstalledCertsArray { tmsh save sys config partitions all tmsh list sys file ssl-cert | awk '/crt/ {print $4}' | sed '/^[[:space:]]*$/d' > /var/tmp/installedCerts.tmp # iterate over tmp file to create array of used certificates while read line; do for i in "${!ignoreCerts[@]}"; do if [[ $line = ${ignoreCerts[$i]} ]]; then ignore="true" else if [[ $ignore != "true" ]];then ignore="" else # do not add cert to array if already added if [[ ! " ${instCertsArr[@]} " =~ " ${line} " ]]; then instCertsArr+=("$line") fi fi fi done done /dev/null 2>&1) if ! [ -z "$hasKey" ];then deleteKeys+=("${cert%.*}.key") fi done } function deleteUnusedCerts { if [ ${#deleteCerts[@]} -eq 0 ]; then echo "-------------------------------------------------------------------------" echo "There are no unused certificates to delete, existing" echo "-------------------------------------------------------------------------" exit 0 else echo "-------------------------------------------------------------------------" echo "The following keys are not in use can can be deleted:" for cert in "${deleteCerts[@]}"; do echo " ${cert}" done echo "-------------------------------------------------------------------------" read -p "would you like to delete these unused certificates? (y/n)?" answer case ${answer:0:1} in y|Y ) createUcsArchive echo "-------------------------------------------------------------------------" echo "deleting certs..." for cert in "${deleteCerts[@]}"; do delete sys file ssl-key $cert echo " $cert" done if [ ${#deleteKeys[@]} -eq 0 ]; then echo "-------------------------------------------------------------------------" echo "no associated keys to delete, exiting" exit 0 else echo "-------------------------------------------------------------------------" echo "deleting keys..." for key in "${deleteKeys[@]}"; do delete sys file ssl-key $cert echo "$key" exit 0 done fi ;; * ) exit 0 ;; esac fi } function createUcsArchive { echo today=`date +%Y-%m-%d.%H.%M.%S` echo "Creating UCS archive auto.${today}.ucs" tmsh save sys ucs ${today}.ucs } # initialise vars instCertsArr=() deleteCerts=() # ignore certs defined here - f5-irile.crt is used to sign F5 iRules ignoreCerts=("f5-irule.crt" "ca-bundle.crt") # build installed certificates array - excluding certs to ignore buildInstalledCertsArray # check if installed certs are used in bigip.conf (including partitions) - ltm sys files are exluded from results buildDeleteCertsArray # build list of associated keys (not all certs will have keys) buildDeleteKeysArray # optionally delete unused certs deleteUnusedCerts Tested this on version: No Version Foundf5-admin Python API Release
Problem this snippet solves: A complete F5 Admin API Python package. This is built for F5 security administrators. It works by providing a direct hook to the F5 TMOS Command Line Interface. Then you can manage F5 configuration objects by using your Python-Fu. Furthermore, the security admin can go to the next level of the playing field, by custom buildout the automation procedures in the full-stack capable, object-oriented Python programming language from there. Real world examples developed to solve my daily F5 secure admin chores can be found under the 'src/bin' folder. Enjoy. How to use this snippet: # !/usr/bin/env python # Let's build a tool together import f5_admin # First we need to let Python know we're going to use the API f5_node = "xx-rhwebdev1" # The F5 node we want to connect to. f5_command = "show sys software" # The F5 command we want to run with f5_admin.F5Client(None,None,None) as client: client.load(f5_node) # Now we're ready to open a remote connection result=client.ssh_command(client.ssh_connect(),f5_command,"") for line in result: # line print the F5 command result in the console print(line) # Congratulation. You just build an automation tool with the API! Test Run the Tool Copy the above code block in a text editor. Then save it into a file name 'test.py'. OrDownload itas raw file. Then you can test run the tool as below: $ python test.py Loading cache_config: /Library/Python/2.7/site-packages/f5_admin-1.0.0-py2.7.egg/f5_admin/conf/xx-rhwebdev1/xx-rhwebdev1.txt Loading complete Setting up remote SSH session to host: xx-rhwebdev1 Please enter the F5 user name: Please enter the F5 password: Execution on the remote SSH host: show sys software Command execution complete. ---------------------------------------------------------- Sys::Software Status Volume Slot Product Version Build Active Status Congratulation! You just build your first tool using this Python package. Please use the url below for more usage guidelines: https://github.com/yangsec888/f5-admin/blob/master/API.md Tested this on version: 13.0
