Generate private key w/ CSR via iControl REST
Problem this snippet solves: Generate a private key w/ CSR How to use this snippet: To create a private key with a CSR via iControl REST: POST URL:https://10.1.1.165/mgmt/tm/sys/crypto/key Use the data below as your payload. For the name field, it must end in .key or you will get a false 404! Code : { "name":"www.testing.com.key", "commonName":"www.testing.com", "keySize":"4096", "keyType":"rsa-private", "options":[{"gen-csr":"www.testing.com"}], "organization":"Let It Snow Corp.", "ou":"Ice Engineering", "city":"Calhoun", "state":"AZ", "admin-email-address":"jerry@letit.snow", "email-address":"beth@letit.snow", "subject-alternative-name":"DNS:www.testing.com", "challenge-password":"myP4ssword" } Tested this on version: 13.0
Connection list via iControlREST API
Problem this snippet solves: This python script will retrieve a filtered list of active connections via the iControl REST API. It supports both the default connection list as well as the detailed list you get by specifying 'all-properties'. It has options to generate output on STDOUT as displayed by TMSH (raw), as a JSON formatted dictionary, or written to an Excel spreadsheet. Currently tested in python 2.7 and python 3.5 against BIGIP 11.6.0. How to use this snippet: $ ./conn-list.py -h usage: conn-list.py [-h] [-a] [-x FILE | -j | -r] -f P=V host positional arguments: host Host name to connect. Specified as [<username>@]<hostname> optional arguments: -h, --help show this help message and exit -a, --all-properties Get detailed connection information -f P=V, --filter P=V You must have at least one filter argument, but may have multiple. Output Type: -x FILE, --xlout FILE Excel workbook to be created. -j, --json JSON formatted output to STDOUT -r, --raw RAW ouptut from API request (default) P=V: P = Connection Property (below), V = Value to match. Multiple -f options are joined as logical AND. age Specifies the age, in seconds, of a connection cs-client-addr Specifies the clientside remote address of the active connections cs-client-port Specifies the clientside remote port of the active connections cs-server-addr Specifies the clientside local address of the active connections cs-server-port Specifies the clientside local port of the active connections protocol Specifies the protocol used for specified connections (for example: tcp, udp) ss-client-addr Specifies the serverside local address of the active connections ss-client-port Specifies the serverside local port of the active connections ss-server-addr Specifies the serverside remote address of the active connections ss-server-port Specifies the serverside remote port of the active connections type Specifies the connnection type used for specified connections (for example: any, mirror, self) $ ./conn-list.py -r -f cs-server-port=80 admin@192.0.2.45 Password: Sys::Connections 192.0.2.31:55345 192.0.2.20:80 192.0.2.31:55345 198.51.100.66:80 tcp 6 (tmm: 1) none Total records returned: 1 $ ./conn-list.py -j -f cs-server-port=80 admin@192.0.62.45 Password: [{"acceleration": "none", "cs-server": "192.0.2.20:80", "protocol": "tcp", "cs-client": "192.0.2.31:55613", "idle": 1, "ss-server": "198.51.100.66:80", "tmm": 1, "ss-client": "192.0.2.31:55613"}] Code : #!/usr/bin/env python """ This script will use F5's iControl REST API to collect current connection data. It enforces the use a at least one filter criteria. This has only been tested on 11.6.0 so far. Should be extendable to 11.5.x and 12.x by adding to or duplicating Record Definitions and RE sections. """ import json import sys import getpass import argparse import re try: # Py3 from urllib.parse import urlparse, parse_qs except ImportError: # Py2 from urlparse import urlparse, parse_qs import requests from openpyxl import Workbook RE_LAST = re.compile('(\w+)$') RE_LAST_TWO = re.compile('([\w/:\.]+)\s+([\w/:\.]+)$') DETAILED_DEF = { '11.6.0': {'rec-sep': '---', # actually second line in record. 'fields': {'Slot': {'re': RE_LAST, 'id': ['slot']}, 'TMM': {'re': RE_LAST, 'id': ['tmm']}, 'Acceleration': {'re': RE_LAST, 'id': ['acceleration']}, 'Protocol': {'re': RE_LAST, 'id': ['protocol']}, 'Idle Time ': {'re': RE_LAST, 'id': ['idle']}, 'Idle Timeout': {'re': RE_LAST, 'id': ['idle_timeout']}, 'Lasthop': {'re': RE_LAST_TWO, 'id': ['lasthop-vlan', 'lasthop-mac']}, 'Client Addr': {'re': RE_LAST_TWO, 'id': ['cs-client', 'ss-client']}, 'Server Addr': {'re': RE_LAST_TWO, 'id': ['cs-server', 'ss-server']}, 'Bits In': {'re': RE_LAST_TWO, 'id': ['cs-bits-in', 'ss-bits-in']}, 'Bits Out': {'re': RE_LAST_TWO, 'id': ['cs-bits-out', 'ss-bits-out']}, 'Packets In': {'re': RE_LAST_TWO, 'id': ['cs-packets-in', 'ss-packets-in']}, 'Packets Out': {'re': RE_LAST_TWO, 'id': ['cs-packets-out', 'ss-packets-out']}, } } } SHRT_RE_AP = \ re.compile( '([\w:\.]+)\s+([\w:\.]+)\s+([\w:\.]+)\s+([\w:\.]+)\s+(\w+)\s+(\d+)\s+\(tmm: (\d+)\)\s+(\w+)') SHRT_RE_CH = \ re.compile( '([\w:\.]+)\s+([\w:\.]+)\s+([\w:\.]+)\s+([\w:\.]+)\s+(\w+)\s+(\d+)\s+\(slot/tmm: (\d+)/(\d+)\)\s+(\w+)') SHORT_DEF = { '11.6.0': {'chs-indc': 'slot', 'fields': {'appliance': {'re': SHRT_RE_AP, 'id': ['cs-client', 'cs-server', 'ss-client', 'ss-server', 'protocol', 'idle', 'tmm', 'acceleration']}, 'chassis': {'re': SHRT_RE_CH, 'id': ['cs-client', 'cs-server', 'ss-client', 'ss-server', 'protocol', 'idle', 'slot', 'tmm', 'acceleration']}, } } } requests.packages.urllib3.disable_warnings() try: dict.iteritems except AttributeError: # Py3 def itervalues(d): return iter(d.values()) def iteritems(d): return iter(d.items()) else: # Py2 def itervalues(d): return d.itervalues() def iteritems(d): return d.iteritems() def main(): parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter) parser.add_argument('-a', '--all-properties', dest='detail', action='store_true', help='Get detailed connection information') outype = parser.add_argument_group("Output Type") outputs = outype.add_mutually_exclusive_group() outputs.add_argument('-x', '--xlout', action='store', metavar="FILE", help='Excel workbook to be created.') outputs.add_argument('-j', '--json', action='store_true', help='JSON formatted output to STDOUT') outputs.add_argument('-r', '--raw', action='store_true', help='RAW ouptut from API request (default)') parser.add_argument('-f', '--filter', required=True, action='append', metavar='P=V', help='You must have at least one filter argument, but may have multiple.') parser.add_argument('host', help='Host name to connect. Specified as [ @] ') parser.epilog = """ P=V: P = Connection Property (below), V = Value to match. Multiple -f options are joined as logical AND. age Specifies the age, in seconds, of a connection cs-client-addr Specifies the clientside remote address of the active connections cs-client-port Specifies the clientside remote port of the active connections cs-server-addr Specifies the clientside local address of the active connections cs-server-port Specifies the clientside local port of the active connections protocol Specifies the protocol used for specified connections (for example: tcp, udp) ss-client-addr Specifies the serverside local address of the active connections ss-client-port Specifies the serverside local port of the active connections ss-server-addr Specifies the serverside remote address of the active connections ss-server-port Specifies the serverside remote port of the active connections type Specifies the connnection type used for specified connections (for example: any, mirror, self) """ args = parser.parse_args() (username, unused, host) = args.host.rpartition('@') if not username: username = raw_input('Username: ') password = getpass.getpass('Password: ') filter_prop = '+'.join(args.filter) filter_prop = filter_prop.replace('=', '+') (ctext, ver) = get_conn_list(host, username, password, filter_prop, args.detail) if not ctext: print('No connection list was returned from {}.'.format(host)) quit() if not args.xlout and not args.json: raw_output(ctext) else: if args.detail: conns, patterns = process_detailed_conns(ctext, ver) else: conns, patterns = process_short_conns(ctext, ver) if args.xlout: excel_output(conns, args.xlout, patterns) elif args.json: json_output(conns) # noinspection PyBroadException def get_conn_list(host, uname, pw, options, detail): conntext = [] version = '' b = requests.session() b.auth = (uname, pw) b.verify = False b.headers.update({'Content-Type': 'application/json'}) b_url = 'https://{}/mgmt/tm'.format(host) if detail: options += "+all-properties" try: resp = b.get(b_url + '/sys/connection/?options=' + options, timeout=6.05) if resp.status_code == requests.codes.ok: j = json.loads(resp.text) version = parse_qs(urlparse(j['selfLink']).query)['ver'][0] conntext = j['apiRawValues']['apiAnonymous'].splitlines() else: sys.stderr.write( 'Error: {} status returned from: {}\n {}\n'.format(resp.status_code, host, resp.reason)) except: sys.stderr.write('Error: Could not get data from {}: {}\n'.format(host, sys.exc_info()[0])) return conntext, version def process_detailed_conns(conntext, ver): connlist = [] patterns = {} detailed_record_length = 0 more_than_one_record = False for ndx, val in enumerate(conntext): if val.startswith(DETAILED_DEF[ver]['rec-sep']): if detailed_record_length == 0: detailed_record_length = ndx else: detailed_record_length = ndx - detailed_record_length more_than_one_record = True break else: for p in DETAILED_DEF[ver]['fields']: if val.lstrip().startswith(p): patterns[ndx - 1] = DETAILED_DEF[ver]['fields'][p] break if not more_than_one_record: # only one record returned detailed_record_length = ndx - 1 ndx = 0 while ndx < len(conntext) - 1: if ndx % detailed_record_length == 1: obj = {} for offset, data_def in iteritems(patterns): match = data_def['re'].search(conntext[ndx + offset]) for ndx2, col_heading in enumerate(data_def['id']): grouping = match.group(ndx2 + 1) obj[col_heading] = int(grouping) if grouping.isnumeric() else grouping connlist.append(obj) ndx += detailed_record_length else: ndx += 1 return connlist, patterns def process_short_conns(ctext, ver): connlist = [] patterns = {} if len(ctext) > 2: if SHORT_DEF[ver]['chs-indc'] in ctext[1]: patterns = SHORT_DEF[ver]['fields']['chassis'] else: patterns = SHORT_DEF[ver]['fields']['appliance'] for row in ctext: obj = {} match = patterns['re'].search(row) if match: for ndx, col_heading in enumerate(patterns['id']): grouping = match.group(ndx + 1) if grouping: obj[col_heading] = int(grouping) if grouping.isnumeric() else grouping connlist.append(obj) return connlist, {1: patterns} def excel_output(conns, fname, patterns): wb = Workbook() ws = wb.active header = [] for h in patterns.values(): header += h['id'] ws.append(header) for r in conns: row = [] for h in header: row.append(r[h]) ws.append(row) if not fname.endswith('.xlsx'): fname += '.xlsx' wb.save(fname) def json_output(conns): print(json.dumps(conns)) def raw_output(data): for l in data: print(l) if __name__ == '__main__': main() Tested this on version: 11.6Merge BIG-IP Config Files
Problem this snippet solves: This script will take a local BIG-IP configuration file, upload it to the BIG-IP, merge it with the existing configuration, then clean up the iControl REST upload folder (/var/config/rest/downloads) by removing the file. How to use this snippet: macdaddy:scripts jrahm$ python merge_config.py usage: merge_config.py [-h] host username filepath Code : def _upload(host, creds, fp): chunk_size = 512 * 1024 headers = { 'Content-Type': 'application/octet-stream' } fileobj = open(fp, 'rb') filename = os.path.basename(fp) if os.path.splitext(filename)[-1] == '.iso': uri = 'https://%s/mgmt/cm/autodeploy/software-image-uploads/%s' % (host, filename) else: uri = 'https://%s/mgmt/shared/file-transfer/uploads/%s' % (host, filename) requests.packages.urllib3.disable_warnings() size = os.path.getsize(fp) start = 0 while True: file_slice = fileobj.read(chunk_size) if not file_slice: break current_bytes = len(file_slice) if current_bytes < chunk_size: end = size else: end = start + current_bytes content_range = "%s-%s/%s" % (start, end - 1, size) headers['Content-Range'] = content_range requests.post(uri, auth=creds, data=file_slice, headers=headers, verify=False) start += current_bytes def _merge_config(host, creds, file): requests.packages.urllib3.disable_warnings() b_url = 'https://%s/mgmt/tm/sys/config' % host b = requests.session() b.auth = creds b.verify = False b.headers.update({'Content-Type': 'application/json'}) options = {} options['file'] = '/var/config/rest/downloads/%s' % file options['merge'] = True payload = {} payload['command'] = 'load' payload['options'] = [options] try: merge = b.post(b_url, json.dumps(payload)) if merge.status_code is not 200: print "Merge failed, check rest log file" exit() except Exception, e: print e def _cleanup_mergefile(host, creds, file): requests.packages.urllib3.disable_warnings() b_url = 'https://%s/mgmt/tm/util/unix-rm' % host b = requests.session() b.auth = creds b.verify = False b.headers.update({'Content-Type': 'application/json'}) payload = {} payload['command'] = 'run' payload['utilCmdArgs'] = '/var/config/rest/downloads/%s' % file try: cleanup = b.post(b_url, json.dumps(payload)) if cleanup.status_code is not 200: print "Cleanup failed, please check system." except Exception, e: print e if __name__ == "__main__": import os, requests, json, argparse, getpass requests.packages.urllib3.disable_warnings() parser = argparse.ArgumentParser(description='Merge a config file into BIG-IP config') parser.add_argument("host", help='BIG-IP IP or Hostname', ) parser.add_argument("username", help='BIG-IP Username') parser.add_argument("filepath", help='Merge file (with Absolute Path)') args = vars(parser.parse_args()) hostname = args['host'] username = args['username'] filepath = args['filepath'] print "%s, enter your password: " % args['username'], password = getpass.getpass() _upload(hostname, (username, password), filepath) filename = os.path.basename(filepath) _merge_config(hostname, (username, password), filename) _cleanup_mergefile(hostname, (username, password), filename) Tested this on version: 12.0
python f5-sdk - Reverse Lookup (Node -> Pool)
Problem this snippet solves: This python bigsuds script prints the list of pools using a specific node. How to use this snippet: rlookup-node.py <hostname> <username> <nodename> Script will prompt for password. This will only search the Common partition. Code : #!/usr/bin/env python __author__ = 'buzzsurfr' __version__ = '0.2' # Standard Library import sys import re # Related Third-Party import getpass # Local Application/Library Specific from f5.bigip import ManagementRoot if len(sys.argv) < 4: print "\n\n\tUsage: %s host user node" % sys.argv[0] sys.exit() # Get login password from CLI userpass = getpass.getpass() # Connect to BIG-IP mgmt = ManagementRoot(sys.argv[1], sys.argv[2], userpass) # Get list of pools and pool members pools = mgmt.tm.ltm.pools.get_collection() # Node to search for node = sys.argv[3] if len(node) < 8 or node[:8] != '/Common/': node = '/Common/'+node print "Pools using Node "+node # Iterate through pool member list (has a list of members per pool referenced) looking for node for pool in pools: member_nodes = [member.fullPath.split(':')[0] for member in pool.members_s.get_collection()] if node in member_nodes: print "\t"+pool.name Tested this on version: 11.5python f5-sdk - Reverse Lookup (Pool -> Virtual Server)
Problem this snippet solves: This python f5-sdk script prints the list of pools using a specific pool. How to use this snippet: rlookup-pool.py <hostname> <username> <poolname> Script will prompt for password. This will only search the Common partition. This also does not check for policies or iRules that may change the value of pool. Code : #!/usr/bin/env python __author__ = 'buzzsurfr' __version__ = '0.2' # Standard Library import sys import re # Related Third-Party import getpass # Local Application/Library Specific from f5.bigip import ManagementRoot if len(sys.argv) < 4: print "\n\n\tUsage: %s host user pool" % sys.argv[0] sys.exit() # Get login password from CLI userpass = getpass.getpass() # Connect to BIG-IP mgmt = ManagementRoot(sys.argv[1], sys.argv[2], userpass) # Pool to search for pool = sys.argv[3] if len(pool) < 8 or pool[:8] != '/Common/': pool = '/Common/'+pool print "Virtual Servers using Pool "+pool # Get list of virtual servers virtual_servers = mgmt.tm.ltm.virtuals.get_collection() # Iterate through pool member list (has a list of members per pool referenced) looking for node for vs in virtual_servers: if pool == vs.pool: print "\t"+vs.name Tested this on version: 11.5
Python module to post and retrieve IControl Rest JSON objects for AVR statistics
Problem this snippet solves: This module simplifies making Python dictionary objects that are converted to IControl rest AVR JSON objects. It also handles making AVR requests and retrieving results as well allowing multiple AVR requests to be queued, posted and retrieved. It also has some basis type checking for the elements of a AVR request. This module requires Bigip 12.1 on the target that statistics are retrieved. How to use this snippet: The main class is rest_avr.avr_req. It is a dictionary class that maps directly to an IControl Rest AVR JSON request as translated by json.dumps. Each dictionary element is an object derived from a customer class for each part of the request. The element classes have add() and clear() functions. if the element class only allows one entry the add() function will replace the existing entry, otherwise it will append the entry to the request element. The rest_avr.avr_req class also has functions to populate the HTTP host and authentication values for the target system. rest_avr.avr_req.post_and_response returns the Python representation of the JSON result of the query. rest_avr.avr_req.add_to_queue() adds the currently constructed request to a queue of requests to post. rest_avr.avr_req.post_and_response_queue() returns a python list of results of queued queries. The following code sample constructs, posts and returns results for an AVR statistics request for specific DNS records and a specificrecord type, then queues multiple quests and posts and returns results. #!/usr/bin/python import json import sys import time import rest_avr #print rest_avr.ShowAVRJsonApi #Populate the url avr_dns_req=rest_avr.avr_req() avr_dns_req.auth('admin','admin') avr_dns_req.url_base('10.10.2.113','dns') #Populate the json object avr_dns_req['analyticsModule'].add('dns') avr_dns_req['reportFeatures'].add('time-aggregated') avr_dns_req['entityFilters'].add('domain-name', 'OPERATOR_TYPE_EQUAL', ['test2.test1.com','test1.test1.com']) avr_dns_req['entityFilters'].add('query-type', 'OPERATOR_TYPE_EQUAL', ['a']) avr_dns_req['viewMetrics'].add('packets') avr_dns_req['viewDimensions'].add('domain-name') avr_dns_req['metricFilters'].add('packets', 'OPERATOR_TYPE_GREATER_THAN', 0) avr_dns_req['sortByMetrics'].add('packets', 'ascending') avr_dns_req['pagination'].add(20, 0) avr_dns_req['timeRange'].add(1461778251000000, None) #Post and retrieve results. result_py=avr_dns_req.post_and_response() if result_py != None: print ('\n' + result_py['results']['timeAggregated'][0]['dimensions'][0]['value'] + " " + result_py['results']['timeAggregated'][0]['metricValues'][0]['value'] + '\n') else: print result_py.error_layer print result_py.error_code print result_py.error_text # Now add multiple requests to a queue avr_dns_req.add_to_queue() avr_dns_req['entityFilters'].clear() avr_dns_req['entityFilters'].add('query-type', 'OPERATOR_TYPE_EQUAL', ['aaaa']) avr_dns_req.add_to_queue() #post and retrieve queued results result_py_q=avr_dns_req.post_and_response_queue() for result_py in result_py_q: if result_py != None: print ('\n' + result_py['results']['timeAggregated'][0]['dimensions'][0]['value'] + " " + result_py['results']['timeAggregated'][0]['metricValues'][0]['value'] + '\n') else: print result_py.error_layer print result_py.error_code print result_py.error_text Code : """ rest_avr provides a python interface to Bigip AVR statistics using the REST API. The main Python rest_avr.avr_req object is a Python dictionary that maps to a JSON object that can be processed with the json.dumps() function An IControl Rest AVR JSON request and response can be initiated with avr_req.post_and_response The simple description of the API can is available at avr_req.ShowJsonApi() Each of these modules has a method to add single or multiple elements as appropriate to the specific module. Once these elements are are populated a RestAPI request can be made with results returned as a python representation. avr_req.auth(user, passw) avr_req.url_base(host, module) avr_req['analyticsModule'].add(module) avr_req['analyticsModule'].clear() avr_req['reportFeatures'].add(metric_name, predicate, value) avr_req['reportFeatures'].clear() avr_req['entityFilters'].add(dimension_name, predicate, values) avr_req['entityFilters'].clear() avr_req['viewMetrics'].add(metric_name) avr_req['viewMetrics'].clear() avr_req['viewDimensions'].add(metric_name, order) avr_req['viewDimensions'].clear() avr_req['metricFilters'].add(metric_name, predicate, valu) avr_req['metricFilters'].clear() avr_req['sortByMetrics'].add(metric_name, orde) avr_req['sortByMetrics'].clear() avr_req['pagination'].add(num_results, skip_result) avr_req['pagination'].clear() avr_req['timeRange'].add(t_from, t_to) avr_req['timeRange'].clear() After a request in constructed a REST API call is initiated with initiated with: avr_req.post_and_response() The response is a python dictionary data structure of the results as processed by json.loads """ from copy import deepcopy import requests import json import sys import time import warnings __author__ = 'Mark Lloyd' __version__ = '1.0' # 05/24/2016 import json import requests import time class BadDictElement(Exception): def __init__(self, key, value, expl): Exception.__init__(self, '{0} {1} {2} '.format(key, value, expl)) class BadTime(Exception): def __init__(self, variable, value): Exception.__init__(self, '{0} {1} should be 16 char decimal in microseconds '.format('a', 'b')) class RequestFailure(Exception): def __init__(self, key, value): Exception.__init__(self, '{0} {1} '.format(key, value)) class analyticsModule(str): """ This class is tied to the structure of the parent class. parent() get's the parent object so we can make the string pseudo-mutable. accessed from within an avr request ['analyticsModule'].add(module) Adds a single string to analyticsModule element . If one exists it is replaced. ['analyticsModule'].clear() Send a null value to the analyticsModule element. See rest_avr.ShowAVRJsonApi for more details """ def parent(self, parent): self.parent = parent def add(self, module): """ avr_req.['analyticsModule'].add(module) Adds a single string to analyticsModule element . If one already exists it is replaced. This should be the same as the module string in avr_req.url_base. """ self.parent['analyticsModule'] = analyticsModule(module) self.parent['analyticsModule'].parent = self.parent def clear(self): """ avr_req.['analyticsModule'].add(module) replaces the analyticsModule mddule with a null string """ self.parent['analyticsModule'] = analyticsModule('') self.parent['analyticsModule'].parent = self.parent class metricFilters(list): """ avr_req.['metricFilters'].add(metric_name, predicate, value) metric name is a string, value is an integer Valid predicates strings are ['OPERATOR_TYPE_EQUAL', 'OPERATOR_TYPE_NOT_EQUAL', 'OPERATOR_TYPE_GREATER_THAN', OPERATOR_TYPE_LOWER_THAN','OPERATOR_TYPE_GREATER_THAN_OR_EQUAL', 'OPERATOR_TYPE_LOWER_THAN_OR_EQUAL']) avr_req['metricFilters'].clear() Clears metricFilters elements See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.append([]) self.valid_metric_predicate = ( ['OPERATOR_TYPE_EQUAL', 'OPERATOR_TYPE_NOT_EQUAL', 'OPERATOR_TYPE_GREATER_THAN', 'OPERATOR_TYPE_LOWER_THAN', 'OPERATOR_TYPE_GREATER_THAN_OR_EQUAL', 'OPERATOR_TYPE_LOWER_THAN_OR_EQUAL']) def add(self, metric_name, predicate, value): """ avr_req.['metricFilters'].add(metric_name, predicate, value) metric name is a string, value is an integer Valid predicates strings are ['OPERATOR_TYPE_EQUAL', 'OPERATOR_TYPE_NOT_EQUAL', 'OPERATOR_TYPE_GREATER_THAN', OPERATOR_TYPE_LOWER_THAN','OPERATOR_TYPE_GREATER_THAN_OR_EQUAL', 'OPERATOR_TYPE_LOWER_THAN_OR_EQUAL'] """ if type(value) is not int: raise BadDictElement(metric_name, value, 'value should be integer') if predicate in self.valid_metric_predicate: # first check if it is already there for metric in self[0]: if metric['metricName'] == metric_name: metric['predicate'] = predicate metric['value'] = value return 0 # if it is not there then just add it. self[0].append({'metricName': metric_name, 'predicate': predicate, 'value': value}) else: raise BadDictElement(metric_name, predicate, 'invalid predicate') def clear(self): """ avr_req['metricFilters'].clear() Clears metricFilters elements """ del self[0][:] class entityFilters(list): """ avr_req.['entityFilters'].add(dimension_name, predicate, values): All values are strings valid predicate is 'OPERATOR_TYPE_EQUAL' ['entityFilters'].clear() Clears the entityFilters element See rest_avr.ShowJsonApi for more details """ def __init__(self): self.append([]) def add(self, dimension_name, predicate, values): """ avr_req.['entityFilters'].add(dimension_name, predicate, values): All values are strings valid predicate is 'OPERATOR_TYPE_EQUAL' """ if predicate is 'OPERATOR_TYPE_EQUAL': # then loop throuth to see if the dimenson name already exists, if so replace for entity in self[0]: if entity['dimensionName'] == dimension_name: entity['predicate'] = predicate entity['values'] = values return 0 # if it is not there then just add it. self[0].append({'dimensionName': dimension_name, 'predicate': predicate, 'values': values}) else: raise BadDictElement(dimension_name, predicate, 'predicate must be OPERATOR_TYPE_EQUAL') def clear(self): """ ['entityFilters'].clear() Clears the entityFilters element """ del self[0][:] class reportFeatures(list): """ avr_req.['reportFeatures'].add( feature) adds report feature string. Multiple features are permitted. ['reportFeatures'].clear() Clears the analyticsModule element. See rest_avr.ShowAVRJsonApi for more details. """ def add(self, feature): """ avr_req.['reportFeatures'].add( feature) adds report feature string. Multiple features are permitted .""" if feature not in self: self.append(feature) def clear(self): """ ['reportFeatures'].clear() Clears the entityFilters element """ del self[:] class sortByMetrics(list): """ avr_req.['sortByMetrics'].add(metric_name, order) valid order names are 'ascending' and 'descending' sortByMetrics is optional in an AVR request. avr_req['sortByMetrics'].clear() Clears the sortByMetrics element. See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.metric_list = [] def add(self, metric_name, order): if metric_name not in self.metric_list: self.append({'metricName': metric_name, 'order': order}) self.metric_list.append(metric_name) def clear(self): """ ['sortByMetrics'].clear() Clears the sortByMetrics element """ del self[:] del self.metric_list[:] class viewDimensions(list): """ avr_req.['viewDimensions'].add(dimension_name): adds view dimension, only one dimension is allowed add will replace element if it already exists avr_req['viewDimensions'].clear() Clears the viewDimensions element. See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.append([]) self[0] = {} def add(self, dimension_name): """ avr_req.['viewDimensions'].add(dimension_name): adds view dimension string, only one dimension is allowed add will replace element if it already exists """ self[0]['dimensionName'] = dimension_name def clear(self, dimension_name): """ ['viewDimensions'].clear() Clears the viewDimensions element """ del self[0][:] class viewMetrics(list): """ avr_req.['viewMetrics'].add(metric_name): appends metric_name string to list. The specification allows multiple view metric elements avr_req['viewMetrics'].clear() Clears the viewMetrics elements See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.metric_list = [] def add(self, metric_name): """ avr_req.['viewMetrics'].add(metric_name): appends metric_name string to list. The specification allows multiple viewMetric elements """ if metric_name not in self.metric_list: self.append({'metricName': metric_name}) self.metric_list.append(metric_name) def clear(self): """ ['viewMetrics'].clear() Clears the viewMetrics elements """ del self[:] del self.metric_list[:] class timeRange(dict): """ avr_req.['timeRange'].add( t_from, t_to) both values are 16 digit numeric value in microseconds of unix/linux time. t_to is optional and can be replace by None timeRange is an optional. avr_req['timeRange'].clear() Clears the timeRange elements See rest_avr.ShowAVRJsonApi for more details. """ def add(self, t_from, t_to): """ avr_req.['timeRange'].add( t_from, t_to) both values are 16 digit numeric value in microseconds of unix/linux time. t_to is optional and can be replace by None timeRange is optional. """ if type(t_from) is long and len(str(t_from)) == 16: self['from'] = t_from else: raise BadTime(t_from + " is 16 digit numeric value in microseconds") if t_to != '' and t_to != 0 and t_to != None: if type(t_to) is long and len(str(t_from)) == 16: self['to'] = t_to else: raise BadTime(t_to + " is 16 digit numeric value in microseconds") else: if 'to' in self.keys(): del self['to'] def clear(self): """ ['timeRange'].clear() Clears the timeRange element """ del self[:] class pagination(dict): """ avr_req.['pagination'].add(num_results, skip_results) both are integer values. avr_req['pagination'].clear() Clears the pagination elements See rest_avr.ShowAVRJsonApi for more details. """ def add(self, num_results, skip_results): """ avr_req.['pagination'].add(num_results, skip_results) both arguments are integers. """ if type(num_results) is int: self['numberOfResults'] = num_results else: raise BadDictElement('number of Results ', num_results, 'must be integer') if type(skip_results) is int: self['skipResults'] = skip_results else: raise BadDictElement('skipResults ', skip_results, 'must be integer') def clear(self): """ ['pagination'].clear() Clears the pagination element """ del self[:] class avr_resp(dict): """ python response error is applicable. """ def __init__(self): self.error_layer = None self.error_code = None self.error_text = None class avr_req(dict): """ The main class for rest_avr. avr_req contains a dictionary that maps to the elements of a Icontrol REST AVR request along with capability of posting that request and receiving a response. The dictionary values are object instances of python classes that correspond to the the JSON values of the object's name/value pair. Each value has two public methods: avr_req.['objectName']add(): adds an element to the appropriate object with type checking. If an element allows more then one instance the add function will append the element If an element allows only one instance the add function will replace the element avr_req.['objectName'].clear()r: clears all elements in the object. printing rest_avr.ShowAVRJsonApi provides documentation for the AVR JASON elements. Further documentation is available on devcentral.f5.com To post an AVR Rest request there are two functions to populate the HTTP/HTTPS request. avr_req.auth(user, passw): provides the username and password avr_req.url_base(host, module) provides the host and the bigip module AVR queries to construct the URL to make the request. Then to post the request and return results in a python representation of the JSON response. avr_req.post_and_response() """ def __init__(self): self['analyticsModule'] = analyticsModule() self['analyticsModule'].parent = self self['pagination'] = pagination() self['metricFilters'] = metricFilters() self['entityFilters'] = entityFilters() self['reportFeatures'] = reportFeatures() self['sortByMetrics'] = sortByMetrics() self['viewDimensions'] = viewDimensions() self['viewMetrics'] = viewMetrics() self['timeRange'] = timeRange() self.avr_session = requests.session() self.avr_session.verify = False self.avr_session.headers.update({'Content-Type': 'application/json'}) # for multiple queued request handling. self.req_queue = [] self.generate_id = None self.done = None self.result = None self.num_requests = 0 self.res_queue = [] def post_and_response(self): """ returns a python representation of the json response to the request. failure returns array ['ERROR','component',error] """ warnings.filterwarnings("ignore") self.generate_request = self.avr_session.post(self.req_url_base + "/generate-report/", data=json.dumps(self)) self.generate_request_py = json.loads(self.generate_request.text) self.result_guid = self.generate_request_py['id'] self.results_status_url = self.req_url_base + "/generate-report/" + self.result_guid + "/?$select=status,reportResultsLink" self.results_url = self.req_url_base + "/report-results/" + self.result_guid self.sleeptime = .5 for i in range(5): time.sleep(self.sleeptime) self.sleeptime *= 2 # double backoff period each time. self.status_results_json = self.avr_session.get(self.results_status_url) self.status_results = json.loads(self.status_results_json.text) if self.status_results['status'] == 'FAILED': self.result = avr_resp() self.result_error_layer = 'REST' self.result_error_code = self.status_results['status'] self.result.error_text = self.status_results if self.status_results['status'] == 'FINISHED': self.raw_results_url = self.status_results['reportResultsLink'] self.results_url = self.raw_results_url.replace('localhost', self.host_name) self.results = self.avr_session.get(self.results_url) if self.results.status_code == 200: self.result = avr_resp() self.result.update(json.loads(self.results.text)) return self.result else: self.result = avr_resp() self.result.error_layer = 'HTTP' self.result.error_code = self.results.status_code self.result.error_text = self.results return self.result else: continue self.result = avr_resp() self.result.error_layer = 'REST_AVR' self.result.error_code = '408' self.result.error_text = 'TIMEOUT' def auth(self, user, passw): """ avr_req.auth(user, passw): username and password """ self.avr_session.auth = (user, passw) def url_base(self, host, module): """ avr_req.url_base(host, module) host and bigip module AVR queries to construct the URL to make the request. """ self.host_name = host self.req_url_base = 'https://%s/mgmt/tm/analytics/%s' % (host, module) self.module_py = {'analyticsModule': module} def add_to_queue(self): "adds request as currently constructed to queue" self.req_queue.append(deepcopy(self)) def clear_queue(self): """" clears request queue """ del self.req_queue[:] def post_and_response_queue(self): """ posts and sends response to from queue of requests. """ warnings.filterwarnings("ignore") for req in self.req_queue: req.generate_request = req.avr_session.post(req.req_url_base + "/generate-report/", data=json.dumps(req)) req.generate_request_py = json.loads(req.generate_request.text) req.generate_id = (req.generate_request_py['id']) req.results_status_url = self.req_url_base + "/generate-report/" + req.generate_id + "/?$select=status,reportResultsLink" self.sleeptime = .5 self.num_requests = len(self.req_queue) for i in range(5): for req in self.req_queue: if req.done is None: time.sleep(self.sleeptime) self.sleeptime *= 2 # double backoff period each time. req.status_results_json = req.avr_session.get(req.results_status_url) req.status_results = json.loads(req.status_results_json.text) if req.status_results['status'] == 'FAILED': req.result = avr_resp() req.result_error['layer'] = 'REST' req.result_error['error'] = req.status_results['status'] req.result_error['text'] = req.status_results if req.status_results['status'] == 'FINISHED': req.raw_results_url = req.status_results['reportResultsLink'] req.results_url = req.raw_results_url.replace('localhost', self.host_name) req.results = self.avr_session.get(req.results_url) if req.results.status_code == 200: req.result = avr_resp() req.result.update(json.loads(req.results.text)) req.done = True self.res_queue.append(req.result) self.num_requests -= 1 else: req.result = avr_resp() req.result_error.layer = 'HTTP' req.result_error.code = req.results.status_code req.result_error.text = req.results self.res_queue.append(req.result) if i == 5: if req.result == False: req.result = avr_resp() req.result.error_layer = 'REST_AVR' req.result.error_error = '408' req.result.error_text = 'TIMEOUT' if self.num_requests == 0: break return self.res_queue ShowAVRJsonApi = """ reportFeatures -------------- Specifies the kind of information that appears in a response from AVR. You may specify one or more of the following values: existing-entities time-aggregated time-series entities-count viewDimensions -------------- Specifies the dimensions for which to calculate a report, such as: {"dimensionName": "domain-name"} You may only specify a single dimension. You may omit this field in a report generation request. viewMetrics ----------- Specifies the list of metrics by which to sort results, such as: { "metricName": "average-tps" }, { "metricName": "transactions" } If you specify either time-aggregated or time-series features, you must specify one metric in a report generation request. sortByMetrics -------------- Specifies the list of metrics to sort by, such as: [{ metricName: "average-tps", order:"descending" } ] Valid values are ascending and descending. Sorting only applies to the time-aggregated feature. You do not need to specify this field in a report generation request. timeRange --------- Specifies the time range, in microseconds, for which to calculate a report, such as: {"from": 1410420888000000, "to": 1410424488000000 } You do not need to specify this field in a report generation request. entityFilters ============= Specifies the entities and values for which to calculate a report. You can specify a single entity with a second level of dimension filters that describe an aspect of the entity. If you specify multiple entity types, the results include only the entities that match all of the criteria. You do not need to specify this field in a report generation request. The following snippet contains two entities with corresponding values: [[{ "dimensionName" : "virtual", "predicate": "OPERATOR_TYPE_EQUAL", "values : ["phpAuction_VS_1"] }, { "dimensionName : "response-code", "predicate": "OPERATOR_TYPE_EQUAL", "values" : ["200"] } ]] metricFilters ------------- Specifies the metric filters for which to calculate a report, such as: [{ "metricName": "transactions", "predicate" : metricFilters "OPERATOR_TYPE_GREATER_THAN" "value": 100 }] You do not need to specify this field in a report generation request. For the existing-entities feature, AVR supports the OPERATOR_TYPE_LIKE predicate. AVR also supports the following predicates: OPERATOR_TYPE_EQUAL OPERATOR_TYPE_NOT_EQUAL OPERATOR_TYPE_GREATER_THAN OPERATOR_TYPE_LOWER_THAN OPERATOR_TYPE_GREATER_THAN_OR_EQUAL OPERATOR_TYPE_LOWER_THAN_OR_EQUAL pagination ---------- Specifies the number of results to return, and the number of results to skip, such as: { numberOfResults : 10, skipResults : 10} To see the second set of ten results, use the example shown here. AVR does not implement the OData query parameters top or skip. In order to see a specific set of results, you must set the number of results to return and then determine how many results to skip. You do not need to specify this field in a report generation request. """ Tested this on version: 12.0
Go library to manage BIG-IP iControl REST API
Problem this snippet solves: This library provides necessary structs and functions to manage the whole REST API. Some REST Calls may require BIG-IP v12.1.x to work properly. How to use this snippet: f5-rest-client implements a REST client to query the F5 BIG-IP iControl REST API. Installation go get -u github.com/e-XpertSolutions/f5-rest-client/f5 Available authentication methods Basic authentication f5Client, err := f5.NewBasicClient(base_url, username, password) Token based authentication f5Client, err := f5.NewTokenClient(base_url, username, password, login_provider_name, skip_ssl_verification) Usage // Copyright 2017 e-Xpert Solutions SA. All rights reserved. // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE file. package main import ( "encoding/json" "log" "github.com/e-XpertSolutions/f5-rest-client/f5" "github.com/e-XpertSolutions/f5-rest-client/f5/net" ) func sexyPrint(label string, a interface{}) { j, err := json.MarshalIndent(a, "", " ") if err != nil { log.Fatal(err) } log.Print("DEBUG ", label, ":\n", string(j)) } func main() { // 1) Basic Authentication f5Client, err := f5.NewBasicClient("https://127.0.0.1", "admin", "admin") // 2) Token Based Authentication // f5Client, err := f5.NewTokenClient("https://127.0.0.1", "admin", "admin", "tmos", true) if err != nil { log.Fatal(err) } f5Client.DisableCertCheck() netClient := net.New(f5Client) self, err := netClient.Self().ListAll() if err != nil { log.Fatal(err) } sexyPrint("SelfIP List:", self) } FEATURES Basic authentication Token based authentication Manage Virtual Server, pool, node, irules, monitors Manage Cluster Management Manage interfaces, vlan, trunk, self ip, route, route domains Manage virtualization features (/vcmp) Manage system related stuffs Add Helper functions to enable, disable or force a node offline Add Helper functions to enable or disable a Virtual Server List expiring certificates List expired certificates Transaction support [new] Manage DNS and global load balancing servers (/gtm) [new] Add support for Stats retrieval on node, pool, virtual and profiles ROADMAP Add support for authentication through external providers Manage access policies (/apm) Manage security (/security) Manage analytics configuration (/analytics) Add support for results pagination Add support for API versioning Add support for new API endpoints coming in v13 Examples Transactions - Create a simple HTTP service f5Client, err := f5.NewBasicClient("https://127.0.0.1", "admin", "admin") if err != nil { log.Fatal(err) } f5Client.DisableCertCheck() // Start new transaction. tx, err := f5Client.Begin() if err != nil { log.Fatal(err) } ltmClient := ltm.New(tx) // Create a HTTP monitor log.Print("Create a HTTP monitor") monitorConfig := ltm.MonitorHTTPConfig{ Name: "http_monitor_" + tx.TransactionID(), Send: "GET / HTTP/1.0\r\n\r\n", Recv: "Hello", } if err := ltmClient.MonitorHTTP().Create(monitorConfig); err != nil { log.Fatal(err) } // Create a Pool log.Print("Create a pool") poolConfig := ltm.PoolConfig{ Name: "pool_" + tx.TransactionID(), Monitor: "/Common/http_monitor_" + tx.TransactionID(), Members: []string{"10.1.10.10:80", "10.1.10.11:80"}, } if err := ltmClient.Pool().Create(poolConfig); err != nil { log.Fatal(err) } // Create a Virtual Server log.Print("Create a Virtual Server") vsConfig := ltm.VirtualServerConfig{ Name: "vs_http_" + tx.TransactionID(), Destination: "10.1.20.130:80", IPProtocol: "tcp", Pool: "pool_" + tx.TransactionID(), SourceAddressTranslation: ltm.SourceAddressTranslation{ Type: "automap", }, Profiles: []string{ "tcp-mobile-optimized", "http", }, } if err := ltmClient.Virtual().Create(vsConfig); err != nil { log.Fatal(err) } // Commit to make the changes persistent. if err := tx.Commit(); err != nil { log.Fatal(err) } List SSL Certificates sysClient := sys.New(f5Client) certs, err := sysClient.FileSSLCert().ListAll() if err != nil { log.Fatal(err) } sexyPrint("Certificates", certs) List expired SSL Certificates sysClient := sys.New(f5Client) certs, err := sysClient.FileSSLCert().ListExpired() if err != nil { log.Fatal(err) } sexyPrint("Expired Certificates", certs) List expiring SSL Certificates sysClient := sys.New(f5Client) // ListExpiring(number_of_seconds) certs, err := sysClient.FileSSLCert().ListExpiring(60 * 60 * 24 * 15) if err != nil { log.Fatal(err) } sexyPrint("Expiring Certificates", certs) Contributing We appreciate any form of contribution (feature request, bug report, pull request, ...). We have no special requirements for Pull Request, just follow the standard GitHub way. License The sources are release under a BSD 3-Clause License. The full terms of that license can be found in LICENSE file of this repository. Code : https://github.com/e-XpertSolutions/f5-rest-client Tested this on version: 11.5
Big-IQ bulk licensing of Big-IP using REST API
Problem this snippet solves: Attached is a link to github which provides the user with an comprehensive example of how to license many BIGIP devices via BIGIQ CM REST API into an existing license pool. Script bulkLicensePool.pl is a standalone script installed directly in the BIGIQ shell. Suggested recommendations: 1. Create a /shared/scripts/. directory 2. scp file to BIGIQ, 3. Usage below. This automation will invoke a task to license many BIGIP's as defined in a bulk_license.csv file. This happens sequentially and is very useful when administrator's goal is to license many BIGIP devices in a programmatic manner. ** tested with perl distribution present on bigiq v5.8.8 How to use this snippet: Usage: ./bulkDiscovery -c bulk_discovery.csv Program: bulkLicensePool.pl Version: v2.00.00 ##### License multiple BIG-IP devices. -r Root credentials for every BIG-IP (such as root:default) - overrides root creds in CSV -a Admin credentials for every BIG-IP (such as admin:admin) - overrides any creds in CSV -v Verbose screen output -s Discover ASM -l Discover LTM -p Discover APM -c Path to CSV file with all BIG-IP devices - REQUIRED -u Update framework if needed -h Help -k Keep the CSV file after this finishes (not recommended if it contains creds) -q BIG-IQ admin credentials in form admin:password - REQUIRED if not using default -g access group name if needed -f Discover AFM csv format: ip, user, pw, cluster-name, framework-action, root-user, root-pw ip: ip address of the BigIP to discover. user, pw: username & password of the BigIP. Will be overridden if -a is specified on the command line. configuration csv example format 1.2.3.4, admin, pw, base-reg-key Code : https://github.com/carldubois/bigiq-cm-restapi-bulk Tested this on version: 12.0Big-IQ bulk trust, discovery and import of Big-IP using REST API
Problem this snippet solves: Attached is a link to github which provides the user with an comprehensive example of how to discover and import many BIGIP device via BIGIQ CM REST API. Script bulkDiscovery.pl is a standalone script installed directly in the BIGIQ shell. Suggested recommendations: 1. Create a /shared/scripts/. directory 2. scp file to BIGIQ, 3. Usage below. This automation will invoke a device trust task to negotiate certificate, discover device to population in resolver groups (maintained per module) and import configuration of BIGIP's as defined in bulk_discovery.csv file. This happens sequentially and is very useful when administrator's goal is to discover and import many BIGIP devices in a programmatic manner. ** tested with perl distribution present on bigiq v5.8.8 How to use this snippet: Usage: ./bulkDiscovery -c bulk_discovery.csv Program: bulkDiscovery.pl Version: v2.00.00 ##### Discover multiple BIG-IP devices. -r Root credentials for every BIG-IP (such as root:default) - overrides root creds in CSV -a Admin credentials for every BIG-IP (such as admin:admin) - overrides any creds in CSV -v Verbose screen output -s Discover ASM -l Discover LTM -p Discover APM -c Path to CSV file with all BIG-IP devices - REQUIRED -u Update framework if needed -h Help -k Keep the CSV file after this finishes (not recommended if it contains creds) -q BIG-IQ admin credentials in form admin:password - REQUIRED if not using default -g access group name if needed -f Discover AFM csv format: ip, user, pw, cluster-name, framework-action, root-user, root-pw ip: ip address of the BigIP to discover. user, pw: username & password of the BigIP. Will be overridden if -a is specified on the command line. configuration csv example format: 1.2.3.4 1.2.3.4, admin, pw 1.2.3.4, admin, pw, ha-name 1.2.3.4,,, ha-name 1.2.3.4, admin, pw,, skip 1.2.3.4, admin, pw,, update, root, root-pw Code : https://github.com/carldubois/bigiq-cm-restapi-bulk Tested this on version: 12.0Big-IQ trust, discovery and import of Big-IP using REST API - Python OO
Problem this snippet solves: Some may want code that is a little more extendable and will scale to provide more of a workflow structure. In this example, we explore the use of Python class to simplify the code using a call to class method doing the work for trust (negotiate cert), discover (populate resolver groups), import (import bigip configuration). With the model you can create new methods to extend functionality. For example: class Workflow contains def Discover, def License, def CreateVirtual, def CreatePool, def AddMembers, def RefSecPolicy. Script will look like: workflow.Discover() workflow.License() workflow.CreateVirtual() workflow.CreatePool() workflow.AddMembers() workflow.RefSecPolicy() Code : https://github.com/carldubois/discapi
