Python module to post and retrieve IControl Rest JSON objects for AVR statistics
Problem this snippet solves: This module simplifies making Python dictionary objects that are converted to IControl rest AVR JSON objects. It also handles making AVR requests and retrieving results as well allowing multiple AVR requests to be queued, posted and retrieved. It also has some basis type checking for the elements of a AVR request. This module requires Bigip 12.1 on the target that statistics are retrieved. How to use this snippet: The main class is rest_avr.avr_req. It is a dictionary class that maps directly to an IControl Rest AVR JSON request as translated by json.dumps. Each dictionary element is an object derived from a customer class for each part of the request. The element classes have add() and clear() functions. if the element class only allows one entry the add() function will replace the existing entry, otherwise it will append the entry to the request element. The rest_avr.avr_req class also has functions to populate the HTTP host and authentication values for the target system. rest_avr.avr_req.post_and_response returns the Python representation of the JSON result of the query. rest_avr.avr_req.add_to_queue() adds the currently constructed request to a queue of requests to post. rest_avr.avr_req.post_and_response_queue() returns a python list of results of queued queries. The following code sample constructs, posts and returns results for an AVR statistics request for specific DNS records and a specificrecord type, then queues multiple quests and posts and returns results. #!/usr/bin/python import json import sys import time import rest_avr #print rest_avr.ShowAVRJsonApi #Populate the url avr_dns_req=rest_avr.avr_req() avr_dns_req.auth('admin','admin') avr_dns_req.url_base('10.10.2.113','dns') #Populate the json object avr_dns_req['analyticsModule'].add('dns') avr_dns_req['reportFeatures'].add('time-aggregated') avr_dns_req['entityFilters'].add('domain-name', 'OPERATOR_TYPE_EQUAL', ['test2.test1.com','test1.test1.com']) avr_dns_req['entityFilters'].add('query-type', 'OPERATOR_TYPE_EQUAL', ['a']) avr_dns_req['viewMetrics'].add('packets') avr_dns_req['viewDimensions'].add('domain-name') avr_dns_req['metricFilters'].add('packets', 'OPERATOR_TYPE_GREATER_THAN', 0) avr_dns_req['sortByMetrics'].add('packets', 'ascending') avr_dns_req['pagination'].add(20, 0) avr_dns_req['timeRange'].add(1461778251000000, None) #Post and retrieve results. result_py=avr_dns_req.post_and_response() if result_py != None: print ('\n' + result_py['results']['timeAggregated'][0]['dimensions'][0]['value'] + " " + result_py['results']['timeAggregated'][0]['metricValues'][0]['value'] + '\n') else: print result_py.error_layer print result_py.error_code print result_py.error_text # Now add multiple requests to a queue avr_dns_req.add_to_queue() avr_dns_req['entityFilters'].clear() avr_dns_req['entityFilters'].add('query-type', 'OPERATOR_TYPE_EQUAL', ['aaaa']) avr_dns_req.add_to_queue() #post and retrieve queued results result_py_q=avr_dns_req.post_and_response_queue() for result_py in result_py_q: if result_py != None: print ('\n' + result_py['results']['timeAggregated'][0]['dimensions'][0]['value'] + " " + result_py['results']['timeAggregated'][0]['metricValues'][0]['value'] + '\n') else: print result_py.error_layer print result_py.error_code print result_py.error_text Code : """ rest_avr provides a python interface to Bigip AVR statistics using the REST API. The main Python rest_avr.avr_req object is a Python dictionary that maps to a JSON object that can be processed with the json.dumps() function An IControl Rest AVR JSON request and response can be initiated with avr_req.post_and_response The simple description of the API can is available at avr_req.ShowJsonApi() Each of these modules has a method to add single or multiple elements as appropriate to the specific module. Once these elements are are populated a RestAPI request can be made with results returned as a python representation. avr_req.auth(user, passw) avr_req.url_base(host, module) avr_req['analyticsModule'].add(module) avr_req['analyticsModule'].clear() avr_req['reportFeatures'].add(metric_name, predicate, value) avr_req['reportFeatures'].clear() avr_req['entityFilters'].add(dimension_name, predicate, values) avr_req['entityFilters'].clear() avr_req['viewMetrics'].add(metric_name) avr_req['viewMetrics'].clear() avr_req['viewDimensions'].add(metric_name, order) avr_req['viewDimensions'].clear() avr_req['metricFilters'].add(metric_name, predicate, valu) avr_req['metricFilters'].clear() avr_req['sortByMetrics'].add(metric_name, orde) avr_req['sortByMetrics'].clear() avr_req['pagination'].add(num_results, skip_result) avr_req['pagination'].clear() avr_req['timeRange'].add(t_from, t_to) avr_req['timeRange'].clear() After a request in constructed a REST API call is initiated with initiated with: avr_req.post_and_response() The response is a python dictionary data structure of the results as processed by json.loads """ from copy import deepcopy import requests import json import sys import time import warnings __author__ = 'Mark Lloyd' __version__ = '1.0' # 05/24/2016 import json import requests import time class BadDictElement(Exception): def __init__(self, key, value, expl): Exception.__init__(self, '{0} {1} {2} '.format(key, value, expl)) class BadTime(Exception): def __init__(self, variable, value): Exception.__init__(self, '{0} {1} should be 16 char decimal in microseconds '.format('a', 'b')) class RequestFailure(Exception): def __init__(self, key, value): Exception.__init__(self, '{0} {1} '.format(key, value)) class analyticsModule(str): """ This class is tied to the structure of the parent class. parent() get's the parent object so we can make the string pseudo-mutable. accessed from within an avr request ['analyticsModule'].add(module) Adds a single string to analyticsModule element . If one exists it is replaced. ['analyticsModule'].clear() Send a null value to the analyticsModule element. See rest_avr.ShowAVRJsonApi for more details """ def parent(self, parent): self.parent = parent def add(self, module): """ avr_req.['analyticsModule'].add(module) Adds a single string to analyticsModule element . If one already exists it is replaced. This should be the same as the module string in avr_req.url_base. """ self.parent['analyticsModule'] = analyticsModule(module) self.parent['analyticsModule'].parent = self.parent def clear(self): """ avr_req.['analyticsModule'].add(module) replaces the analyticsModule mddule with a null string """ self.parent['analyticsModule'] = analyticsModule('') self.parent['analyticsModule'].parent = self.parent class metricFilters(list): """ avr_req.['metricFilters'].add(metric_name, predicate, value) metric name is a string, value is an integer Valid predicates strings are ['OPERATOR_TYPE_EQUAL', 'OPERATOR_TYPE_NOT_EQUAL', 'OPERATOR_TYPE_GREATER_THAN', OPERATOR_TYPE_LOWER_THAN','OPERATOR_TYPE_GREATER_THAN_OR_EQUAL', 'OPERATOR_TYPE_LOWER_THAN_OR_EQUAL']) avr_req['metricFilters'].clear() Clears metricFilters elements See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.append([]) self.valid_metric_predicate = ( ['OPERATOR_TYPE_EQUAL', 'OPERATOR_TYPE_NOT_EQUAL', 'OPERATOR_TYPE_GREATER_THAN', 'OPERATOR_TYPE_LOWER_THAN', 'OPERATOR_TYPE_GREATER_THAN_OR_EQUAL', 'OPERATOR_TYPE_LOWER_THAN_OR_EQUAL']) def add(self, metric_name, predicate, value): """ avr_req.['metricFilters'].add(metric_name, predicate, value) metric name is a string, value is an integer Valid predicates strings are ['OPERATOR_TYPE_EQUAL', 'OPERATOR_TYPE_NOT_EQUAL', 'OPERATOR_TYPE_GREATER_THAN', OPERATOR_TYPE_LOWER_THAN','OPERATOR_TYPE_GREATER_THAN_OR_EQUAL', 'OPERATOR_TYPE_LOWER_THAN_OR_EQUAL'] """ if type(value) is not int: raise BadDictElement(metric_name, value, 'value should be integer') if predicate in self.valid_metric_predicate: # first check if it is already there for metric in self[0]: if metric['metricName'] == metric_name: metric['predicate'] = predicate metric['value'] = value return 0 # if it is not there then just add it. self[0].append({'metricName': metric_name, 'predicate': predicate, 'value': value}) else: raise BadDictElement(metric_name, predicate, 'invalid predicate') def clear(self): """ avr_req['metricFilters'].clear() Clears metricFilters elements """ del self[0][:] class entityFilters(list): """ avr_req.['entityFilters'].add(dimension_name, predicate, values): All values are strings valid predicate is 'OPERATOR_TYPE_EQUAL' ['entityFilters'].clear() Clears the entityFilters element See rest_avr.ShowJsonApi for more details """ def __init__(self): self.append([]) def add(self, dimension_name, predicate, values): """ avr_req.['entityFilters'].add(dimension_name, predicate, values): All values are strings valid predicate is 'OPERATOR_TYPE_EQUAL' """ if predicate is 'OPERATOR_TYPE_EQUAL': # then loop throuth to see if the dimenson name already exists, if so replace for entity in self[0]: if entity['dimensionName'] == dimension_name: entity['predicate'] = predicate entity['values'] = values return 0 # if it is not there then just add it. self[0].append({'dimensionName': dimension_name, 'predicate': predicate, 'values': values}) else: raise BadDictElement(dimension_name, predicate, 'predicate must be OPERATOR_TYPE_EQUAL') def clear(self): """ ['entityFilters'].clear() Clears the entityFilters element """ del self[0][:] class reportFeatures(list): """ avr_req.['reportFeatures'].add( feature) adds report feature string. Multiple features are permitted. ['reportFeatures'].clear() Clears the analyticsModule element. See rest_avr.ShowAVRJsonApi for more details. """ def add(self, feature): """ avr_req.['reportFeatures'].add( feature) adds report feature string. Multiple features are permitted .""" if feature not in self: self.append(feature) def clear(self): """ ['reportFeatures'].clear() Clears the entityFilters element """ del self[:] class sortByMetrics(list): """ avr_req.['sortByMetrics'].add(metric_name, order) valid order names are 'ascending' and 'descending' sortByMetrics is optional in an AVR request. avr_req['sortByMetrics'].clear() Clears the sortByMetrics element. See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.metric_list = [] def add(self, metric_name, order): if metric_name not in self.metric_list: self.append({'metricName': metric_name, 'order': order}) self.metric_list.append(metric_name) def clear(self): """ ['sortByMetrics'].clear() Clears the sortByMetrics element """ del self[:] del self.metric_list[:] class viewDimensions(list): """ avr_req.['viewDimensions'].add(dimension_name): adds view dimension, only one dimension is allowed add will replace element if it already exists avr_req['viewDimensions'].clear() Clears the viewDimensions element. See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.append([]) self[0] = {} def add(self, dimension_name): """ avr_req.['viewDimensions'].add(dimension_name): adds view dimension string, only one dimension is allowed add will replace element if it already exists """ self[0]['dimensionName'] = dimension_name def clear(self, dimension_name): """ ['viewDimensions'].clear() Clears the viewDimensions element """ del self[0][:] class viewMetrics(list): """ avr_req.['viewMetrics'].add(metric_name): appends metric_name string to list. The specification allows multiple view metric elements avr_req['viewMetrics'].clear() Clears the viewMetrics elements See rest_avr.ShowAVRJsonApi for more details. """ def __init__(self): self.metric_list = [] def add(self, metric_name): """ avr_req.['viewMetrics'].add(metric_name): appends metric_name string to list. The specification allows multiple viewMetric elements """ if metric_name not in self.metric_list: self.append({'metricName': metric_name}) self.metric_list.append(metric_name) def clear(self): """ ['viewMetrics'].clear() Clears the viewMetrics elements """ del self[:] del self.metric_list[:] class timeRange(dict): """ avr_req.['timeRange'].add( t_from, t_to) both values are 16 digit numeric value in microseconds of unix/linux time. t_to is optional and can be replace by None timeRange is an optional. avr_req['timeRange'].clear() Clears the timeRange elements See rest_avr.ShowAVRJsonApi for more details. """ def add(self, t_from, t_to): """ avr_req.['timeRange'].add( t_from, t_to) both values are 16 digit numeric value in microseconds of unix/linux time. t_to is optional and can be replace by None timeRange is optional. """ if type(t_from) is long and len(str(t_from)) == 16: self['from'] = t_from else: raise BadTime(t_from + " is 16 digit numeric value in microseconds") if t_to != '' and t_to != 0 and t_to != None: if type(t_to) is long and len(str(t_from)) == 16: self['to'] = t_to else: raise BadTime(t_to + " is 16 digit numeric value in microseconds") else: if 'to' in self.keys(): del self['to'] def clear(self): """ ['timeRange'].clear() Clears the timeRange element """ del self[:] class pagination(dict): """ avr_req.['pagination'].add(num_results, skip_results) both are integer values. avr_req['pagination'].clear() Clears the pagination elements See rest_avr.ShowAVRJsonApi for more details. """ def add(self, num_results, skip_results): """ avr_req.['pagination'].add(num_results, skip_results) both arguments are integers. """ if type(num_results) is int: self['numberOfResults'] = num_results else: raise BadDictElement('number of Results ', num_results, 'must be integer') if type(skip_results) is int: self['skipResults'] = skip_results else: raise BadDictElement('skipResults ', skip_results, 'must be integer') def clear(self): """ ['pagination'].clear() Clears the pagination element """ del self[:] class avr_resp(dict): """ python response error is applicable. """ def __init__(self): self.error_layer = None self.error_code = None self.error_text = None class avr_req(dict): """ The main class for rest_avr. avr_req contains a dictionary that maps to the elements of a Icontrol REST AVR request along with capability of posting that request and receiving a response. The dictionary values are object instances of python classes that correspond to the the JSON values of the object's name/value pair. Each value has two public methods: avr_req.['objectName']add(): adds an element to the appropriate object with type checking. If an element allows more then one instance the add function will append the element If an element allows only one instance the add function will replace the element avr_req.['objectName'].clear()r: clears all elements in the object. printing rest_avr.ShowAVRJsonApi provides documentation for the AVR JASON elements. Further documentation is available on devcentral.f5.com To post an AVR Rest request there are two functions to populate the HTTP/HTTPS request. avr_req.auth(user, passw): provides the username and password avr_req.url_base(host, module) provides the host and the bigip module AVR queries to construct the URL to make the request. Then to post the request and return results in a python representation of the JSON response. avr_req.post_and_response() """ def __init__(self): self['analyticsModule'] = analyticsModule() self['analyticsModule'].parent = self self['pagination'] = pagination() self['metricFilters'] = metricFilters() self['entityFilters'] = entityFilters() self['reportFeatures'] = reportFeatures() self['sortByMetrics'] = sortByMetrics() self['viewDimensions'] = viewDimensions() self['viewMetrics'] = viewMetrics() self['timeRange'] = timeRange() self.avr_session = requests.session() self.avr_session.verify = False self.avr_session.headers.update({'Content-Type': 'application/json'}) # for multiple queued request handling. self.req_queue = [] self.generate_id = None self.done = None self.result = None self.num_requests = 0 self.res_queue = [] def post_and_response(self): """ returns a python representation of the json response to the request. failure returns array ['ERROR','component',error] """ warnings.filterwarnings("ignore") self.generate_request = self.avr_session.post(self.req_url_base + "/generate-report/", data=json.dumps(self)) self.generate_request_py = json.loads(self.generate_request.text) self.result_guid = self.generate_request_py['id'] self.results_status_url = self.req_url_base + "/generate-report/" + self.result_guid + "/?$select=status,reportResultsLink" self.results_url = self.req_url_base + "/report-results/" + self.result_guid self.sleeptime = .5 for i in range(5): time.sleep(self.sleeptime) self.sleeptime *= 2 # double backoff period each time. self.status_results_json = self.avr_session.get(self.results_status_url) self.status_results = json.loads(self.status_results_json.text) if self.status_results['status'] == 'FAILED': self.result = avr_resp() self.result_error_layer = 'REST' self.result_error_code = self.status_results['status'] self.result.error_text = self.status_results if self.status_results['status'] == 'FINISHED': self.raw_results_url = self.status_results['reportResultsLink'] self.results_url = self.raw_results_url.replace('localhost', self.host_name) self.results = self.avr_session.get(self.results_url) if self.results.status_code == 200: self.result = avr_resp() self.result.update(json.loads(self.results.text)) return self.result else: self.result = avr_resp() self.result.error_layer = 'HTTP' self.result.error_code = self.results.status_code self.result.error_text = self.results return self.result else: continue self.result = avr_resp() self.result.error_layer = 'REST_AVR' self.result.error_code = '408' self.result.error_text = 'TIMEOUT' def auth(self, user, passw): """ avr_req.auth(user, passw): username and password """ self.avr_session.auth = (user, passw) def url_base(self, host, module): """ avr_req.url_base(host, module) host and bigip module AVR queries to construct the URL to make the request. """ self.host_name = host self.req_url_base = 'https://%s/mgmt/tm/analytics/%s' % (host, module) self.module_py = {'analyticsModule': module} def add_to_queue(self): "adds request as currently constructed to queue" self.req_queue.append(deepcopy(self)) def clear_queue(self): """" clears request queue """ del self.req_queue[:] def post_and_response_queue(self): """ posts and sends response to from queue of requests. """ warnings.filterwarnings("ignore") for req in self.req_queue: req.generate_request = req.avr_session.post(req.req_url_base + "/generate-report/", data=json.dumps(req)) req.generate_request_py = json.loads(req.generate_request.text) req.generate_id = (req.generate_request_py['id']) req.results_status_url = self.req_url_base + "/generate-report/" + req.generate_id + "/?$select=status,reportResultsLink" self.sleeptime = .5 self.num_requests = len(self.req_queue) for i in range(5): for req in self.req_queue: if req.done is None: time.sleep(self.sleeptime) self.sleeptime *= 2 # double backoff period each time. req.status_results_json = req.avr_session.get(req.results_status_url) req.status_results = json.loads(req.status_results_json.text) if req.status_results['status'] == 'FAILED': req.result = avr_resp() req.result_error['layer'] = 'REST' req.result_error['error'] = req.status_results['status'] req.result_error['text'] = req.status_results if req.status_results['status'] == 'FINISHED': req.raw_results_url = req.status_results['reportResultsLink'] req.results_url = req.raw_results_url.replace('localhost', self.host_name) req.results = self.avr_session.get(req.results_url) if req.results.status_code == 200: req.result = avr_resp() req.result.update(json.loads(req.results.text)) req.done = True self.res_queue.append(req.result) self.num_requests -= 1 else: req.result = avr_resp() req.result_error.layer = 'HTTP' req.result_error.code = req.results.status_code req.result_error.text = req.results self.res_queue.append(req.result) if i == 5: if req.result == False: req.result = avr_resp() req.result.error_layer = 'REST_AVR' req.result.error_error = '408' req.result.error_text = 'TIMEOUT' if self.num_requests == 0: break return self.res_queue ShowAVRJsonApi = """ reportFeatures -------------- Specifies the kind of information that appears in a response from AVR. You may specify one or more of the following values: existing-entities time-aggregated time-series entities-count viewDimensions -------------- Specifies the dimensions for which to calculate a report, such as: {"dimensionName": "domain-name"} You may only specify a single dimension. You may omit this field in a report generation request. viewMetrics ----------- Specifies the list of metrics by which to sort results, such as: { "metricName": "average-tps" }, { "metricName": "transactions" } If you specify either time-aggregated or time-series features, you must specify one metric in a report generation request. sortByMetrics -------------- Specifies the list of metrics to sort by, such as: [{ metricName: "average-tps", order:"descending" } ] Valid values are ascending and descending. Sorting only applies to the time-aggregated feature. You do not need to specify this field in a report generation request. timeRange --------- Specifies the time range, in microseconds, for which to calculate a report, such as: {"from": 1410420888000000, "to": 1410424488000000 } You do not need to specify this field in a report generation request. entityFilters ============= Specifies the entities and values for which to calculate a report. You can specify a single entity with a second level of dimension filters that describe an aspect of the entity. If you specify multiple entity types, the results include only the entities that match all of the criteria. You do not need to specify this field in a report generation request. The following snippet contains two entities with corresponding values: [[{ "dimensionName" : "virtual", "predicate": "OPERATOR_TYPE_EQUAL", "values : ["phpAuction_VS_1"] }, { "dimensionName : "response-code", "predicate": "OPERATOR_TYPE_EQUAL", "values" : ["200"] } ]] metricFilters ------------- Specifies the metric filters for which to calculate a report, such as: [{ "metricName": "transactions", "predicate" : metricFilters "OPERATOR_TYPE_GREATER_THAN" "value": 100 }] You do not need to specify this field in a report generation request. For the existing-entities feature, AVR supports the OPERATOR_TYPE_LIKE predicate. AVR also supports the following predicates: OPERATOR_TYPE_EQUAL OPERATOR_TYPE_NOT_EQUAL OPERATOR_TYPE_GREATER_THAN OPERATOR_TYPE_LOWER_THAN OPERATOR_TYPE_GREATER_THAN_OR_EQUAL OPERATOR_TYPE_LOWER_THAN_OR_EQUAL pagination ---------- Specifies the number of results to return, and the number of results to skip, such as: { numberOfResults : 10, skipResults : 10} To see the second set of ten results, use the example shown here. AVR does not implement the OData query parameters top or skip. In order to see a specific set of results, you must set the number of results to return and then determine how many results to skip. You do not need to specify this field in a report generation request. """ Tested this on version: 12.0
Microsoft 365 IP Steering python Script
Hello! Hola! I have created a small and rudimentary script that generates a datagroup with MS 365 IPv4 and v6 addresses to be used by an iRule or policy. There are other scripts that solve this same issue but either they were: based on iRulesLX which forces you to enable iRuleLX only for this, and made me run into issues when upgrading (memory table got filled with nonsense) based on the XML version of the list which MS changed to a JSON file. This script is a super simple bash script that calls another super simple python file, and a couple of helper files. The biggest To Do are: Add a more secure approach to password usage. Right now, it is stored in a parameters file locked away with permissions. There should be a better way. Add support for URLs. You can find the contents here:https://github.com/teoiovine-novared/fetch-office365/tree/main I appreciate advice, (constructive) criticism and questions all the same! Thank you for your time.
Python Bigsuds - License BIG-IP
Problem this snippet solves: This bigsuds script will license a BIGIP How to use this snippet: This bigsuds script will license a BIGIP It's a port of some of Erick's: https://devcentral.f5.com/wiki/icontrol.BigIpLicensingCommandLineTool.ashx and update of the pycontrol v2: pyControl v2 - License BIGIP - DevCentral Still missing the proxy functionality in Erick's though. Usage: license-bigip.py --bigip < IP|hostname> --username < username> --server < license_server_hostname> --reg_keys < regkeys> --license < license_file> --eula < eula_file> Ex. From CLI: (virt1)user1@desktop:python $./license-bigip.py --help Usage: license-bigip.py [options] Options: -h, --help show this help message and exit -b BIGIP, --bigip=BIGIP -u UNAME, --username=UNAME -s SERVER_HOSTNAME, --server=SERVER_HOSTNAME -r REG_KEYS_STRING, --reg_keys=REG_KEYS_STRING -l LOCAL_LICENSE_FILE_NAME, --license=LOCAL_LICENSE_FILE_NAME -e LOCAL_EULA_FILE_NAME, --eula=LOCAL_EULA_FILE_NAME Providing New Keys: (virt1)user1@desktop:python $./license-bigip.py --server activate.f5.com --bigip 10.11.50.201 --username admin --reg_keys "B6907-36850-30441-96521-8395199" Enter your password for username: admin Password: Attempting to get license online. License server requires you to submit EULA. reg keys provided ['B6907-36850-30441-96521-8395172'] Getting dossier using keys:['B6907-36850-30441-96521-8395199'] License Found. Attempting to installing License on BIGIP: License status = STATE_ENABLED Use Existing Keys: (virt1)user1@desktop:python $./license-bigip.py --server activate.f5.com --bigip 10.11.50.201 --username admin Enter your password for username: admin Password: Attempting to get license online. License server requires you to submit EULA. Reg Key list is empty, attempting to retrieve existing keys from the unit Getting dossier using keys:[B6907-36850-30441-96521-8395199, P459718-6102942, B273271-7144254, N123537-7304104] License Found. Attempting to installing License on BIGIP: License status = STATE_ENABLED Script license-bigip.py Code : #!/usr/bin/env python ''' ---------------------------------------------------------------------------- The contents of this file are subject to the "END USER LICENSE AGREEMENT FOR F5 Software Development Kit for iControl"; you may not use this file except in compliance with the License. The License is included in the iControl Software Development Kit. Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. The Original Code is iControl Code and related documentation distributed by F5. The Initial Developer of the Original Code is F5 Networks, Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 1996-2004 F5 Networks, Inc. All Rights Reserved. iControl (TM) is a registered trademark of F5 Networks, Inc. Alternatively, the contents of this file may be used under the terms of the GNU General Public License (the "GPL"), in which case the provisions of GPL are applicable instead of those above. If you wish to allow use of your version of this file only under the terms of the GPL and not to allow others to use your version of this file under the License, indicate your decision by deleting the provisions above and replace them with the notice and other provisions required by the GPL. If you do not delete the provisions above, a recipient may use your version of this file under either the License or the GPL. ---------------------------------------------------------------------------- ''' def usage (): print "Usage:" print "%s --bigip --username --server \ --reg_keys --license --eula " % sys.argv[0] print "ex. " print " Will attempt to re-license with existing keys on unit using license server activate.f5.com" print " %s --bigip 192.168.1.245 --username admin" % sys.argv[0] print " Will attempt to re-license with provided reg_keys CSV string using license server activate.f5.com" print " %s --bigip 192.168.1.245 --username admin --reg_keys \"XXXX-XXXX-XXXX-XXXX,XXXX-XXXX,XXXX-XXXX\" " % sys.argv[0] def get_license_from_F5_License_Server ( server_hostname, dossier_string, eula_string, email, firstName, lastName, companyName, phone, jobTitle, address, city, stateProvince, postalCode, country ): try: license_string = "" # Unfortunately, F5 wsdl on license server references http but F5 only accepts https so as an ugly workaround need to # download wsdl, save to disk, replace links http with https, and have SUDS client reference local file instead #(eg. url = "file:///home/admin/f5wsdl.xml") download_url = "https://" + server_hostname + "/license/services/urn:com.f5.license.v5b.ActivationService?wsdl" # Check to see if there's a copy of wsdl file on disk first # Careful with this behavior if you switch server hostnames. local_wsdl_file_name = str(server_hostname) + '-f5wsdl-w-https.xml' wsdl_data = [] try: with open(local_wsdl_file_name, 'r') as fh_wsdl: wsdl_data = fh_wsdl.read() except: print "Can't find a locally stored WSDL file." if not wsdl_data: print "Attempting to fetch wsdl online." f5wsdl = urllib2.urlopen(download_url) newlines = [] for line in f5wsdl: # do the replacing here newlines.append(line.replace('http://' + server_hostname , 'https://' + server_hostname)) fh_local = open(local_wsdl_file_name,'w') fh_local.writelines(newlines) fh_local.close() # put url going to pass to client in file format url = "file:" + urllib.pathname2url(os.getcwd()) + "/" + local_wsdl_file_name #Now create client object using wsdl from disk instead of the interwebs. client = Client(url) # NOT using below as will just try actually licensing and fail then if needed # try: # # ping() method should return string containing date # print "Checking License Service Reachability..." # return_ping_date = client.service.ping() # except: # print "License SOAP service unreachable. Check network connectivity." # return transaction = client.factory.create('ns0:LicenseTransaction') # If eula isn't present on first call to getLicense, transaction will fail # but it will return a eula after first attempt transaction = client.service.getLicense( dossier = dossier_string, eula = eula_string, email = email, firstName = firstName , lastName = lastName, companyName = companyName, phone = phone, jobTitle = jobTitle, address = address, city = city, stateProvince = stateProvince, postalCode = postalCode, country = country, ) #Extract the eula offered from first try eula_string = transaction.eula if transaction.state == "EULA_REQUIRED": #Try again, this time with eula populated transaction = client.service.getLicense( dossier = dossier_string, eula = eula_string, email = email, firstName = firstName , lastName = lastName, companyName = companyName, phone = phone, jobTitle = jobTitle, address = address, city = city, stateProvince = stateProvince, postalCode = postalCode, country = country, ) if transaction.state == "LICENSE_RETURNED": license_string = transaction.license else: print "Can't retrieve license from Licensing server" print "License server returned error: Number:" + str(transaction.fault.faultNumber) + " Text: " + str(transaction.fault.faultText) return license_string except: print "Can't retrieve License from Server" traceback.print_exc(file=sys.stdout) def get_reg_keys(obj): try: reg_keys = [] reg_keys = obj.Management.LicenseAdministration.get_registration_keys() return reg_keys except: print "Get Reg Keys error. Check log." traceback.print_exc(file=sys.stdout) def get_dossier (obj, reg_keys ): try: dossier_string = obj.Management.LicenseAdministration.get_system_dossier ( reg_keys ) return dossier_string except: print "Get Dossier error. Check log." traceback.print_exc(file=sys.stdout) def get_eula_file (obj): try: eula_char_array = obj.Management.LicenseAdministration.get_eula_file( ) eula_string = base64.b64decode(eula_char_array) return eula_string except: print "Get eula_file. Check log." traceback.print_exc(file=sys.stdout) def install_license (obj, license_string ): try: license_char_array = base64.b64encode(license_string) obj.Management.LicenseAdministration.install_license ( license_file_data = license_char_array ) except: print "Install License error. Check log." traceback.print_exc(file=sys.stdout) def get_license_status (obj): try: license_status = obj.Management.LicenseAdministration.get_license_activation_status() return license_status except: print "Get License Status error. Check log." traceback.print_exc(file=sys.stdout) ### IMPORT MODULES ### import os import sys import time import traceback import base64 import urllib import urllib2 import getpass from suds.client import Client from optparse import OptionParser import bigsuds # from suds import WebFault # import logging # logging.getLogger('suds.client').setLevel(logging.DEBUG) # logging.getLogger('suds.metrics').setLevel(logging.DEBUG) # logging.getLogger('suds').setLevel(logging.DEBUG) #### SET CONFIG VARIABLES #### #Misc EULA Variables email = "example.icontrol@f5.com" firstName = "example" lastName = "iControl" companyName = "F5" phone = "2062725555" jobTitle = "DEV OPS" address = "111 EXAMPLE ICONTROL RD" city = "Seattle" stateProvince = "WA" postalCode = "98119" country = "United States" parser = OptionParser() parser.add_option("-b", "--bigip", action="store", type="string", dest="bigips", default="192.168.1.245") parser.add_option("-u", "--username", action="store", type="string", dest="uname", default="admin") parser.add_option("-s", "--server", action="store", type="string", dest="server_hostname", default="activate.f5.com" ) parser.add_option("-r", "--reg_keys", action="store", type="string", dest="reg_keys_string" ) parser.add_option("-l", "--license", action="store", type="string", dest="local_license_file_name") parser.add_option("-e", "--eula", action="store", type="string", dest="local_eula_file_name") (options, args) = parser.parse_args() ### INITIALIZE BIGIP OBJECT ### if options.bigip and options.uname: print "Enter your password for username: %s" % options.uname upass = getpass.getpass() else: usage() sys.exit() #hardcoded so don't have to provide any arguments. #print "Enter your password for username: %s" % options.uname #upass = getpass.getpass() # Can re-license a list of BIG-IPs IF you don't provide a list of regkeys on CLI. # As reg key split function does not accommodate list of keys for multiple devices. if options.bigips: bigip_list = options.bigips.split(",") for i in bigip_list: print "\nAttempting to License BIGIP " + i reg_keys = [] license_string = "" eula_string = "" reg_keys_string = "" local_license_file_name = "" local_eula_file_name = "" server_hostname = "authem.f5net.com" uname = "admin" b = bigsuds.BIGIP( hostname = i, username = uname, password = upass, ) ########## START MAIN LOGIC ###### if local_license_file_name: try: print "Attempting to retrive License from local disk ..." with open(local_license_file_name, 'r') as fh_license: license_string = fh_license.read() except: print "Can't Open license file named: \"" + local_license_file_name + "\" on disk." #sys.exit() print "No worries. Will attempt to retrieve one from online." if not license_string: print "Attempting to get license online." print "License server requires you to submit EULA." if options.local_eula_file_name: print "Attempting to retrive EULA from local disk first..." try: with open(options.local_eula_file_name, 'r') as fh_eula: eula_string = fh_eula.read() except: print "Can't find EULA file named : \"" + options.local_eula_file_name + "\" on disk." print "No worries. Will attempt to retrieve one during transaction with License Server." # Could also try seeing if Target BIG-IP has one stored # eula_file = get_eula_file(b) if options.reg_keys_string: reg_keys = options.reg_keys_string.split(",") print "reg keys provided" print reg_keys if len(reg_keys) < 1: print "Reg Key list is empty, attempting to retrieve existing keys from the unit" reg_keys = get_reg_keys(b) print "Getting dossier using keys:" + str(reg_keys) dossier_string = get_dossier( b, reg_keys ) # print "dossier = " + str(dossier_output) license_string = get_license_from_F5_License_Server( options.server_hostname, dossier_string, eula_string, email, firstName, lastName, companyName, phone, jobTitle, address, city, stateProvince, postalCode, country ) if license_string: print "License Found. Attempting to installing License on BIGIP:" install_license ( b, license_string ) else: print "Sorry. Could not retrieve License. Check your connection" license_status = get_license_status ( b ) print "License status = " + str(license_status)Create CNAME Resource Record in Zone Runner
Problem this snippet solves: This pyControl v2 script will create a CNAME Record in Zone Runner How to use this snippet: This pyControl v2 script will create a CNAME Record in Zone Runner Usage: Create-CNAME-Record.py Code : #!/bin/env python ''' ---------------------------------------------------------------------------- The contents of this file are subject to the "END USER LICENSE AGREEMENT FOR F5 Software Development Kit for iControl"; you may not use this file except in compliance with the License. The License is included in the iControl Software Development Kit. Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. The Original Code is iControl Code and related documentation distributed by F5. The Initial Developer of the Original Code is F5 Networks, Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 1996-2004 F5 Networks, Inc. All Rights Reserved. iControl (TM) is a registered trademark of F5 Networks, Inc. Alternatively, the contents of this file may be used under the terms of the GNU General Public License (the "GPL"), in which case the provisions of GPL are applicable instead of those above. If you wish to allow use of your version of this file only under the terms of the GPL and not to allow others to use your version of this file under the License, indicate your decision by deleting the provisions above and replace them with the notice and other provisions required by the GPL. If you do not delete the provisions above, a recipient may use your version of this file under either the License or the GPL. ---------------------------------------------------------------------------- ''' import sys import pycontrol.pycontrol as pc import time ####################################################################################### # Example of how to create a Resource Record in Zone Runner (ex. in this case a CNAME) # This example passes in the 'fromurl' keyword as True (default is False), which tells pycontrol # to fetch the WSDL from the remote BigIP. # For more information: # https://devcentral.f5.com/s/wiki/iControl.Management__ResourceRecord__add_cname.ashx ####################################################################################### if pc.__version__.startswith('2.0'): pass else: print "Requires pycontrol version 2.x!" sys.exit() if len(sys.argv) < 4: print "Usage %s ip_address username password" % sys.argv[0] sys.exit() a = sys.argv[1:] b = pc.BIGIP( hostname = a[0], username = a[1], password = a[2], fromurl = True, wsdls = ['Management.ResourceRecord','Management.Zone', 'Management.View']) # create short cut r = b.Management.ResourceRecord # Variables for USER to fill in view = "external" zone_name = "gslb.f5lab.com." domain_name = "www.glslb.f5lab.com" cname = "www.gslb2.f5lab.com" ttl = 60 #Start creating required objects viewzone_obj = r.typefactory.create('Management.ViewZone') viewzone_obj.view_name = view viewzone_obj.zone_name = zone_name view_zone_seq = r.typefactory.create('Management.ViewZoneSequence') view_zone_seq.item = viewzone_obj cname_object = r.typefactory.create('Management.CNAMERecord') cname_object.domain_name = domain_name cname_object.cname = cname cname_object.ttl = ttl cname_object_seq = r.typefactory.create('Management.CNAMERecordSequence') cname_object_seq.item = cname_object cname_object_seq_seq = r.typefactory.create('Management.CNAMERecordSequenceSequence') cname_object_seq_seq.item = cname_object_seq print "Creating CNAME RECORD: " print " \"" + domain_name + " " + str(ttl) + " IN " + cname + "\"\n" try: r.add_cname( view_zones = view_zone_seq, cname_records = cname_object_seq_seq ) except Exception, e: print "Error adding Record %s" % cname_object print e print "Retrieving Resource Records for zone \"" + zone_name + "\":\n" get_rrs_output = [] try: get_rrs_output = r.get_rrs( view_zones = view_zone_seq ) except Exception, e: print "Error retrieving resource records %s" % zone_name print e for element in get_rrs_output: for resource_record in element: print resource_record
delete_devices
Problem this snippet solves: Introduced: EM_v3.0 Use the delete_devices API to remove devices currently being managed by an Enterprise Manager. How to use this snippet: Prototype Python ic.Management.EM.deletedevices(deviceaddresses) C#, Java, and Perl ic.getManagementEM().delete_devices(deviceAddresses); Inputs Name Type Description IP addresses String Sequence This parameter specifies the IP addresses or hostnames of the F5 device(s) that you want to delete from the list of managed devices. Return values There is no explicit return when this API executes successfully. Exceptions The table lists error categories that Enterprise Manager can raise if the API call fails. Exception Name This exception is raised when... AccessDenied The credentials supplied by the client do not satisfy the F5 device's validation requirements. OperationFailed The API call causes an operation error. InvalidArgument The API call contains an invalid argument. Code : To view a code sample, click the relevant code type. C# (.net) Java Perl Pythonget_task_status
Problem this snippet solves: Introduced : EM_v3.0 Use the get_task_status API to determine the current status of a device discovery task. How to use this snippet: Python ic.Management.EM.get_task_status(ids = task_ids) C#, Java, and Perl ic.Management.EM.get_task_status(ids = task_ids); Inputs Name Type Description tasks String Sequence This parameter specifies the task IDs of the tasks for which you want to know the status. Return Values Name Type Description task TaskStatus Sequence The ID of the new discovery task. This ID can return the following values: TASK_STATUS_UNKNOWN TASK_STATUS_PENDING TASK_STATUS_STARTED TASK_STATUS_FAILED TASK_STATUS_COMPLETE TASK_STATUS_RUNNING TASK_STATUS_CANCELING TASK_STATUS_CANCELED TASK_STATUS_ABANDONED TASK_STATUS_TERMINATED TASK_STATUS_TIMED_OUT TASK_STATUS_RESCHEDULED Exceptions Exception Name This exception is raised when... AccessDenied The credentials supplied by the client do not satisfy the F5 device's validation requirements. OperationFailed The API call causes an operation error. InvalidArgument The API call contains an invalid argument. Code : To view a code sample, click the relevant code type. C# (.net) Java Perl PythonLTM Stats via pycontrol version 2
Problem this snippet solves: This pyControl v2 script prints out status and statistics for LTM virtual servers and pools. How to use this snippet: This pyControl v2 script will print out status and statistics for all LTM virtual servers and pools. Usage: ltmStats.py Code : #!/usr/bin/python ''' ---------------------------------------------------------------------------- The contents of this file are subject to the "END USER LICENSE AGREEMENT FOR F5 Software Development Kit for iControl"; you may not use this file except in compliance with the License. The License is included in the iControl Software Development Kit. Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. The Original Code is iControl Code and related documentation distributed by F5. The Initial Developer of the Original Code is F5 Networks, Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 1996-2004 F5 Networks, Inc. All Rights Reserved. iControl (TM) is a registered trademark of F5 Networks, Inc. Alternatively, the contents of this file may be used under the terms of the GNU General Public License (the "GPL"), in which case the provisions of GPL are applicable instead of those above. If you wish to allow use of your version of this file only under the terms of the GPL and not to allow others to use your version of this file under the License, indicate your decision by deleting the provisions above and replace them with the notice and other provisions required by the GPL. If you do not delete the provisions above, a recipient may use your version of this file under either the License or the GPL. ---------------------------------------------------------------------------- ''' __author__ = "jason.rahm" def get_virtuals(obj): vips = obj.get_list() status = obj.get_object_status(virtual_servers = vips) combined = zip(vips,status) for x in combined: print "Virtual: ", x[0] print "\t", x[1].availability_status print "\t", x[1].enabled_status print "\t", x[1].status_description print "\n"*2 def get_pools(obj): pools = obj.get_list() status = obj.get_object_status(pool_names = pools) combined = zip(pools,status) for x in combined: print "Pool: ", x[0] print "\t", x[1].availability_status print "\t", x[1].enabled_status print "\t", x[1].status_description print "\n"*2 def get_virtuals_stats(obj): stats = obj.get_all_statistics() for x in stats.statistics: print "Virtual Server: ", x.virtual_server.name for y in x.statistics: print "\t%s = %s" % (y.type, (y.value.high<<32)|y.value.low) def get_pools_stats(obj): stats = obj.get_all_statistics() for x in stats.statistics: print "Pool: ", x.pool_name for y in x.statistics: print "\t%s = %s" % (y.type, (y.value.high<<32)|y.value.low) if __name__ == "__main__": import pycontrol.pycontrol as pc import getpass from sys import argv if pc.__version__[:3] == '2.0': pass else: print "Requires pycontrol version 2.x!" sys.exit() if len(argv) != 3: exit("Usage ltmStats.py ") host = argv[1] uname = argv[2] upass = getpass.getpass() b = pc.BIGIP( hostname = host, username = uname, password = upass, fromurl = True, wsdls = ['LocalLB.VirtualServer', 'LocalLB.Pool'] ) v = b.LocalLB.VirtualServer p = b.LocalLB.Pool print "Virtual Server status...\n" get_virtuals(v) print "Virtual Server stats...\n" get_virtuals_stats(v) print "Pool status...\n" get_pools(p) print "Pool stats...\n" get_pools_stats(p)
iRule stats formatter
Problem this snippet solves: When you have a load of iRule stats in text format from your F5 device and need to get them into a nicer format. The following Python 3 script takes in a text file in the following format: ------------------------------------------------------------------------------------------------------ Ltm::Rule Event: /web_testing/test_environment_rule:HTTP_RESPONSE ------------------------------------------------------------------------------------------------------ Priority 12 Executions Total 31686860 Failures 0 Aborts 0 CPU Cycles on Executing Average 404058 Maximum 10703959 Minimum 264201 (raw) ------------------------------------------------------------------------------------------------------ Ltm::Rule Event: /web_testing/test_environment_rule:HTTP_REQUEST ------------------------------------------------------------------------------------------------------ Priority 899 Executions Total 31686860 Failures 0 Aborts 0 CPU Cycles on Executing Average 404058 Maximum 10703959 Minimum 264201 Put through the following python script to output a CSV file for further data manipulation. How to use this snippet: Python3 script, to use run the following (can also add in '--o' to define an output file, if not will replace the file extension '.txt' with '.csv' by default): python statformating.py --i my_irule_stats.txt output will be something like Openning 'my_irule_stats.txt' Saving output csv to 'my_irule_stats.csv' Usage/help output: usage: statformating.py [-h] [--i INPUT] [--o OUTPUT] optional arguments: -h, --help show this help message and exit --i INPUT iRule Stats File input file name --o OUTPUT iRule Stats File output csv file name Code : import re import os import argparse def iruleStatsFormat(inputFile, outputFile): print('Openning \'{}\''.format(inputFile)) iruleStats = open(inputFile, 'rt').read() iruleStats = re.sub(r'[ ]{2,}', ' ', iruleStats) iruleStats = re.sub(r'\n\s\(raw\)\s{1,}', '', iruleStats) iruleStats = re.sub(r'[-]{2,}\n', '', iruleStats) iruleStats = re.sub(r'\n ', r'\n', iruleStats) iruleStats = re.sub(r'CPU Cycles on Executing\n', '', iruleStats) iruleStats = re.sub(r'Executions \n', '', iruleStats) iruleStats = re.sub(r'\nPriority (\d{1,})\nTotal (\d{1,})\nFailures (\d{1,})\nAborts (\d{1,})\nAverage (\d{1,})\nMaximum (\d{1,})\nMinimum (\d{1,})', r'\t\1\t\2\t\3\t\4\t\5\t\6\t\7', iruleStats) iruleStats = re.sub(r'Ltm::Rule Event: /(.*?)/(.*?):(.*?\t)', r'\1\t\2\t\3', iruleStats) iruleStats = re.sub(r'Ltm::Rule Event: (.*?):(.*?\t)', r'Common\t\1\t\2', iruleStats) iruleStats = re.sub(r'\n{2,}', r'\n', iruleStats) iruleStats = re.sub(r'\t', r',', iruleStats) print('Saving output csv to \'{}\''.format(outputFile)) with open(outputFile, 'wt') as f: print(iruleStats, file=f) if __name__=='__main__': parser = argparse.ArgumentParser() parser.add_argument("--i", dest='input', help="iRule Stats File input file name", type=str) parser.add_argument("--o", dest='output', help="iRule Stats File output csv file name", type=str, default="") args = parser.parse_args() if args.input and os.path.isfile(args.input): if not args.output: args.output = args.input[:-3] + 'csv' iruleStatsFormat(args.input, args.output) else: parser.print_help()
python f5-vserver-tool
Problem this snippet solves: What the tool does: List all virtual server, nodes and pools and their assignment for a big-ip system or device group print sync status of a device group assign a pool for a specific virtual server How to use this snippet: Usage https://github.com/FalcoSuessgott/f5-vserver-tool Print out information List virtual server and their assigned pools and their respective nodes [user@host ~]$ loadbalancer -l Password: Virtual Server Pool Pool member xxx-8010 xxxx 1.1.1.1:100, 1.1.1.1:200 xxx-8020 xxxx 1.1.1.1:100, 1.1.1.1:200 ....` List all virtual server [user@host ~]$ loadbalancer --list-vserver Password: vserver-1 vserver-2 vserver-3 ....` List all pools [user@host ~]$ loadbalancer --list-pools Password: pool-1 pool-2 pool-3 .... List all nodes [user@host ~]$ loadbalancer --list-nodes Password: node1 node2 node3 .... List all virtual server [user@host ~]$ loadbalancer --list-vserver Password: vserver-1 vserver-2 vserver-3 .... Print out sync-status [user@host ~]$ loadbalancer --show-sync-status Password: deviceGroup is currently In Sync Set assign pool1 to vserver1 [user@host ~]$ loadbalancer -s vserver1 pool1 Password: Changing pool for "vserver1" to "pool1" Synchronizing new configuration to device group "devicegroup" devicegroup is In Sync Print out sample config file [user@host ~]$ loadbalancer -m [AUTH] user = "user" password = "password" [BASIC] devicegroup = "devicegroup" loadbalancer = "fqdn" Code : https://github.com/FalcoSuessgott/f5-vserver-tooliRules Runtime Calculator Spreadsheet Generator
Problem this snippet solves: This script auto-generates the iRules Runtime Calculator Excel spreadsheet by connecting to BIG-IP, collecting the iRule and its stats, and then using the XlsxWriter module to populate the spreadsheet. Tested on BIG-IP 12.1, 13.1, 14.1, 15.0 with python 3.5, 3.7 from Windows 10 and MacOS Mojave Article details if interested. Screenshots: How to use this snippet: usage: runtime_calc_v2.py [-h] host username rule Tested this on version: 13.0 Download here: GitHub Gist
