BIG-IQ Reporting for Subscription and ELA Programs
October 2018 Rev:D Scope This document describes the following: When reporting is needed and how the report is used How BIG-IQ reporting determines what to include in the report Submission models – Manual and Automatic How to find “missing reports” Detailed Report contents When reporting is needed and how the report is used For subscription and ELA programs, administrator must use BIG-IQ to issue licenses to BIG-IP VE instances and keep track of usage. By the 10th day of the month, a usage report is submittedfrom BIG-IQ to F5 for processing from the previous calendar month. The report contains information on the license grant and revoke operations across your infrastructure where BIG-IQ is managing the licenses. How BIG-IQ reporting determines what to include in the report It is important to understand how BIG-IQ determines what to include in the report because it affects what you need to send to F5 on a periodic basis. BIG-IQ keeps track of all the licensing activities (grant of license, revoke of license) in its database as a time series of events. Each time a report is generated, BIG-IQ remembers the end date/time of the report and subsequent reports only report on transactions occurring after the last report. Consider the following reporting activities: “Report 1” will contain all transactions plus all outstanding license grants (not yet revoked) from the system’s first start up until 6/2/2018 12:12:00Z. “Report 2” will contain all transactions plus all outstanding license grants from 6/2/2018 12:12:01Z to 7/3/2018 13:45:00Z. When “Report 3” is generated, it will contain all transactions plus all outstanding grants from 7/3/2017 13:45:01Z to the date and time the report was generated. The BIG-IQ reporting system thus never takes a “date range” for administrators to report on, instead, BIG-IQ keeps track of what’s been reported and what has not been reported so you don’t have to keep track of it. To illustrate, perhaps by an extreme example, consider the following. Perhaps you were“testing” the reporting capability to see how it works and/or to debug connectivity issues to F5. In this case, the generated Reports 2, 3, 4, 5, 6, and 7 and all of those reports must be submitted for the June 2018 reporting period. Reason is report #7 only contains reporting transactions from 6/28/2018 23:21:00Z to 7/3/2018 13:45:00Z. This would leave a reporting gap between 6/2/2018 12:12:01Z (the May report to the far left of the diagram) and 6/28/2018 23:21:00Z. It is OK to generate multiple reports during the month, but please keep each one and when the submission date arrives, send all of the reports in to prevent reporting gaps. Submission models – Manual and Automatic There are two ways to submit reports: Automatic – submit over the internet directly from BIG-IQ Manual – extract the report, and EMAIL the report to F5 Automatic Automatic report submission is the least-effort method of submitting usage reporting to F5. However, it requires the following: BIG-IQ must be able to resolve api.f5.com from your DNS configuration BIG-IQ must be able connect to api.f5.com to post 443 (SSL). You can test this by logging into your BIG-IQ system as “root” (SSH connection), and telneting to the service: $ telnet api.f5.com 443 Trying 104.219.110.164... Connected to api.f5.com. Escape character is '^]'. ^C If you see the “Trying <ip>” address message from telnet, this means DNS is working and your BIG-IQ system is able to resolve the IP address for api.f5.com. If you see the two yellow highlighted lines, it means BIG-IQ was able to connect to api.f5.com on port 443. This usually indicates you’ll be able to submit a report to F5 automatically. If this does not work as described and you want to submit automatically, you’ll need to enable a firewall rule (or whatever ACL'ish thing you're using) allowing your BIG-IQ to reach api.f5.com over port 443. Alternatively, BIG-IQ can be configured to use a network proxy to make a connection to api.f5.com. The proxy configuration is shown in the screen shot below: Please note, because BIG-IQ is transferring the report over SSL, the content is encrypted in transit from your BIG-IQ to F5’s api.f5.com end-point. To submit a report automatically, log into BIG-IQ as “admin” or a License Manager persona, click the “Devices” table, open “LICENSE MANAGEMENT” on the left navigation, then click “Licenses”. This is shown below: Next, in the type, select “Utility Billing Report”, select and move the Licenses you are using to license your BIG-IPs, and click “Generate and automatically submit report to F5”. In the lower right, click submit, and the system will attempt the transaction and indicate if it was a success or failure. NOTE: If the automatic submission failed, you’ll need to manually extract, download and email the report to F5. Currently, there is no way to retry sending an already-created report. Manual Manual report submission is needed when your BIG-IQ cannot submit a report automatically. For manual submission, you’ll extract the report from BIG-IQ, then email it to F5 to SalesLicensingPrograms (at) f5.com. Starting at the same place in the BIG-IQ UI, you’ll click Report, then pick type as “Utility Billing Report”, move the licenses you are using as part of the program to the Selected column/box, choose “Generate and manually submit report to F5”. Please ignore the message asking you to call F5 support unless you’ve encountered a product problem doing this workflow. Finally, click Download, then attach the report to an EMAIL and send to vesubscriptions (at) f5.com. Automated Reports Currently, BIG-IQ does not have a scheduling mechanism. You can automate your monthly reports by leveraging a sample script on GitHub and then using your “crontab” scheduler (or equivalent) to create by the 10th day of the calendar month. Ensure that you have chosen the Generate and automatically submit report to F5 Reporting Option. How to find “missing reports” In the event you get a notice from F5 for a “missing report” for a time period during the reporting period, you can find the reports directly on your BIG-IQ system. The reports are written to /var/config/rest/license-reports – simply log into BIG-IQ as “root” and SCP (secure copy) the reports from BIG-IQ to your system, then attach and send the reports to F5 manually to SalesLicensingPrograms (at) f5.com. Please note: If you are using BIG-IQ in an HA cluster, the contents of /var/config/rest/license - reports is not replicated between the active and standby BIG-IQ systems. If there was a failover event during the month, you may need to look the contents of /var/config/rest/license-reports in both systems to locate the missing report(s). Report Contents The report is JSON format – the table below describes each field and following the table is a sample JSON report. Elements Description product Identifies the F5 product submitting the report version Indicates the version of the F5 product sending the report reportType Indicates the type of reporting being sent poolType Indicates the type of license pool regkey BIG-IQ system regkey poolRegkey Service catalog regkey poolName Name of your pool periodStarted Report coverage starting date & time periodEnded Report coverage ending date & time For Each Device: id Unique BIG-IP id, if device is licensed as 'managed' or 'unmanaged' address IP address of BIG-IP hostname Name of BIG-IP, if device is licensed as 'managed' or 'unmanaged' type How device was licensed: managed, unmanaged, unreachable sku Service catalog assigned to device uom Unit-of-measure (yearly for subscription and ELA) granted Date/time of license grant revoked Date/time of license revocation (if still licensed, this is omitted) Sample Report { "product": "big-iq", "version": "5.0.0.0.0.3007", "reportType": "pool usage", "poolType": "CLPv2", "regKey": "U8917-466961-104-8159544-3215874", "poolRegkey": "xxxxx-xxxxx-xxxxx-xxxxx-xxxxxxx", "poolName": "xxxxx-xxxxx-xxxxx-xxxxx-xxxxxxx", "periodStarted": "2016-06-03T17:20:07Z", "periodEnded": "2016-06-04T12:24:14Z", "records": [ { "id": "f80f00e0-000f-4f4f-8012f-1fee0f6ff500", "address": "10.128.10.10", "hostname": "BIG-IP-001.sassy.molassy", "type": "MANAGED", "sku": "F5-BIG-MSP-LTM-200M", "uom": "hourly", "granted": "2016-06-02T21:41:09Z" }, { "id": "a3225f5f-8ffb-40ff-90ff-9547d00f755f", "address": "10.128.10.20", "hostname": "BIG-IP-002.sassy.molassy", "type": "MANAGED", "sku": "F5-BIG-MSP-LTM-200M", "uom": "hourly", "granted": "2016-06-02T21:44:10Z", "revoked": "2016-06-03T21:45:07Z" } ] } Support Need additional help or have questions? Need additional help or have questions? Contact your SE for any additional questions or email SalesLicensingPrograms (at) f5.com. As always, let us know what else we can write about regarding licensing and how we can help you manage your BIG-IP environments. Happy adminining.
Create or edit "Chargeback Tag" in ELA license
Problem this snippet solves: You are using BIG-IQ Centralized Manager or BIG-IQ License Manager to manage your ELA licenses. The Chargeback Tag is visible on the customer usage reports, and can be useful if you bill across multiple departments within the account. When you first provision a license to a BIG-IP, you have the option to add a free text entry into a field call "Chargeback Tag". However, if you missed this or want to change this tag at a later time, it is currently not possible to add or edit the entry via the BIG-IQ GUI. How to use this snippet: You will need to copy the bash code below and then create a simple CSV file including details of the BIG-IP’s IP Address or MachineID and corresponding Chargeback Tag text you want to create or edit into the /shared directory of BIG-IQ. You will then run the script, which will confirm "Updating" followed by each IP address or MachineID and corresponding Chargeback Tag text. The Chargeback Tag can be seen in the BIG-IQ GUI: Devices >> License Management >> Licenses >> select ELA name >> Select Offering Name of the license type. The Chargeback Tag can also be seen in the reports created and shared each quarter by the Customer Success team. In the /shared directory of BIG-IQ, create a CSV file named update.csv and enter the value pair on a line for each BIG-IP device tag to chage using the format: <BIG-IP IP address1>,<new tag text> <BIG-IP IP address2>,<new tag text> Or <BIG-IP MachineID 1>,<new tag text> <BIG-IP MachineID 2>,<new tag text> For example: cd /shared vi update.csv Enter values: 10.11.10.43,new tag value 10.23.12.100, new tag value Or xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx,new tag value yyyyyyy-yyyy-yyy-yyy-yyyyyyyyyyyyy,new tag value Save the file. Please note that no inverted commas are needed, and spaces are accepted within the Chargeback Tag value Create a file named update.sh in the same directory using your favourite editor, copy and paste the provided code below and make this file executable with: chmod u+x update.sh Run the script by entering ./update.sh If everything runs correctly, the output showing: Updating <ip address> to 'new tag' Or Updating <MachineId> to 'new tag' will be shown for each entry within the update.csv file Code : > /var/tmp/offerings > /var/tmp/lic-audit curl -s localhost:8100/cm/device/licensing/pool/utility/licenses/ | jq .items[].regKey -r | while read -r regkey ; do curl -s localhost:8100/cm/device/licensing/pool/utility/licenses/$regkey/offerings | jq .items[].id -r | while read -r off ; do curl -s localhost:8100/cm/device/licensing/pool/utility/licenses/$regkey/offerings/$off/members/ | jq .items[].id -r | while read -r id ; do curl -s localhost:8100/cm/device/licensing/pool/utility/licenses/$regkey/offerings/$off/members/$id | jq '.|{add:.deviceAddress,selfLink:.selfLink,deviceMachineId:.deviceMachineId}' -c >> /var/tmp/offerings done done done curl -s localhost:8100/cm/device/licensing/audit/ | jq '.items[]|{add:.address,id:.id,stat:.status,deviceMachineId:.machineId}' -c | grep GRAN >> /var/tmp/lic-audit cat update.csv | while read -r line ; do IP=$(echo $line | cut -d ',' -f1) VAL=$(echo $line | cut -d ',' -f2) echo "Updating $IP to '$VAL'" auditid=$(grep $IP /var/tmp/lic-audit | head -n1 | jq .id -r ) curl -s localhost:8100/cm/device/licensing/audit/$auditid | jq 'del(.generation,.lastUpdateMicros,.chargebackTag)' | jq --arg chargebackTag "$VAL" '. + {chargebackTag: $chargebackTag}' > /var/tmp/update-lic-audit curl -s localhost:8100/cm/device/licensing/audit/$auditid -XPUT -d @/var/tmp/update-lic-audit -o /dev/null offeringid=$(grep $IP /var/tmp/offerings | jq .selfLink -r | egrep -o '/cm/device/.*') curl -s "localhost:8100$offeringid" | jq 'del(.generation,.lastUpdateMicros,.chargebackTag)' | jq --arg chargebackTag "$VAL" '. + {chargebackTag: $chargebackTag}' > /var/tmp/update-offering curl -s "localhost:8100$offeringid" -XPUT -d @/var/tmp/update-offering -o /dev/null done Tested this on version: BIG-IQ 8.3.0Code to create unreachable ELA license files from BIG-IQ
Problem this snippet solves: *NOTE* if you are upgrading your BIG-IP,please refer to F5 solution:https://support.f5.com/csp/article/K13540950 BIG-IQ traditionally expects to be able to reach any BIG-IP devices it is going to license. This code helps create a license file from the ELA SKU offerings which can be applied on an Unreachable BIG-IP. I've added some troubleshoting steps at the end of the article, Dossier errors seen on the BIG-IP, just in case! How to use this snippet: SSH into the BIG-IP device and run the following command to gain the MAC address of the management interface tmsh show sys mac-address | grep -i interface [root@bigip1:Active:Standalone] config # tmsh show sys mac-address | grep -i interface ll:50:56:xx:xx:36net interfacemgmtmac-address xxxxxxxxxxxxxxxxxnet interface1.3mac-address xxxxxxxxxxxxxxxxxnet interface1.1mac-address xxxxxxxxxxxxxxxxx net interface1.2mac-address In the example above the MAC address we need is “ll:50:56:xx:xx:36” Now SSH into the BIG-IQ Move into the /shared directory (cd /shared) Copy over the Create-license.PY python script and run it by typing python Create-license.py The script runs and will prompt you for the following information [root@Preece-bigiq-cm1:Active:Standalone] shared # python Create-license.py Enter BIG-IQ user ID: admin Enter BIG-IQ Password: Enter Management IP address of BIG-IQ: 44.131.176.101 Enter Management IP address of BIG-IP to be licensed: 44.131.176.22 Enter Management MAC address of BIG-IP to be licensed: ll:50:56:xx:kk:36 Enter the name of the License Pool from which to take BIG-IP license: Load-18 Enter the license name to be assigned to the BIG-IP: F5-BIG-MSP-BT-1GIPIF-LIC-DEV Enter hypervisor used, valid options are: aws, azure, gce, hyperv, kvm, vmware,xen: vmware Optional: Enter chargeback tag if required: Department-A Optional: Enter tenant name if required: Customer-B Once the details have been filled in the script authenticates to the BIG-IQ and generates the license (30 seconds) If everything went well, you will be presented with a success message. The license file is saved as IP-address_bigip.license in the same directory as you run the script Using SCP copy the new license file from the BIG-IQ to your desktop. Copy the license file into the /config directory of the BIG-IP device. Rename the file, copy ip-address.bigip.license bigip.license Reload the license by typing reloadlic Observe the BIG-IP device restart its services and show as active. You can review in the GUI (System—License) and provision modules as needed. Code : import getpass # used to hide the users password input import json import os import requests from time import sleep """ This script uses the BIG-IQ API to license an unreachable (dark site) BIG-IP. The BIG-IQ licensing API needs certain details provided in order to license an appliance, these details can either be provided in a file call lic-data.json or if that file does not exist you will be prompted to enter them. The minimum contents of lic-data.json should be: { "licensePoolName": " -- Enter License Pool Name here. License Pool name can be found in BIG-IQ GUI -- ", "command": "assign", "address": " -- Enter MGMT IP Address of BIG-IP here -- ", "assignmentType": "UNREACHABLE", "macAddress": " -- Enter MAC address of MGMT IP for the BIG-IP here -- ", "hypervisor": " -- Enter hypervisor value here options are; aws, azure, gce, hyperv, kvm, vmware, xen: --", "unitOfMeasure": "yearly", "skuKeyword1": "-- Enter License Name here. License Name (or Offering name) can be found in the BIG-IQ GUI -- " } Additional Optional key:value pairs can be added to the JSON file to afix useful tags to the license. The json file with optional key:value pairs looks like: { "licensePoolName": " -- Enter License Pool Name here. License Pool name can be found in BIG-IQ GUI -- ", "command": "assign", "address": " -- Enter MGMT IP Address of BIG-IP here -- ", "assignmentType": "UNREACHABLE", "macAddress": " -- Enter MAC address of MGMT IP for the BIG-IP here -- ", "hypervisor": " -- Enter hypervisor value here options are; aws, azure, gce, hyperv, kvm, vmware, xen: --", "unitOfMeasure": "yearly", "skuKeyword1": "-- Enter License Name here. License Name (or Offering name) can be found in the BIG-IQ GUI -- ", "chargebackTag": "OPTIONAL: Remove this line if you are not going to use it", "tenant": "OPTIONAL: Remove this line if you are not going to use it" } A completed minimal lic-data.json file will look like this: { "licensePoolName": "byol-pool-utility", "command": "assign", "address": "10.1.1.10", "assignmentType": "UNREACHABLE", "macAddress": "06:ce:c2:43:b3:05", "hypervisor": "kvm", "unitOfMeasure": "yearly", "skuKeyword1": "F5-BIG-MSP-BT-P3-3GF-LIC-DEV" } lic-data.json must reside in the directory from which you execute this python script. """ def bigiqAuth(_bigiqAuthUrl, _bigiqCredentials): """ This function authenticates with BIG-IQ and collects the authentication token provided. Theo token will be used for subsequent calls to BIG-IQ """ _errFlag=0 try: _bigiqAuthInfo=_bigiq_session.post(_bigiqAuthUrl, data=json.dumps(_bigiqCredentials), verify=False) print(_bigiqAuthUrl) _bigiqAuthInfo.raise_for_status() print("Response code: %s" %_bigiqAuthInfo.status_code) except requests.exceptions.HTTPError as err: print(err) _errFlag=1 #end try if _errFlag==0: _bigiqResponse=_bigiqAuthInfo.json() _bigiqToken=_bigiqResponse['token'] for _token in _bigiqToken: if (_token == 'token'): _bigiqAuthToken=(_bigiqToken[_token]) # End if # Next _authHeaders={ "X-F5-Auth-Token": "{_authToken}".format(_authToken=_bigiqAuthToken) } else: _authHeaders=0 #end if print("** Completed Authentication ***") return(_authHeaders); #End Def def extractLicense(_rawLicenseJSON): """ This function pulls the generated license from BIG-IQ """ for _license in _rawLicenseJSON: if (_license=='licenseText'): _extractedLicense=_rawLicenseJSON[_license] #end if if (_license=='status'): if (_rawLicenseJSON[_license]=="FINISHED"): print("***** License has been assigned *****") else: _extractedLicense="FAILED" #end if #end if #next return(_extractedLicense); #End def def licenseData(): """ This function read the lic-data.json file. If it does not exist you will be prompted to enter the necessary values. """ if os.path.exists('lic-data.json'): with open('./lic-data.json') as licfile: _licdata = json.load(licfile) else: _bigipAddress=raw_input("Enter Management IP address of BIG-IP to be licensed: ") _bigipMACaddress=raw_input("Enter Management MAC address of BIG-IP to be licensed: ") _licensePoolName=raw_input("Enter the name of the License Pool from which to take BIG-IP license: ") _licenseSKU=raw_input("Enter the license name to be assigned to the BIG-IP: ") _hypervisorType=raw_input("Enter hypervisor used, valid options are: aws, azure, gce, hyperv, kvm, vmware, xen: ") _chargebackTag=raw_input("Optional: Enter chargeback tag if required: ") _tenantTag=raw_input("Optional: Enter tenant name if required: ") _licdata={ "licensePoolName": "{_licensePool}".format(_licensePool=_licensePoolName), "command": "assign", "address": "{_bigipIP}".format(_bigipIP=_bigipAddress), "assignmentType": "UNREACHABLE", "macAddress": "{_bigipMAC}".format(_bigipMAC=_bigipMACaddress), "hypervisor": "{_hypervisor}".format(_hypervisor=_hypervisorType), "unitOfMeasure": "yearly", "skuKeyword1": "{_license}".format(_license=_licenseSKU), "chargebackTag": "{_chargeback}".format(_chargeback=_chargebackTag), "tenant": "{_tenant}".format(_tenant=_tenantTag) } # End if return(_licdata); def urlConstruction(_bigiqUrl, _bigiqIP): """ This function rewrites the selflink URL returned by BIG-IQ to reflect BIG-IQ management IP address rather than localhost """ count=0 _urlDeConstruct=_bigiqUrl.split("/") _urlReConstruct="" for _urlElement in _urlDeConstruct: #print("%d %s" %(count,_urlElement)) if (_urlElement=="https:"): _urlReConstruct=_urlReConstruct+_urlElement+"//" elif (_urlElement=="localhost"): _urlReConstruct=_urlReConstruct+_bigiqIP else: if (_urlElement!=""): _urlReConstruct=_urlReConstruct+"/"+_urlElement #end if #end if count+=1 #Next return(_urlReConstruct); #End Def _userID=raw_input("Enter BIG-IQ user ID: ") _password=getpass.getpass(prompt="Enter BIG-IQ Password: ") _bigiqAddress=raw_input("Enter Management IP address of BIG-IQ: ") _credPostBody={ "username": "{_uname}".format(_uname=_userID), "password": "{_pword}".format(_pword=_password), "loginProvideriName": "RadiusServer" } _deviceToBeLicensed=licenseData() _bigipAddress=_deviceToBeLicensed['address'] print("BIG-IP Address is: %s" %_bigipAddress) _bigiq_session=requests.session() _bigiq_auth_url="https://{_bigiqIP}/mgmt/shared/authn/login".format(_bigiqIP=_bigiqAddress) # Authenticates with BIG-IQ _bigiqAuthHeader=bigiqAuth(_bigiq_auth_url, _credPostBody) # if _bigiqAuthHeader==0: print("Unable to authenticate with BIG-IQ. Check BIG-IQ reachability and credentials") else: _bigiq_url1="https://{_bigiqIP}/mgmt/cm/device/tasks/licensing/pool/member-management".format(_bigiqIP=_bigiqAddress) # # --- This section requests the license from BIG-IQ. Posting the criteria as laid out in the _deviceToBeLicensed JSON blob # _errFlag=0 try: _bigiqLicenseDevice=_bigiq_session.post(_bigiq_url1, headers=_bigiqAuthHeader, data=json.dumps(_deviceToBeLicensed), verify=False) _bigiqLicenseDevice.raise_for_status() print("Response code: %s" %_bigiqLicenseDevice.status_code) except requests.exceptions.HTTPError as err: print("Issue received, check rquest and or check connectivity %s" %err) _errFlag=1 #end try if _errFlag==0: #print(_bigiqLicenseDevice.status_code) _bigiqResponse=_bigiqLicenseDevice.json() print(_bigiqResponse) print(_bigiqResponse['selfLink']) _bigiqLicenseStatus_url=_bigiqResponse['selfLink'] _bigiqLicenseStatus_url=urlConstruction(_bigiqLicenseStatus_url, _bigiqAddress) print(_bigiqLicenseStatus_url) print("--- Standby for 30 seconds whilst BIG-IQ generates license ---") sleep(30) _errFlag1=0 try: _licenseStatus=_bigiq_session.get(_bigiqLicenseStatus_url, headers=_bigiqAuthHeader, verify=False) _licenseStatus.raise_for_status() print("Response code: %s" %_licenseStatus.status_code) except requests.exceptions.HTTPError as err: print("Issue received, check rquest and or check connectivity %s" %err) _errFlag=1 #end try if _errFlag==0: print(_licenseStatus.content) _licenseStatusDetail=_licenseStatus.json() _licenseOutput=extractLicense(_licenseStatusDetail) if (_licenseOutput=="FAILED"): print("***** License Assignment Failed. Most likely a valid license already exists for device, revoke it before applying a new license *****") else: _licenseFname=(_bigipAddress+"_bigip.license") _licensefile=open(_licenseFname, "w") _licensefile.write("%s" %_licenseOutput) _licensefile.close() print(_licenseOutput) print("***** SUCCESS, the license is stored here %s *****" %_licenseFname) #end if #end if #end if #end if Tested this on version: 13.x, 14.x, 15.x and 16.x Troubleshooting When you apply the license to the BIG-IP you may see an error similar to: License is not operational (expired or digital signature does not match contents) This could simply be that you copy and paste the license file badly, please use MD5SUM on the BIG-IQ to the output license file and compare to the same file on the BIG-IP Example: md5sum 10.2.3.4_bigip.license You can also review the /var/log/ltm file for "Dossier error" messages Dossier error: 1 (MAC address is mismatched) Dossier error: 12 (Hypervisor is mismatched) If this does not help, please open a support case and attach a recent qkview file.
