AFM reporting no data
Hi! I have an AFM installation here that seems to be working very well as firewall and ddos protection, but the problem is that none of the reports are working. I have a logging profile created for all the VSs and the publisher is set as local-db-publisher everywhere. Logs working: Reports not working: It is also possible to observe some javascript errors being report in console: My logging profile: security log profile Log_Local { dos-network-publisher local-db-publisher ip-intelligence { log-publisher local-db-publisher } network { Log_Local { filter { log-ip-errors enabled log-tcp-errors enabled } publisher local-db-publisher } } port-misuse { log-publisher local-db-publisher } protocol-dns-dos-publisher local-db-publisher protocol-inspection { log-publisher local-db-publisher } protocol-sip-dos-publisher local-db-publisher traffic-statistics { active-flows enabled log-publisher local-db-publisher missed-flows enabled reaped-flows enabled syncookies enabled syncookies-whitelist enabled } } Am I doing something wrong? Thanks!BIG-IQ Reporting for Subscription and ELA Programs
October 2018 Rev:D Scope This document describes the following: When reporting is needed and how the report is used How BIG-IQ reporting determines what to include in the report Submission models – Manual and Automatic How to find “missing reports” Detailed Report contents When reporting is needed and how the report is used For subscription and ELA programs, administrator must use BIG-IQ to issue licenses to BIG-IP VE instances and keep track of usage. By the 10th day of the month, a usage report is submittedfrom BIG-IQ to F5 for processing from the previous calendar month. The report contains information on the license grant and revoke operations across your infrastructure where BIG-IQ is managing the licenses. How BIG-IQ reporting determines what to include in the report It is important to understand how BIG-IQ determines what to include in the report because it affects what you need to send to F5 on a periodic basis. BIG-IQ keeps track of all the licensing activities (grant of license, revoke of license) in its database as a time series of events. Each time a report is generated, BIG-IQ remembers the end date/time of the report and subsequent reports only report on transactions occurring after the last report. Consider the following reporting activities: “Report 1” will contain all transactions plus all outstanding license grants (not yet revoked) from the system’s first start up until 6/2/2018 12:12:00Z. “Report 2” will contain all transactions plus all outstanding license grants from 6/2/2018 12:12:01Z to 7/3/2018 13:45:00Z. When “Report 3” is generated, it will contain all transactions plus all outstanding grants from 7/3/2017 13:45:01Z to the date and time the report was generated. The BIG-IQ reporting system thus never takes a “date range” for administrators to report on, instead, BIG-IQ keeps track of what’s been reported and what has not been reported so you don’t have to keep track of it. To illustrate, perhaps by an extreme example, consider the following. Perhaps you were“testing” the reporting capability to see how it works and/or to debug connectivity issues to F5. In this case, the generated Reports 2, 3, 4, 5, 6, and 7 and all of those reports must be submitted for the June 2018 reporting period. Reason is report #7 only contains reporting transactions from 6/28/2018 23:21:00Z to 7/3/2018 13:45:00Z. This would leave a reporting gap between 6/2/2018 12:12:01Z (the May report to the far left of the diagram) and 6/28/2018 23:21:00Z. It is OK to generate multiple reports during the month, but please keep each one and when the submission date arrives, send all of the reports in to prevent reporting gaps. Submission models – Manual and Automatic There are two ways to submit reports: Automatic – submit over the internet directly from BIG-IQ Manual – extract the report, and EMAIL the report to F5 Automatic Automatic report submission is the least-effort method of submitting usage reporting to F5. However, it requires the following: BIG-IQ must be able to resolve api.f5.com from your DNS configuration BIG-IQ must be able connect to api.f5.com to post 443 (SSL). You can test this by logging into your BIG-IQ system as “root” (SSH connection), and telneting to the service: $ telnet api.f5.com 443 Trying 104.219.110.164... Connected to api.f5.com. Escape character is '^]'. ^C If you see the “Trying <ip>” address message from telnet, this means DNS is working and your BIG-IQ system is able to resolve the IP address for api.f5.com. If you see the two yellow highlighted lines, it means BIG-IQ was able to connect to api.f5.com on port 443. This usually indicates you’ll be able to submit a report to F5 automatically. If this does not work as described and you want to submit automatically, you’ll need to enable a firewall rule (or whatever ACL'ish thing you're using) allowing your BIG-IQ to reach api.f5.com over port 443. Alternatively, BIG-IQ can be configured to use a network proxy to make a connection to api.f5.com. The proxy configuration is shown in the screen shot below: Please note, because BIG-IQ is transferring the report over SSL, the content is encrypted in transit from your BIG-IQ to F5’s api.f5.com end-point. To submit a report automatically, log into BIG-IQ as “admin” or a License Manager persona, click the “Devices” table, open “LICENSE MANAGEMENT” on the left navigation, then click “Licenses”. This is shown below: Next, in the type, select “Utility Billing Report”, select and move the Licenses you are using to license your BIG-IPs, and click “Generate and automatically submit report to F5”. In the lower right, click submit, and the system will attempt the transaction and indicate if it was a success or failure. NOTE: If the automatic submission failed, you’ll need to manually extract, download and email the report to F5. Currently, there is no way to retry sending an already-created report. Manual Manual report submission is needed when your BIG-IQ cannot submit a report automatically. For manual submission, you’ll extract the report from BIG-IQ, then email it to F5 to SalesLicensingPrograms (at) f5.com. Starting at the same place in the BIG-IQ UI, you’ll click Report, then pick type as “Utility Billing Report”, move the licenses you are using as part of the program to the Selected column/box, choose “Generate and manually submit report to F5”. Please ignore the message asking you to call F5 support unless you’ve encountered a product problem doing this workflow. Finally, click Download, then attach the report to an EMAIL and send to vesubscriptions (at) f5.com. Automated Reports Currently, BIG-IQ does not have a scheduling mechanism. You can automate your monthly reports by leveraging a sample script on GitHub and then using your “crontab” scheduler (or equivalent) to create by the 10th day of the calendar month. Ensure that you have chosen the Generate and automatically submit report to F5 Reporting Option. How to find “missing reports” In the event you get a notice from F5 for a “missing report” for a time period during the reporting period, you can find the reports directly on your BIG-IQ system. The reports are written to /var/config/rest/license-reports – simply log into BIG-IQ as “root” and SCP (secure copy) the reports from BIG-IQ to your system, then attach and send the reports to F5 manually to SalesLicensingPrograms (at) f5.com. Please note: If you are using BIG-IQ in an HA cluster, the contents of /var/config/rest/license - reports is not replicated between the active and standby BIG-IQ systems. If there was a failover event during the month, you may need to look the contents of /var/config/rest/license-reports in both systems to locate the missing report(s). Report Contents The report is JSON format – the table below describes each field and following the table is a sample JSON report. Elements Description product Identifies the F5 product submitting the report version Indicates the version of the F5 product sending the report reportType Indicates the type of reporting being sent poolType Indicates the type of license pool regkey BIG-IQ system regkey poolRegkey Service catalog regkey poolName Name of your pool periodStarted Report coverage starting date & time periodEnded Report coverage ending date & time For Each Device: id Unique BIG-IP id, if device is licensed as 'managed' or 'unmanaged' address IP address of BIG-IP hostname Name of BIG-IP, if device is licensed as 'managed' or 'unmanaged' type How device was licensed: managed, unmanaged, unreachable sku Service catalog assigned to device uom Unit-of-measure (yearly for subscription and ELA) granted Date/time of license grant revoked Date/time of license revocation (if still licensed, this is omitted) Sample Report { "product": "big-iq", "version": "5.0.0.0.0.3007", "reportType": "pool usage", "poolType": "CLPv2", "regKey": "U8917-466961-104-8159544-3215874", "poolRegkey": "xxxxx-xxxxx-xxxxx-xxxxx-xxxxxxx", "poolName": "xxxxx-xxxxx-xxxxx-xxxxx-xxxxxxx", "periodStarted": "2016-06-03T17:20:07Z", "periodEnded": "2016-06-04T12:24:14Z", "records": [ { "id": "f80f00e0-000f-4f4f-8012f-1fee0f6ff500", "address": "10.128.10.10", "hostname": "BIG-IP-001.sassy.molassy", "type": "MANAGED", "sku": "F5-BIG-MSP-LTM-200M", "uom": "hourly", "granted": "2016-06-02T21:41:09Z" }, { "id": "a3225f5f-8ffb-40ff-90ff-9547d00f755f", "address": "10.128.10.20", "hostname": "BIG-IP-002.sassy.molassy", "type": "MANAGED", "sku": "F5-BIG-MSP-LTM-200M", "uom": "hourly", "granted": "2016-06-02T21:44:10Z", "revoked": "2016-06-03T21:45:07Z" } ] } Support Need additional help or have questions? Need additional help or have questions? Contact your SE for any additional questions or email SalesLicensingPrograms (at) f5.com. As always, let us know what else we can write about regarding licensing and how we can help you manage your BIG-IP environments. Happy adminining.BIG-IQ Reporting for Subscriptions and ELA - How to Generate and Revoke Licenses for Isolated BIG-IP VE Instances
October 2018 Rev:D Scenario Some network designs may disallow BIG-IQ from reaching BIG-IP VE instances over the network due to network design, network partitioning, security reasons... you get the idea. BIG-IQ’s usual operation requires BIG-IQ to connect to port 443 on the BIG-IP VE’s management IP address. However, when the networks are partitioned in a way that does not allow the BIG-IQ to connect to the BIG-IP, a system outside of BIG-IQ is required to: POST an operation to generate a license GET the license from BIG-IQ Copy the license to the target BIG-IP VE instance And later, when the BIG-IP VE is no longer operational, revoking the license from BIG-IQ. This document explains how to manage licenses on BIG-IP VEs in this network design. Step 1: Gather your BIG-IP VE information for the licensing operation In order to successfully license BIG-IP VE, BIG-IQ needs specific information from the BIG-IP VE to be licensed. Log into the BIG-IP VE instance using “root” credentials Execute the following command: # get_dossier -b TBD -c Record the MAC address. You will also need to know the hypervisor your BIG-IP VE is running on. Example: [root@biq1-yabba-dabba-do:Active:Standalone] config # get_dossier -b d -c F5_BIOS_ID=Phoenix Technologies LTD 6.00 04/05/2016 VMware Virtual Platform None F5_MAC=00:50:56:bf:02:02 F5_PROBES=0x01000013 F5_SYS_UUID=543ff2ec-1c2f-844a-23d0-1e226b90678e F5_KEY=d F5_VERSION=BIG-IQ 6.0.1 0.0.469 Step 2: Requesting a License from BIG-IQ for your BIG-IP VE For this step, you need to have the following information: The name of the license pool in BIG-IQ you want to license the BIG-IP VE from The IP address of the BIG-IP VE device The MAC address (from step 1) The hypervisor type The offering name you wish to assign to the BIG-IP VE instance We’ll walk you through each one. The name of the pool comes from your BIG-IQ centralized management console. To get the name of the pool: Log into BIG-IQ Click the Devices tab On the left navigation, open LICENSE MANAGEMENT On the left navigation, click on Licenses Here you will see your license pools – the pool names are shown in the red rectangle in the example screen shot below. In this case, there are two pools “Engineering-Pool” and “IT-Pool” The next one we’ll discuss is the offering (service catalog) name, this will become important when we form the JSON request for the post. Click on your pool name, this will open the properties of the pool It looks as shown in the following diagram, highlighted by the red rectangle: The BIG-IQ API allows you to specify one or two “search keywords” to be used for matching the offering you wish to assign to your BIG-IP VE instance. For example, the first search keyword could be the feature level you’d like and the second the throughput, such as “LTM” and “10G”. Or, you can just specify the entire offering name as the only search keyword for an exact patch, such as “F5-BIG-MSP-LTM-5G-LIC-DEV”. Once you have the information available, form your JSON request, here is an example you can start with, we’ll take you through each part: { "licensePoolName": "Engineering-Pool", "command": "assign", "address": "192.0.2.3", "assignmentType": "UNREACHABLE", "macAddress": "FA:16:3E:1B:6D:34", "hypervisor": "vmware", "unitOfMeasure": "yearly", "skuKeyword1": "LTM", "skuKeyword2": "10G" } Elements Description licensePoolName Name of your pool as previously discussed in this article command The request to BIG-IQ to assign/allocate a license for this instance of BIG-IP VE address IP of your BIG-IQ (use the management IP) assignmentType must be 'unreachable': this instructs BIG-IQ to avoid trying to POST the license to the BIG-IP VE macAddress The MAC addressed obtained using method previously discussed hypervisor Underlying hypervisor hosting the BIG_IP VE you wish to license. Valid values are "aws","azure","gce","vmware","hyperv","kvm", and "xen". unitOfMeasure Must be "yearly" when you are using a license pool as part of the F5 subscription license or Enterprise Licensing Agreement (ELA) programs. skuKeyword1 Is the first search keyword skuKeyword2 Is the second search keyword The search keywords are optional; if none are specified, you’ll get the first license pool BIG-IQ matches on. If only one is specified, you’ll get the first license pool finds matching the search string. Here is an example with a single search keyword: { "licensePoolName": "Engineering-Pool", "command": "assign", "address": "192.0.2.3", "assignmentType": "UNREACHABLE", "macAddress": "FA:16:3E:1B:6D:34", "hypervisor": "vmware", "unitOfMeasure": "yearly", "skuKeyword1": "F5-BIG-MSP-LTM-3G-LIC-DEV" } Next, you need to POST your JSON body to BIG-IQ. You’ll need to enable basic authentication on BIG-IQ in order to use username and password authentication, the default is to use token- based authentication. To enable basic authentication, log into BIG-IQ console as “root” and enter “set-basic-auth on”. For the example below, we’ll use CURL – the JSON body is stored in a file named “getlicense.txt”. # curl -k -l -H'Accept:application/json' -H'Content-Type:application/json' -X POST -T getlicense.txt "https://admin:admin@10.255.65.16/mgmt/cm/device/tasks/licensing/pool/member-management" In this case, the BIG-IQ is at 10.255.65.16 and we’re using the admin login account on BIG-IQ with password “admin”. Your POST must go to “/mgmt/cm/device/tasks/licensing/pool/member- management” This results in the following return, if BIG-IQ has accepted your request: { "address": "192.0.2.3", "assignmentType": "UNREACHABLE", "command": "assign", "generation": 1, "hypervisor": "vmware", "id": "64100009-e20c-4add-9d3f-6cf4ed6fde31", "identityReferences": [ { "link": "https://localhost/mgmt/shared/authz/users/admin" } ], "kind": "cm:device:tasks:licensing:pool:member-management:devicelicensingassignmenttaskstate", "lastUpdateMicros": 1531492957761866, "licensePoolName": "Engineering-Pool", "macAddress": "FA:16:3E:1B:6D:34", "ownerMachineId": "9890115d-3e0b-4e7f-a9a0-ebb22c5747e4", "selfLink": "https://localhost/mgmt/cm/device/tasks/licensing/pool/member-management/64100009-e20c-4add-9d3f-6cf4ed6fde31", "skuKeyword1": "F5-BIG-MSP-LTM-3G-LIC-DEV", "status": "STARTED", "taskWorkerGeneration": 1, "unitOfMeasure": "yearly", "userReference": { "link": "https://localhost/mgmt/shared/authz/users/admin" } } IG-IQ is an asynchronous, task-based design, as such, the license is not returned when the POST completes, but rather, the acceptance of your request to BIG-IQ. To get the license, you must poll the “selfLink” (highlighted above). # curl -k -l -H'Accept:application/json' -H'Content-Type:application/json' -X GET "https://admin:admin@10.255.65.16/mgmt/cm/device/tasks/licensing/pool/member-management/64100009-e20c- 4add-9d3f-6cf4ed6fde31" If successful, this GET will return the license for the BIG-IP VE instance described in the JSON request in the “licenseText” field of the JSON body. For brevity, it is omitted. If there was an error encountered, the error text is returned in the “errorMessage” field of the JSON return body. Step 2: Place the License on BIG-IP VE To complete the licensing operation, you’ll need to extract the license from the “licenseText” element in the returned JSON. You’ll need to login to the BIG-IP VE instance as “root” and do the following: Place the licenseText on the BIG-IP VE instance at the following location and file name: /config/bigip.license Restart BIG-IP VE services: bigstart restart At this point, your BIG-IP VE is licensed. Step 3: Revoking and Returning the License to BIG-IQ When your BIG-IP VE instance is no longer needed, it is time to return the license to the pool. This is called a revoke operation. You’ll need to gather the following information: The name of the license pool on BIG-IQ The IP address of the BIG-IP VE The MAC address of the BIG-IP VE Please see the information in the “Step 1” section on how to obtain and gather this information. The JSON body is then formed, here is an example: { "licensePoolName": "load7afterfix", "command": "revoke", "address": "192.0.2.3", "assignmentType": "UNREACHABLE", "macAddress": "FA:16:3E:1B:6D:34" } The “command” must be “revoke” to return the license to the pool. Here is the example CURL command to the revoke, the JSON body is stored in a file called “revokelicense.txt”: # curl -k -l -H'Accept:application/json' -H'Content-Type:application/json' -X POST -T revokelicense.txt "https://admin:admin@10.255.65.16/mgmt/cm/device/tasks/licensing/pool/member- management" If successful, this request will return a JSON body similar to the following: { "address": "192.0.2.3", "assignmentType": "UNREACHABLE", "command": "revoke", "generation": 1, "id": "c7348b6a-6973-4372-9b66-f07c40bd0fd5", "identityReferences": [ { "link": "https://localhost/mgmt/shared/authz/users/admin" } ], "kind": "cm:device:tasks:licensing:pool:member-management:devicelicensingassignmenttaskstate", "lastUpdateMicros": 1531490803422235, "licensePoolName": "load7afterfix", "macAddress": "FA:16:3E:1B:6D:34", "ownerMachineId": "9890115d-3e0b-4e7f-a9a0-ebb22c5747e4", "selfLink":"https://locahost/mgmt/cm/device/tasks/licensing/pool/member-management/c7348b6a-6973-4321-8f66-f07e41bf0ed4", "status": "STARTED", "taskWorkerGeneration": 1, "userReference": { "link": "https://localhost/mgmt/shared/authz/users/admin" } } Similar to the licensing operation, you can poll the selfLink using a GET operation to monitor BIG-IQ’s progress in finishing up the revoke.iControl - Run and fetch APM Reports
Hi, Is it possible to run and download APM reports via iControl? Either JSON or CSV would be fine. I've had a play around with the /mgmt/tm/apm/report/ endpoint, however this seems to only give some basic properties of the report as opposed to actually generating and fetching a report. I've also looked around DevCentral but haven't found any discussions on this. If not iControl, is another method available? I did see this thread (https://devcentral.f5.com/questions/apm-reporting-by-tmsh) regarding TMSH. An RFE was mentioned, however it was apparently still open as of 2015. (ID 413709 - Need command-line based reporting capabilities with CSV results (or scheduling of reports). I've already played around with TMSH and couldn't find anything useful. Thanks, Andrew
APM logs (BIGIP 13.0.0)
Hi all One of my APM users was denied by the access policy. Trying to troubleshoot the problem, i tried to search the session id in the reports. But only last day is shown (even when i try to show "all sessions"). How could i extend the reported days? And (this i suppose has nothing to do with the reports, but not sure), i dont kwnow how to change the /var/log/apm files rotation (only 4 days stored). Thanks for your help!
Scheduled Report - Top Attacks By Security Policy
Hello, I am looking to generate a scheduled report to display the top attacks by each security policy I have (4) and I am using v12.1.2. I navigate to "Security ›› Reporting : Scheduled Reports" and enter the required fields such as name, SMTP info, frequency, etc. But the question is obviously in the exact configuration of the report. If I use... Dynamic Report Time Period: Last Week Show Results: Top 20 Top Report Criteria: Security Policy Select Measure: Requests .. then I get a report with a chart that shows the amount of requests each of those policies has received. Cool. But that is not exactly what I want. So I hit the plus button to "Use top result from security policy to report" > "Attack Type" Is there any way that I can "Use ALL results from security policy to report > Attack Type" ?? Any other info on how I could get something like this working to see attack types / violation for the last week from all my policies is appreciated. It is hard to share this info with others when I can only see traffic from my largest virtual server in these reports.
APM logs (BIGIP 13.0.0)
Hi all One of my APM users was denied by the access policy. Trying to troubleshoot the problem, i tried to search the session id in the reports. But only last day is shown (even when i try to show "all sessions"). How could i extend the reported days? And (this i suppose has nothing to do with the reports, but not sure), i dont kwnow how to change the /var/log/apm files rotation (only 4 days stored). Thanks for your help!
