Telemetry Streaming: getting HTTP statistics via SNMP
Hi F5 community, I am looking to get HTTP statistics (total count, and broken by response code) metrics from Telemetry Streaming via SNMP (seems to be the most viable option). F5-BIGIP-LOCAL-MIB::ltmHttpProfileStat oid: .1.3.6.1.4.1.3375.2.2.6.7.6 However, the stats don't seem to come out correct at all: I do see deltas happening, but they don't match at all the traffic rate I expect to see. Furthermore, I have done some tests where I would start a load testing tool (vegeta) to fire concurrent HTTP requests, for which I do see the logs from the virtual server, but no matching increment in the above SNMP OID entries on none of the profiles configured. What am I doing wrong? does something need to be enabled on the HTTP profile in use to collect those stats? Best, Owayss
Troubleshooting F5 WAF Log Shipping to Microsoft Sentinel SIEM: Issues Isolating ASM Logs
We have an issue with shipping logs from F5 WAF to Microsoft Sentinel SIEM. The issue is peculiar. We do not want to ship either F5Telemetry_system_CL logs or F5Telemetry_LTM_CL logs, only F5Telemetry_ASM_CL logs. We have simplified the command to the most basic one, at first it was working and we managed to ship LTM and ASM logs but when we tried to granulate for just LTM Logs, nothing is being sent. I will include the commands we used at the bottom. Any help would be appreciated, as well as some guidance differentiating between ASM and LTM logs. Command working for LTM and ASM not system logs curl -ku <username>:<password> -H 'Content-Type: application/json' https://192.0.0.0/mgmt/shared/telemetry/declare --data-raw \ '{ "class": "Telemetry", "My_Listener": { "class": "Telemetry_Listener", "port": 6514 } "Pull_Consumer": { "class": "Telemetry_Pull_Consumer", "type": "default", "systemPoller": ["Poller"] }, "My_Consumer": { "class": "Telemetry_Consumer", "type": "Azure_Log_Analytics", "workspaceId": "secret", "passphrase": { "cipherText": "secret" }, } }' New command was successful, but nothing is being sent curl -ku <username>:<password> -H 'Content-Type: application/json' https://192.0.0.0/mgmt/shared/telemetry/declare --data-raw \ ' { "class": "Telemetry", "controls": { "class": "Controls", "logLevel": "info", "debug": false }, "My_Consumer": { "class": "Telemetry_Consumer", "type": "Azure_Log_Analytics", "workspaceId": secret", "passphrase": { "cipherText": "secret" } } }
Telemetry Streaming Question
Hi, I'm after a little assistance, I have installed the Telemetry Streaming extension to use Opentelemetry as an endpoint. I've been able to achieve this and my receiver is receiving the metrics. The declaration I am POST-ing is as follows { "class": "Telemetry", "My_System": { "class": "Telemetry_System", "systemPoller": { "interval": 60 } }, "My_Listener": { "class": "Telemetry_Listener", "port": 6514 }, "My_Consumer": { "class": "Telemetry_Consumer", "type": "OpenTelemetry_Exporter", "host": "10.1.2.3", "port": 4317, "metricsPath": "/v1/metrics", "convertBooleansToMetrics": true, "enable": true, "trace": false, "allowSelfSignedCert": true, "exporter": "protobuf", "protocol": "https" } } This is succesful as I say, but the received metrics don't include a hostname (which is important as there are some profile duplications on the 40+ devices looked afer), is there a way to insert the hostname into the messages exported from the systems? Thanks in advance.
Telemetry Streaming - Re-starting restnoded
Hello Everyone, I have a dilemma ever since I set up telemetry streaming. I noticed that the restnoded daemon is restarting (some days are more frequent than others) but I can't get my hand into the root cause of it and how to solve it. I have been keeping a close eye on "/var/log/restnoded/restnoded.log" but couldnt pin point what could cause the restnoded daemon to restart. Regards, Sarah.
Most efficient methods for Connection logging?
Does anyone have real world experience with logging connections at a high rate? If so, which methods are you using to collect and transmit the data? We have a requirement to log all connections going through our F5 devices. Things like the client/server-side IPs/ports as well as HTTP details for HTTP VIPs and DNS details from our GTMs. It's the Whitehouse M-21-31 mandate if anyone if familiar with it. I've used Request Logging Profiles and various iRules with HSL to collect this type of data before, but I've never been too concerned about overhead because I would only apply them as needed, like when t-shooting an issue with a VIP. Our busiest appliance pushes around 150k conn/sec and 5k HTTP req/sec, so I now have consider the most efficient methods to avoid any kind of impact to traffic flows. I've done some lab testing with several different methods but I can't do any meaningful load tests in that environment. Below are some of my opinions based on my lab testing so far. Data Collection AVR - I like that this single feature can meet all the requirements for collecting TCP, HTTP, and DNS data. It would also be relatively easy to perform audits to ensure the VIPs have the necessary Analytics profiles as we can manage it from the AVR profiles themselves. My main concern is the overhead that results from the traffic analysis. I assume it has to maintain a large database where it stores all the analyzed data even if we just ship it off to Splunk. Even the data shipped off to Splunk includes several different logs for each connection (each with a different 'Entity'). Request Logging Profile - This is fairly flexible and should have low overhead since the F5 doesn't need to analyze any of the data like AVR does. This only collects HTTP data so we still need another solution to collect details for non HTTP VIPs. It would be a pain to audit since we don't have use any kind of deployment templates or automation. iRule - This provides a lot of flexibility and it is capable of collecting all the necessary data, but I don't know how well performance overhead compares to AVR. This would also be a pain to audit due to lack of deployment templates and automation. Data Transmission HSL UDP Syslog - I imagine this is the most efficient method to send events, but it's likely only a matter of time before we are required to use TCP/TLS. Telemetry Streaming - This is the more modern method and it offers some interesting features like System Poller, which could eventually allow us to move away from SNMP polling. We would need a workaround for our GTM-only devices because they cannot run a TS listener.Does Telemetry Streaming generic HTTP push consumer forwards events to cloud SIEM platform?
We have set up Generic_HTTP consumer in Telemetry Streaming to forward events to cloud platform but the events are not forwarded. We observing this logs in /var/log/restnoded/restnoded.log Does Generic_HTTP consumer supports forwarding Telemetry streaming data to the cloud SIEM platforms? Is there any other configuration we need to follow? Also, does it supports only http protocol or both ?
