Telemetry Streaming

10 Topics

01 - Visualization of F5 BIG-IP metrics on Grafana using Prometheus and Telemetry Streaming service
This user guide isall about configuration and deployment of telemetry streaming service on F5 BIG-IP device and scraps those metrics by Prometheus which will be finally visualized by the Grafana. One can select the relevant metrics scraped by the Prometheus and visualize them on the Grafana which will be demonstrated later in the guide. Note: More detailed steps along with configuration images can be found on : https://nishalrai.com.np/2022/08/18/visualization-of-f5-big-ip-metrics-on-grafana-using-prometheus-and-telemetry-streaming-service/ This guide is heavily based on the work performed by Michael O'Leary and one can view onhere. The purpose of this guide is to document a little more elaborated guide for both learning and deployment aspects and also address the possible issues that could be faced during the process of deployment. Telemetry streaming (TS)is an iControl LX extension delivered as a TMOS-independent RPM file with the ability to declaratively aggregate, normalize and forward statistics and events from the BIG-IP to a consumer application by posting a single TS JSON declaration to TS’s declarative REST API endpoint. Additional information about Telemetry streaming can be foundhere. Prometheus is an open-source monitoring solution that stores time series data like metrics whereas Grafana allows visualizing the data stored in Prometheus and also supports a wide range of other sources. A short briefing about the architecture diagram in case of this user-deployment case scenario, the F5 BIG-IP system is on standalone mode with a management IP of 172.20.100.173, and both Prometheus and Grafana services are running on the same host with an IP address of 192.168.180.191 where the service port for Prometheus is on default – 9090 and the service port for Grafana is 5000. The whole deployment guide is broadly divided into the following sections and one can jump to the required step if they have achieved the previous configuration successfully: Section I: Download and install Telemetry Streaming Section II: Telemetry Streaming Declaration on the F5 BIG-IP device Section III: Configuration of Prometheus Section IV: Configuration on Grafana using Prometheus as a data source Section I: Download and install Telemetry Streaming We need to first download and install the telemetry streaming package on the F5 BIG-IP device. Since the telemetry streaming package is an RPM file that can be downloaded and can install through GUI or curl command on the CLI of the F5 BIG-IP device. In this user manual guide, we will download and then upload the telemetry streaming package on the BIG-IP using the iControl/iApp LX framework. One can use the alternative way which can be foundhere. First, we need to download the RPM file, one can find the latest telemetry streaming RPM file on the F5 Telemetry site on GitHub and download the latest RPM file. The GitHub page to download telemetry streaming can be foundhere. After downloading the file, you need to access your F5 BIG-IP GUI with your admin privilege account then follow the following steps: Go To iApps module > Package Management LX > Import > Browse to the downloaded location > Select Section II: Telemetry Streaming Declaration on the F5 BIG-IP device Once the download and installation of the F5 telemetry streaming package have been completed, we need to send a Telemetry Streaming declaration to configure a Telemetry Streaming pull consumer target. Before we jump into this configuration, we need to create a new user with an administrator role on the F5 BIG-IP device and you can just continue with the default admin user on the further configuration. We can create a new user in the following steps: Go to System > Users > User List Click on Create button Input the new user’s name and password Select role as administrator then add Click on the Finished button As we’re using Prometheus on this user-guide manual so, the Telemetry Streaming consumer target will be Prometheus which is hosted on 192.168.180.191:5000 We can either use Postman or using curl command on the CLI of the F5 BIG-IP device to configure a Telemetry Streaming pull consumer target. Configuration using Postman application Just follow the following steps for the configuration of the telemetry streaming consumer target using the Postman application. Step I: Open the Postman and create a new tab Step II: Select the GET method and paste the following link https://<big-ip-management-ip-address>/mgmt/shared/telemetry/declare Step III: Browse on Auth field and fill up the credentials Use the credentials used to log into F5 BIG-IP (in this case, recently created new user) Step IV: Select on Body option Change the method into POST, then select raw sub-option and then JSON data format. Past the Telemetry Streaming declaration on the body section and then click on the send button. { "class": "Telemetry", "My_Poller": { "class": "Telemetry_System_Poller", "interval": 0 }, "My_System": { "class": "Telemetry_System", "enable": "true", "systemPoller": [ "My_Poller" ] }, "metrics": { "class": "Telemetry_Pull_Consumer", "type": "Prometheus", "systemPoller": "My_Poller" } } Step V: Verify the response as the success status Select GET HTTP method on https://<big-ip-management-ip-address>/mgmt/shared/telemetry/declare Step VI: Verify the available metrics Create a new tab on Postman: -On the URL section https://<big-ip-management-ip-address>/mgmt/shared/telemetry/pullconsumer/metrics -On the authorization section, use the same credentials used before
Nishal_Rai
Aug 27, 2022 Place Technical Forum
7.2KViews
4likes
4Comments
Deploying BIG-IP Telemetry Streaming with Azure Sentinel as its consumer.
AZURE SENTINEL and BIG-IP ...with Telemetry Streaming! This work was completed as a collaboration of Remo Mattei r.mattei@f5.com and Bill Wester b.wester@f5.com, feel free to email us if you have questions. One of the things that I have discovered recently is how neat it is to be able to leverage Azures new Sentinel to receive and display telemetry data from F5's BIG-IP devices. The devices don't even have to be in Azure, you could have dedicated hardware BIG-IPs and still send via Telemetry Streaming to Sentinel as your destination for statistics and logs. Let us explore a bit more on how to get all of the moving pieces together to a single cohesive implementation. Telemetry Streaming is a way for you to forward events and statistics from the BIG-IP system to your preferred data consumer and visualization application. You can do all of this byPOSTinga single JSON declaration to a declarative REST API endpoint. Telemetry Streaming uses a declarative model, meaning you provide a JSON declaration rather than a set of imperative commands. More info can be found here: https://clouddocs.f5.com/products/extensions/f5-telemetry-streaming/latest/userguide/about-telemetry.html BIG-IP allows you to send logs to several external providers. Splunk, awell knownone, is one of the most used out there. However, the new Azure Sentinel, a cloud solution, is something that many customers can take advantages from.This section,will help in understanding on how to setup BIG-IP to get the logs to Azure Sentinel.  Setup BIG-IP First of all, this is broken into two parts, one shows the logs of the BIG-IP System Metrics, like what OS, what modules are installed etc. The second, is about themodule ASM. The two have a few things in common. They use the TS RPM file which is added to the BIG-IP, and the declaration, which tells the BIG-IP where to send the stream of data. To send data relate to BIG-IP System Metrics it is required to have AVR provisioned on the device. ASM is not required but we use it here as an example of how to enable another module. Here is a screenshot from the Azure which shows the required modules.One more important thing is that ASM will need to have AFM also enabled otherwise you will not get logs in Azure. ASM Once enabled the required modules it will show System Metrics Common components that you must install for this to work First you need Telemetry Streaming: The TS RPM can be found here on GITHUB: https://github.com/F5Networks/f5-telemetry-streaming/releases/ You can use Visual Studio Code to install the RPM or your favorite way... Here are some screen shots form VS Code, using the F5 Plugin. NOTE: in order to useVSCodeto push AS3, DOetcyou must install the F5 Plugin. Use the command options in Mac it’scommand+shift+P(here you can search for RPM by just typing it in the box) Select AS3and make sure to install both AS3 and TS: Select the version: (probably latest is best here) The Telemetry Streaming declaration looks like this: { "class": "Telemetry", "My_Listener": { "class": "Telemetry_Listener", "port": 6514 }, "Poller": { "class": "Telemetry_System_Poller", "interval": 60, "enable": true, "trace": false, "allowSelfSignedCert": false, "host": "localhost", "port": 8100, "protocol": "http", "actions": [ { "enable": true, "includeData": {}, "locations": { "system": true, "virtualServers": true, "httpProfiles": true, "clientSslProfiles": true, "serverSslProfiles": true } } ] }, "Pull_Consumer": { "class": "Telemetry_Pull_Consumer", "type": "default", "systemPoller": [ "Poller" ] }, "Azure_Consumer": { "class": "Telemetry_Consumer", "type": "Azure_Log_Analytics", "workspaceId": "workspaceID", "passphrase": { "cipherText": "primkey" } }, "schemaVersion": "1.12.0" } NOTE: You will need to get theworkspaceIDand theprimarykey. You can use the azure cli for that: azmonitor log-analytics workspace list --out table CustomerId Location Name ProvisioningState PublicNetworkAccessForIngestion PublicNetworkAccessForQuery ResourceGroup RetentionInDays ------------------------------------ ------------- ---------------------------------------------------------- ------------------- --------------------------------- ----------------------------- ------------------------- ----------------- a05d4bfb-27c8-49a6-96e2-351d2dc78c61 eastus adrianLA Succeeded Enabled Enabled adrian_rg_01 7 63be43ed-b3f5-4e9f-bc92-226bb3393d11 eastus DefaultWorkspace-77c6ebef-d849-4527-a355-742d8d7d3fdc-EUS Succeeded Enabled Enabled defaultresourcegroup-eus 30 2ccbd35a-dfdf-4a5e-ab5f-1d5314f52e4b southeastasia DefaultWorkspace-77c6ebef-d849-4527-a355-742d8d7d3fdc-SEA Succeeded Enabled Enabled defaultresourcegroup-sea 30 9436f742-069a-4e29-aac0-e1258f7b1f87 westus2 calalangakslog Succeeded Enabled Enabled calalang-rg 30 ac071b51-f0c6-43b6-8bef-16b9197fde0f westus2 edgar-log Succeeded Enabled Enabled defaultresourcegroup-eus 31 555ae8d5-75bc-4058-becf-df510c09f8d3 westus2 DefaultWorkspace-77c6ebef-d849-4527-a355-742d8d7d3fdc-WUS2 Succeeded Enabled Enabled defaultresourcegroup-wus2 30 f633bdb1-d560-43cd-a664-cc7a93ed8781 westus2 edgar-log-analytics Succeeded Enabled Enabled edgar-rg 30 9334eb7c-16fc-4db9-a84f-5824a7177ccb centralus DefaultWorkspace-77c6ebef-d849-4527-a355-742d8d7d3fdc-CUS Succeeded Enabled Enabled defaultresourcegroup-cus 30 091c2cf3-853d-4297-9001-41d2109c28ec westus DefaultWorkspace-77c6ebef-d849-4527-a355-742d8d7d3fdc-WUS Succeeded Enabled Enabled defaultresourcegroup-wus 30 52471748-d9c7-46ba-9f9f-72ed8e92a201 westus remo-analytics Succeeded Enabled Enabled remo-telemetry 30 bc8e90ca-f59c-4fbf-a28b-213fe1cfcfda westus wester-log Succeeded Enabled Enabled wester_rg 30 Here you can see the name of the resource group then run the following command: azmonitor log-analytics workspace get-shared-keys --resource-groupwester_rg--workspace-name wester-log Which will print out theprimarykey The workspace isCustomerIdfrom the main table.  To install this declaration you can use POSTMAN, curl, or Visual Studio Code; we used Visual Studio Code.  Copy the text into a newVScodetab, make sure it’s in json format and then use the command pallet to post it Verify by using the TS version at the bottom ofVSCode, it will execute a GET to the BIG-IP that is connected.  ASM In order to use ASM you will need to configure a VIP with the IP of 255.255.255.254, and the port to the 6514, as well as aniRule. This can be done with an AS3 declaration or TMSH.  Sample of AS3 declaration { "class":"ADC", "schemaVersion":"3.10.0", "remark":"Example depicting creation of BIG-IP module log profiles", "Common": { "Shared": { "class":"Application", "template":"shared", "telemetry_local_rule": { "remark":"Only required when TS is a local listener", "class":"iRule", "iRule":"when CLIENT_ACCEPTED {\nnode127.0.0.1 6514\n}" }, "telemetry_local": { "remark":"Only required when TS is a local listener", "class":"Service_TCP", "virtualAddresses": [ "255.255.255.254" ], "virtualPort":6514, "iRules": [ "telemetry_local_rule" ] }, "telemetry": { "class":"Pool", "members": [ { "enable":true, "serverAddresses": [ "255.255.255.254" ], "servicePort":6514 } ], "monitors": [ { "bigip":"/Common/tcp" } ] }, "telemetry_hsl": { "class":"Log_Destination", "type":"remote-high-speed-log", "protocol":"tcp", "pool": { "use":"telemetry" } }, "telemetry_formatted": { "class":"Log_Destination", "type":"splunk", "forwardTo": { "use":"telemetry_hsl" } }, "telemetry_publisher": { "class":"Log_Publisher", "destinations": [ { "use":"telemetry_formatted" } ] }, "telemetry_traffic_log_profile": { "class":"Traffic_Log_Profile", "requestSettings": { "requestEnabled":true, "requestProtocol":"mds-tcp", "requestPool": { "use":"telemetry" }, "requestTemplate":"event_source=\"request_logging\",hostname=\"$BIGIP_HOSTNAME\",client_ip=\"$CLIENT_IP\",server_ip=\"$SERVER_IP\",http_method=\"$HTTP_METHOD\",http_uri=\"$HTTP_URI\",virtual_name=\"$VIRTUAL_NAME\",event_timestamp=\"$DATE_HTTP\"" } }, "telemetry_security_log_profile": { "class":"Security_Log_Profile", "application": { "localStorage":false, "remoteStorage":"splunk", "protocol":"tcp", "servers": [ { "address":"255.255.255.254", "port":"6514" } ], "storageFilter": { "requestType":"illegal-including-staged-signatures" } }, "network": { "publisher": { "use":"telemetry_publisher" }, "logRuleMatchAccepts":false, "logRuleMatchRejects":true, "logRuleMatchDrops":true, "logIpErrors":true, "logTcpErrors":true, "logTcpEvents":true } } } } } To post an AS3 declaration like above use Visual Studio Code Use the command menu and select F5 Post an AS3 Declaration from the tab you have pasted the code  OUTPUT from the declaration above: iRuleused Assign the Telemetry Policy to the Virtual Service by selecting the option in the advanced menu Once you have the modules installed, and configured the appropriate settings, like above, then you will see data coming in Azure Sentinel. Here is an example: ASM System Metrics For System Metrics to work, you will need to have AVR installed, you do not need an AS3 declaration or aniRule. Once you have AVR installed, and have pushed the declaration to the BIG-IP, you will need to execute the following command in your BIG-IP.  tmshmodify analytics global-settings{offbox-protocoltcpoffbox-tcp-addresses add { 127.0.0.1 }offbox-tcp-port 6514 use-offboxenabled } tmshsave /sys config Check the logs in your BIG-IP less /var/log/restnoded/restnoded.log You will see something like: Fri, 18 Sep 2020 06:36:04 GMT - info: [telemetry] Starting systempollerPoller::Poller. Interval = 60 sec. Fri, 18 Sep 2020 06:36:04 GMT - info: [telemetry] 1 consumer plug-in(s) loaded Next you will need to go into the Azure Portal, and you can find a nice pre-defined Sentinel Workbook to view and start to work with: You will select the "template" and then fill out the correct workspace from the dropdown, then select the correct hostname from the dropdown and you will start to see data showing up. Azure Sentinel displaying the workbook As you enable more modules, they will show up in the Azure Sentinel and will show how it’s enabled.You can also add / modify / enhance the workbook to show more data that is in Sentinel sent from the BIG-IP. Remo and I hope you found this article helpful and enjoy using BIG-IPs with Sentinel!
William_Wester
Oct 26, 2020 Place Technical Articles
4.2KViews
3likes
9Comments
How I did it - "Visualizing Data with F5 TS and the Elastic ELK Stack"
With the F5 BIG-IP and Telemetry Streaming I have the ability to send BIG-IP metrics to a variety of third-party analytics vendors. One of the more popular of these is Elastic. Elastic's ELK Stack, (acronym for Elasticsearch, Logstash, Kibana) provides a platform where I can store, search, analyze and visualize my BIG-IP telemetry data. With said, here's an overview of "How I did it"; integrating and visualizing data with the ELK Stack. P.S. Make sure to stay for the movie. Application Services 3 Extension (AS3) There are several resources, (logging profiles, log publishers, iRules, etc.) that must be configured on the BIG-IP to enable remote logging. I utilized AS3 to deploy and manage these resources. I used Postman to apply a single REST API declaration to the AS3 endpoint. Telemetry Streaming (TS) F5's Telemetry Streaming, (TS) service enables the BIG-IP to stream telemetry data to a variety of third-party analytics providers. Aside from the aforementioned resources, configuring TS to stream to a consumer, (Logstash in this instance), is simply a REST call away. Just as I did for AS3, I utilized Postman to post a single declaration to the BIG-IP. Elastic (ELK) Stack Configuring the ELK stack to receive and ingest BIG-IP telemetry is a fairly simple process. Logstash, (the "L" in ELK) is the data processor I used to ingest data into the stack. To accomplish this, I applied the sample Logstash configuration file. The configuration file specifies, (among other items) the listener port, message format, and the Elasticsearch index naming format. Dashboards Getting telemetry data into Elasticsearch is great but only if you can make use of it. If I'm going to utilize the data, I need to visualize the data; (should probably trademark that). For visualization, i created a couple sample dashboards. The dashboards, (community-supported and perhaps not suitable for framing) report various relevant BIG-IP performance metrics and WAF incident information. F5 BIG-IP Advanced WAF Insights F5 BIG-IP Performance Metrics Check it Out Rather than walk you through the entire configuration, how about a movie? Click on the link (image) below for a brief walkthrough demo integrating F5's BIG-IP with Elastic's ELK stack using F5 Telemetry Streaming. Try it Out Liked what you saw? If that's the case, (as I hope it was) try it out for yourself. Checkout F5's CloudDocs for guidance on configuring your BIG-IP(s) with the F5 Automation Toolchain.The various configuration files, (including the above sample dashboards) used in the demo are available on the GitHub solution repository Enjoy!
Greg_Coward
Oct 15, 2021 Place Technical Articles
3.5KViews
0likes
0Comments
02 - Visualization of F5 BIG-IP metrics on Grafana using Prometheus and Telemetry Streaming service
Configuration using CLI of F5 BIG-IP device Following steps for the configuration of telemetry streaming consumer target using CLI of F5 BIG-IP device are discussed below: Once you have accessed your F5 BIG-IP device CLI terminal then access either your default admin credentials or the new user you’ve recently created on the above section. Then execute the following commands on the terminal: On the username and password section, you either enter your default admin credentials or the new user you’ve recently created has the administrator privilege. curl -u username:password -k https://localhost/mgmt/shared/telemetry/declare Note: -k, --insecure to be made secure by using the CA certificate bundle installed by default. This makes all connections considered "insecure" fail unless -k, --insecure is used. ChangChange into tmp directory and create a file called ts-config.json and I am using vi editor for it. cd /tmp vi ts-config.json Paste the Telemetry Streaming declaration and then save the file and exit the vi editor. { "class": "Telemetry", "My_Poller": { "class": "Telemetry_System_Poller", "interval": 0 }, "My_System": { "class": "Telemetry_System", "enable": "true", "systemPoller": [ "My_Poller" ] }, "metrics": { "class": "Telemetry_Pull_Consumer", "type": "Prometheus", "systemPoller": "My_Poller" } } Then execute the following command on the terminal on thesame directory /tmp and change the username and password section with your F5 BIG-IP device credentialshaving the administrator privilege. curl -X POST -u username:password -khttps://localhost/mgmt/shared/telemetry/declare-d @ts-config.json -H “content-type:application/json” To verify the available metrics curl -u username:password -k https://localhost/mgmt/shared/telemetry/pullconsumer/metrics Section III: Configuration of Prometheus Once the telemetry streaming service has been successfully configured and the metrics are available on the path. We need to configure Prometheus in order to scrape the metrics data on the predefined path. The following are the steps to configure the Prometheus: Note: On this user-guide demonstration, both Grafana and Prometheus are installed on the same host with different service ports as mentioned earlier. CentOS 7 is used as the OS for this host machine and you may have different syntax to view the following status check. First, check the status of the Prometheus sudo systemctl status prometheus.service View the current working directory and change into /etc/prometheus pwd cd /etc/prometheus ls -al global: scrape_interval: 10s scrape_configs: - job_name: 'TelemetryStreaming' scrape_timeout: 30s scrape_interval: 30s scheme: https tls_config: insecure_skip_verify: true metrics_path: '/mgmt/shared/telemetry/pullconsumer/metrics' basic_auth: username: 'F5-BIG-IP-username' password: 'F5-BIG-IP-password' static_configs: - targets: ['BIGIP-managementIP:443'] Then restart the Prometheus service and check the status of the Prometheus service. sudo systemctl restart prometheus.service sudo systemctl status prometheus.service Note: If the configuration is correct, then the Prometheus service will be enabled otherwise, the status of the Prometheus service will be disabled. To further verify whether instances has been discovered on the Prometheus: -Go tohttp://prometheus-ip:service/port - Click on the Status option and select the Target option Section IV: Configuration on Grafana using Prometheus as a data source In this section, we need to connect Prometheus as a data source on Grafana Once the data source has been successfully configured on Grafna then Create a new dashboard and select Prometheus as the data source then select the relevant metrics and change the refresh interval as required. Save and apply the panel. Then,Save the dashboard and view the metrics on the Grafana dashboard. The possible issue that can arise during the configuration If you use the default TS declare from the official telemetry streaming document website then you may fail to view the available metrics on the mentioned link: https://<f5-management-ip>/mgmt/shared/telemetry/pullconsumer/metrics
Nishal_Rai
Aug 27, 2022 Place Technical Forum
3.4KViews
2likes
3Comments
How I did it - "Visualizing Metrics with F5 Telemetry Streaming and Datadog"
In some recent installments of the “How I Did it” series, we’ve taken a look at how F5 Telemetry Streaming, (TS) integrates with third-party analytics providers like Splunk and Elastic.In this article we continue our analytics vendor journey with the latest supported vendor, Datadog. Datadog is an analytics and monitoring platform offered in a Software-as-a-Service (SaaS) model.The platform provides a centralized visibility and monitoring of applications and infrastructure assets.While Datadog typically relies upon its various agents to capture logs and transmit back to the platform, there is an option for sending telemetry over HTTP.This is where F5’s TS comes into play. For the remainder of this article, I'll provide a brief overview of the services required to integrate the BIG-IP with Datadog.Rather than including the step-by-step instructions, I've included a video walkthrough of the configuration process.After all, seeing is believing! Application Services 3 Extension (AS3) There are several resources, (logging profiles, log publishers, iRules, etc.) that must be configured on the BIG-IP to enable remote logging. I utilized AS3 to deploy and manage these resources. I used Postman to apply a single REST API declaration to the AS3 endpoint. Telemetry Streaming (TS) F5's Telemetry Streaming, (TS) service enables the BIG-IP to stream telemetry data to a variety of third-party analytics providers. Aside from the aforementioned resources, configuring TS to stream to a consumer, (Datadog in this instance), is simply a REST call away. Just as I did for AS3, I utilized Postman to post a single declaration to the BIG-IP. Datadog Preparing my Datadog environment to receive telemetry from BIG-IPs via telemetry streaming is extremely simple.I will need to generate an API key which will in turn be used by a telemetry streaming extension to authenticate and access the Datadog platform.Additionally, the TS consumer provides for additional options, (Datadog region, compression settings, etc.) to be configured on the TS side. Dashboard Once my Datadog environment starts to ingest telemetry data from my BIG-IP, I’ll visualize the datausing a custom dashboard.The dashboard, (community-supported and may not be suitable for framing) report various relevant BIG-IP performance metrics and information. F5 BIG-IP Performance Metrics Check it Out Rather than walk you through the entire configuration, how about a movie? Click on the link (image) below for a brief walkthrough demo integrating F5's BIG-IP with Datadog using F5 Telemetry Streaming. Try it Out Liked what you saw? If that's the case, (as I hope it was) try it out for yourself. Checkout F5's CloudDocs for guidance on configuring your BIG-IP(s) with the F5 Automation Toolchain.The various configuration files, (including the above sample dashboards) used in the demo are available on the GitHub solution repository Enjoy!
Greg_Coward
Dec 22, 2021 Place Technical Articles
1.7KViews
1like
0Comments
Telemetry Streaming - Re-starting restnoded
Hello Everyone, I have a dilemma ever since I set up telemetry streaming. I noticed that the restnoded daemon is restarting (some days are more frequent than others) but I can't get my hand into the root cause of it and how to solve it. I have been keeping a close eye on "/var/log/restnoded/restnoded.log" but couldnt pin point what could cause the restnoded daemon to restart. Regards, Sarah.
Sarah
Sep 24, 2023 Place Technical Forum
700Views
0likes
2Comments
Most efficient methods for Connection logging?
Does anyone have real world experience with logging connections at a high rate? If so, which methods are you using to collect and transmit the data? We have a requirement to log all connections going through our F5 devices. Things like the client/server-side IPs/ports as well as HTTP details for HTTP VIPs and DNS details from our GTMs. It's the Whitehouse M-21-31 mandate if anyone if familiar with it. I've used Request Logging Profiles and various iRules with HSL to collect this type of data before, but I've never been too concerned about overhead because I would only apply them as needed, like when t-shooting an issue with a VIP. Our busiest appliance pushes around 150k conn/sec and 5k HTTP req/sec, so I now have consider the most efficient methods to avoid any kind of impact to traffic flows. I've done some lab testing with several different methods but I can't do any meaningful load tests in that environment.Below are some of my opinions based on my lab testing so far. Data Collection AVR - I like that this single feature can meet all the requirements for collecting TCP, HTTP, and DNS data. It would also be relatively easy to perform audits to ensure the VIPs have the necessary Analytics profiles as we can manage it from the AVR profiles themselves. My main concern is the overhead that results from the traffic analysis. I assume it has to maintain a large database where it stores all the analyzed data even if we just ship it off to Splunk. Even the data shipped off to Splunk includes several different logs for each connection (each with a different 'Entity'). Request Logging Profile- This is fairly flexible and should have low overhead since the F5 doesn't need to analyze any of the data like AVR does. This only collects HTTP data so we still need another solution to collect details for non HTTP VIPs. It would be a pain to audit since we don't have use any kind of deployment templates or automation. iRule - This provides a lot of flexibility and it is capable of collecting all the necessary data, but I don't know how well performance overhead compares to AVR. This would also be a pain to audit due to lack of deployment templates and automation. Data Transmission HSL UDP Syslog- I imagine this is the most efficient method to send events, but it's likely only a matter of time before we are required to use TCP/TLS. Telemetry Streaming - This is the more modern method and it offers some interesting features like System Poller, which could eventually allow us to move away from SNMP polling. We would need a workaround for our GTM-only devices because they cannot run a TS listener.
Roland00
Sep 14, 2023 Place Technical Forum
697Views
0likes
1Comment
Telemetry Streaming Question
Hi, I'm after a little assistance, I have installed the Telemetry Streaming extension to use Opentelemetry as an endpoint. I've been able to achieve this and my receiver is receiving the metrics. The declaration I am POST-ing is as follows { "class": "Telemetry", "My_System": { "class": "Telemetry_System", "systemPoller": { "interval": 60 } }, "My_Listener": { "class": "Telemetry_Listener", "port": 6514 }, "My_Consumer": { "class": "Telemetry_Consumer", "type": "OpenTelemetry_Exporter", "host": "10.1.2.3", "port": 4317, "metricsPath": "/v1/metrics", "convertBooleansToMetrics": true, "enable": true, "trace": false, "allowSelfSignedCert": true, "exporter": "protobuf", "protocol": "https" } } This is succesful as I say, but the received metrics don't include a hostname (which is important as there are some profile duplications on the 40+ devices looked afer), is there a way to insert the hostname into the messages exported from the systems? Thanks in advance.
Andy
Oct 10, 2023 Place Technical Forum
599Views
0likes
4Comments
High-level Pathways to Security Visibility
Editor's Note: The F5 Beacon capabilities referenced in this article hosted on F5 Cloud Services are planning a migration to a new SaaS Platform - Check out the latesthere. (10 Minute Read) Introduction In previous articles we identified the elements needed to gain visibility into adaptive application security postures. This entails observing the security configuration (static) and monitoring telemetry (dynamic) coming from different control points (ref. Visibility and Orchestration).We also suggested that security visibility should be integrated in the software development and/or deployment lifecycle as part of a shift-left strategy (ref. Shift-left Security Visibility). Now, we’ll focus on identifying a high-level pathway to achieve application security visibility. First, we need to identify the constraints that frame the effort.We will then identify concrete examples of insertion with F5 technologies. The end-goal is to ensure that you keep close control over the application security by embracing a holistic approach to visibility integrated in the software development/deployment lifecycle. Constraints Inserting security visibility in your enterprise is part of the shift-left strategy (ref. url-to-shift-left-sec-vis.). (https://www.f5.com/company/blog/beyond-visibility-is-operability) In order to be practical, we need to make sure that the pathway adheres to the following guidelines: Friction – The solution should not introduce any friction into the pipeline - For example, the tools used by the DEVOPS and SECOPS teams (e.g. Gitlab, Jenkins) should be the same avoiding gated interdependencies where a change by one group is blocked/delayed by the other. Programmability – The security-centric solutions implemented during the journey need to be highly programmable – This will ensure that the tools adapt to the environment (e.g. services, micro-services), the supporting infrastructure (e.g. cloud, containers), and the application. Automation – Enabling automation is key. This can be achieved by ensuring the tools deployed can be automatically configured without intervention as part of a pipeline.One way to ensure this is to leverage declarative application programing interfaces (API) (link-to-f5-declarative-interface) Scalability – Applications can span across infrastructure that is infinitely scalable like public cloud, across availability zones and geographies.This requires that any solution that is deployed to secure/protect applications and workloads be able to scale.To scale horizontally, the solution can be implemented across multiple workloads in multiple instances.To scale vertically, the solution should be able to handle increasing amounts of traffic in single/few instances. Transparency – From a performance and functionality standpoint, the solutions inserted to gain security visibility cannot impact the application.For example, when a proxy is inserted, it cannot add latency between the client and the workload. It also cannot affect the functionality provided by the workload. Resiliency – Inserting a solution to support your applications security and visibility should be resilient.Any failure of the process providing visibility should be flagged and not affect the application’s/workload’s performance or availability. Visibility Insertion All F5 solutions can be inserted in the application delivery infrastructure to provide security visibility.This comes in the form security-aware proxies.The BIG-IP or NGINX Plus platforms are particularly well-suited for insertion in infrastructure requiring inline low-latency and powerful application security and visibility.Deploying F5 solutions can easily be done observing all the constraints mentioned above. Friction Thanks to the available form factors and programmatic templates provided, implementing BIG-IP or NGINX Plus in the infrastructure is easily achieved using appropriate templates. For example, when working with AWS, a BIG-IP can easily be deployed using a Cloud Formation Template (CFT) found here.From the enterprise git (Gitlab, Github, Bitbucket etc.) repository, BIG-IP can be deployed directly by cloning/forking the F5 repository and integrating with the pipeline (ref. Clouddocs Article). Programmability The BIG-IP Advanced Web Application Firewall (Advanced WAF) configuration is highly programable.The advantage is that the configuration can be stored and or modified easily outside of the BIG-IP. For example, an base policy aimed at protecting against OWASP Top 10 Risks can look like the following: { "policy": { "name": "Complete_OWASP_Top_Ten", "description": "A generic, OWASP Top 10 protection items v1.0", "template": { "name": "POLICY_TEMPLATE_RAPID_DEPLOYMENT" }, "fullPath": "/Common/Complete_OWASP_Top_Ten", "enforcementMode":"transparent", "signature-settings":{ "signatureStaging": false, "minimumAccuracyForAutoAddedSignatures": "high" }, "protocolIndependent": true, "caseInsensitive": true, "general": { "trustXff": true }, "data-guard": { "enabled": true }, "policy-builder-server-technologies": { "enableServerTechnologiesDetection": true }, "blocking-settings": { "violations": [ { "alarm": true, "block": true, "description": "ASM Cookie Hijacking", "learn": false, "name": "VIOL_ASM_COOKIE_HIJACKING" }, { "alarm": true, "block": true, "description": "Access from disallowed User/Session/IP/Device ID", "name": "VIOL_SESSION_AWARENESS" }, { "alarm": true, "block": true, "description": "Modified ASM cookie", "learn": true, "name": "VIOL_ASM_COOKIE_MODIFIED" }, { "alarm": true, "block": true, "description": "XML data does not comply with format settings", "learn": true, "name": "VIOL_XML_FORMAT" }, { "name": "VIOL_FILETYPE", "alarm": true, "block": true, "learn": true } ], "evasions": [ { "description": "Bad unescape", "enabled": true, "learn": true }, { "description": "Apache whitespace", "enabled": true, "learn": true }, { "description": "Bare byte decoding", "enabled": true, "learn": true }, { "description": "IIS Unicode codepoints", "enabled": true, "learn": true }, { "description": "IIS backslashes", "enabled": true, "learn": true }, { "description": "%u decoding", "enabled": true, "learn": true }, { "description": "Multiple decoding", "enabled": true, "learn": true, "maxDecodingPasses": 3 }, { "description": "Directory traversals", "enabled": true, "learn": true } ] }, "xml-profiles": [ { "name": "Default", "defenseAttributes": { "allowDTDs": false, "allowExternalReferences": false } } ], "session-tracking": { "sessionTrackingConfiguration": { "enableTrackingSessionHijackingByDeviceId": true } } } } In the example above, aspects of a security policy like evasion techniques, or cookie consumption settings can easily be programmed in the configuration and handled like any other application code for versioning, editing or storing.The standard JSON format can be managed in a Git repository for use in any environment.Documentation for JSON representations of WAF policies can be found here. This is also true for all F5 security platforms including NGINX App Protect or Essential App Protect (ref.NGINX Configuration Guide and EAP API Users Guide). Similarly, configuring BIG-IP to forward security information telemetry to appropriate facilities can be achieved with the use of the Telemetry Streaming framework.For example, in order to configure BIG-IP to send telemetry data to a centralized visibility tool (F5 Beacon, or ELK for example) it can be configured with a declaration like: "class": "Telemetry", "controls": { "class": "Controls", "logLevel": "debug" }, "TS_Poller": { "class": "Telemetry_System_Poller", "interval": 60 }, "TS_Listener": { "class": "Telemetry_Listener", "port": 6514 }, "TS_Consumer": { "class": "Telemetry_Consumer", "type": "Generic_HTTP", "host": "my.visibility-host.url", "protocol": "http", "port": 8888, "path": "/", "method": "POST", "headers": [ { "name": "content-type", "value": "application/json" } ] } } The above declaration identifies the host where it will send telemetry – in this case debug data Scalability, Transparency and Resiliency F5 provides highly scalable, resilient and transparent solutions that can be inserted in any infrastructure to secure and provide visibility into web applications.Discussing these aspects of BIG-IP, NGINX Plus, or NGINX App Protect is beyond the scope of this article.For more information on scalability and high-availability you can refer to Performance of NGINX and NGINX Plus, NGINX App Protect Application Security Testing or BIG-IP Datasheet. Conclusion This article is meant to offer a path to visibility using F5 technology by inserting BIG-IP and configure it to provide application security and generate telemetry to gain visibility into the application's security posture. The aim is for you build a blueprint to systematically watch over your adaptive valuable applications and workloads across your infrastructure.
Romain
Feb 02, 2021 Place Technical Articles
359Views
0likes
0Comments
Telemetry Streaming: getting HTTP statistics via SNMP
Hi F5 community, I am looking to get HTTP statistics (total count, and broken by response code) metrics from Telemetry Streaming via SNMP (seems to be the most viable option). F5-BIGIP-LOCAL-MIB::ltmHttpProfileStat oid: .1.3.6.1.4.1.3375.2.2.6.7.6 However, the stats don't seem to come out correct at all: I do see deltas happening, but they don't match at all the traffic rate I expect to see. Furthermore, I have done some tests where I would start a load testing tool (vegeta) to fire concurrent HTTP requests, for which I do see the logs from the virtual server, but no matching increment in the above SNMP OID entries on none of the profiles configured. What am I doing wrong? does something need to be enabled on the HTTP profile in use to collect those stats? Best, Owayss
okabtoul
Nov 06, 2024 Place Technical Forum
19Views
0likes
0Comments