cancel
Showing results for 
Search instead for 
Did you mean: 
Login & Join the DevCentral Connects Group to watch the Recorded LiveStream (May 12) on Basic iControl Security - show notes included.
Kai_Chung
F5 Employee
F5 Employee

Signal from BIG-IP

BIG-IP has an extensible API which means that you can install packages onto it which expand its capabilities. Application Services 3 Extension (AS3) and Telemetry Streaming (TS) are declarative extensions which this document will utilize to declaratively configure BIG-IP with an Application Service and then send related data to Beacon.

Install Telemetry Streaming

Telemetry Streaming (TS) is an iControl LX Extension delivered as a TMOS-independent RPM file. Installing the TS Extension on BIG-IP enables you to declaratively aggregate, normalize, and forward statistics and events from the BIG-IP to a consumer application by POSTing a single TS JSON declaration to TS’s declarative REST API endpoint. Telemetry Streaming uses a declarative model, meaning you provide a JSON declaration rather than a set of imperative commands.

Telemetry Streaming is the following:

  • A javascript iControl LX plug-in
  • A project based on the intent of the appsvcs_integration iApp
  • A declarative interface for configuring telemetry on BIG-IP
  • Atomic (TS declarations)

The following are the steps to install Telemetry Streaming in BIG-IP.

Step 1:    Download the latest RPM package from the Release Assets on GitHub by going to this link: https://github.com/F5Networks/f5-telemetry-streaming/releases

Step 2:    Sign in to BIG-IP, navigate to iApps > Package Management LX and click on Import.

0151T000003pyBkQAI.png

Step 3:    In iApps window, click Choose File to select downloaded RPM file and then click Upload.

0151T000003pyBpQAI.png


This completes the installation of Telemetry Stream in BIG-IP.

Install Application Services 3 Extension

In addition to Telemetry Streaming, log streams from LTM, AFM, ASM, and APM could be configured with AS3.

Application Services 3 Extension (AS3) is a flexible, low-overhead mechanism for managing application-specific configurations on a BIG-IP system. AS3 uses a declarative model, meaning you provide a JSON declaration rather than a set of imperative commands. The declaration represents the configuration which AS3 is responsible for creating on a BIG-IP system. AS3 is well-defined according to the rules of JSON Schema, and declarations validate according to JSON Schema. AS3 accepts declaration updates via REST (push), reference (pull), or CLI (flat file editing).

The following are prerequisites for using AS3:

  • You must be using BIG-IP version 12.1.x (if using AS3 v3.1.0 or later) or v13.0 or later to use AS3.
  • To use AS3, your BIG-IP user account must have the Administrator role.
  • You should be familiar with the F5 BIG-IP and F5 terminology. For general information and documentation on the BIG-IP system, see the F5 Knowledge Center.
  • You must manually install AS3 before the AS3 RESTful API is available.

AS3 is also javascript iControl LX plug-in like

The following are the steps to install Application Services 3 Extension in BIG-IP.

Step 4:    Download the latest RPM package from the f5-appsvs-extension Release page on GitHub by using this link: https://github.com/F5Networks/f5-appsvcs-extension/releases

Step 5:    Sign in to BIG-IP, navigate to iApps > Package Management LX and click on Import.

0151T000003pyBuQAI.png

Step 6:    In iApps window, click Choose File to select downloaded RPM file and then click Upload.

0151T000003pyBqQAI.png

This completes the installation of Application Services 3 Extension in BIG-IP.

Postman Setup and API calls for BIG-IP Ingestion

In this section, Postman is used to make API calls to different services using a prebuilt Postman Collection (set of API calls) and Postman Environment (set of variables) to ingest BIG-IP into Beacon.

Step7:   Download and install Postman from this link: https://www.postman.com/downloads/

Step 8:    Open Postman, click Settings>General and click OFF for SSL certificate verification

0151T000003pyBlQAI.png

Step 9:    Click here to import the Collection and Environment into Postman application. Check Collections tab to verify Beacon_Guide has been installed.

0151T000003pyBzQAI.png

Step 10: Click the eye icon (Environment quick look) next to Beacon_Guide.

0151T000003pyBrQAI.png

Step 11: In Beacon_Guide Environment popup window, complete the following information under the CURRENT VALUE column by click on the pencil icon next to each field, and then click outside of popup.

  • USERNAME: < Beacon Login>
  • PASSWORD: < Beacon Password>
  • BIGIP_HOST: <Hostname / IP of BIG-IP>
  • BIGIP_PORT: 443
  • BIGIP_USER: <Admin User>
  • BIGIP_PASS: <Password>

0151T000003pyC4QAI.png

Step 12: Navigate to Collections>Beacon_Guide>F5 Beacon>Login and click Send.

0151T000003pyC9QAI.png

Step 13: Check the status to make sure it is 200 OK. If not, repeat Step 11.

0151T000003pyC0QAI.png

Step 14: Navigate to Collections>Beacon_Guide>F5 Beacon>Create Telemetry Token, click Send, and then copy “accessToken” string.

0151T000003pyC1QAI.png

Step 15: Navigate to Collections>Beacon_Guide>BIG-IP>AUTH and click Send.

Step 16: Check the status to make sure it is 200 OK. If not, repeat Step 11.

Step 17: Copy the following JSON script and paste it to a text editor, replace F5 Beacon Access Token from Step 14.

{
     "class": "Telemetry",
     "Poller": {
       "class": "Telemetry_System_Poller",
       "interval": 60,
       "enable": true,
       "trace": false,
       "allowSelfSignedCert": false,
       "host": "localhost",
       "port": 8100,
       "protocol": "http",
       "actions": [
         {
           "enable": true,
           "includeData": {},
           "locations": {
             "system": true,
             "virtualServers": true,
             "httpProfiles": true,
             "clientSslProfiles": true,
             "serverSslProfiles": true,
             "pools": true,
             "aWideIps": true,
             "aaaaWideIps": true,
             "cnameWideIps": true,
             "mxWideIps": true,
             "naptrWideIps": true,
             "aPools": true,
             "aaaaPools": true,
             "cnamePools": true,
             "mxPools": true,
             "naptrPools": true,
             "srvPools": true
           }
         },
         {
           "enable": true,
           "setTag": {
             "tenant": "`T`",
             "application": "`A`"
           }
         }
       ]
     },
     "Beacon_Consumer": {
       "class": "Telemetry_Consumer",
       "type": "Generic_HTTP",
       "host": "ingestion.ovr.prd.f5aas.com",
       "protocol": "https",
       "port": 50443,
       "path": "/beacon/v1/ingest-telemetry-streaming",
       "method": "POST",
       "enable": true,
       "trace": false,
       "headers": [
         {
           "name": "grpc-metadata-x-f5-ingestion-token",
           "value": "`>@/passphrase`"
           }
       ],
       "passphrase": {
         "cipherText": "F5 Beacon Access Token"
       }
     },
     "schemaVersion": "1.0.0"
   }

Step 18: Navigate to Collections>Beacon_Guide>BIG-IP>TS Declare>Body, paste edited JSON script from Step 17 into the window and click Send

0151T000003pyCAQAY.png

Step 19:    Sign in to Beacon and navigate to Beacon>Configuration, click on the Sources tab to verify the BIG-IP is added as one of the sources.

0151T000003pyDCQAY.png

This completes the signal from BIG-IP setup.

Signal from NGNIX

Gathering NGINX metrics to ingest into Beacon is by using Telegraf, an open source server agent to collect metrics from stacks, sensors and systems. Telegraf supports 200+ plugins to help collect the metrics. Plugins are modular ways to update Telegraf to tell it what data to collect (input plugin) and where to send it (output plugin). Telegraf’s agent can run directly on the server, VM, or container from which you wish to collect metrics or in a centralized location that polls the desired services.

For this guide, Telegraf is used to collect metrics from an Ubuntu VM running NGINX. Telegraf by default will also collect system metrics such as CPU and memory.

Configure NGINX

For our example we are using a fresh install of NGINX. Prior to starting make sure that you can reach the default index page for NGINX. This will allow you to generate metrics at the end of this section by hitting your webserver.

NGINX recommends that you break up your configuration into different files instead of having one large config file within /etc/nginx/nginx.conf. To follow these guidelines, we are going to leave the default config file alone and create a new file just for our NGINX status page called /etc/nginx/conf.d/status.conf. This allows us to have a single place to look for the changes we are making.

Step 1:    In Ubuntu running NGINX, create a new file /etc/nginx/conf.d/status.conf and add the contents below. This will tell NGINX that if it receives a request to /status on port 8485 that it should return status metrics.

server {
   listen 8485;
   location /status {
       stub_status;
       allow 127.0.0.1;
   }
}

Step 2:    Restart your NGINX service.

sudo systemctl restart nginx

Step 3:    Verify that our new status page is responding properly by using CURL.

curl http://localhost:8485/status

NGINX is ready for Telegraf to collection metrics and forward them to Beacon.

Install Telegraf

Step 4:    Use the following command to add the InfluxData repository

wget -qO- https://repos.influxdata.com/influxdb.key | sudo apt-key add -
source /etc/lsb-release
echo "deb https://repos.influxdata.com/${DISTRIB_ID,,} ${DISTRIB_CODENAME} stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

Step 5:    Use the following command to install and start Telegraf service

sudo apt-get update && sudo apt-get install telegraf
sudo systemctl start telegraf

Configure Telegraf

Step 6:    Create a new file /etc/telegraf/telegraf.d/nginx.conf for the NGINX input plugin and copy the following contents below into it.

[[inputs.nginx]]
 urls = ["http://localhost:8485/status"]
 response_timeout = "5s"

Step 7:    Create a new file /etc/telegraf/telegraf.d/beacon.conf for the Beacon output plugin, copy the following content into text editor, replace <Your_Token> with token from Step 14 of previous section, and then copy the edited contents into the new file.

[[outputs.http]]
 url = "https://ingestion.ovr.prd.f5aas.com:50443/beacon/v1/ingest-metrics"
 timeout = "120s"
 method = "POST"
 insecure_skip_verify = true
 data_format = "influx"
 content_encoding = "identity"
[outputs.http.headers]
 Content-Type = "text/plain; charset=utf-8"
 X-F5-Ingestion-Token = "<Your_Token>"

Step 8:    Use the following command to verify directory tree.

tree /etc/telegraf/

0151T000003pyBvQAI.png

Step 9:    Use the following command to restart Telegraf service.

sudo systemctl start telegraf

Step 10: Use the following command to verify Telegraf is running properly and ensuring it says active.

sudo systemctl status telegraf

0151T000003pyCBQAY.png

Step 11:    Sign in to Beacon and navigate to Beacon>Configuration, click on the Sources tab to verify the NGINX is added as one of the sources.

0151T000003pyDHQAY.png

This completes the signal from NGINX setup.

Relevant insight

The following steps create insights for BIG-IP and NGINX.

BIG-IP Insight

Step 1:    Sign in to Beacon and navigate to Beacon>Insights and click Create.

Step 2:    Select the following options and click Run.

  • Source Type: bigip-system
  • Metric: systemCpuUsage
  • Filter: Source <your BIG-IP hostname>
  • Functions: Mean

0151T000003pyBwQAI.png

Step 3:    View the system status in the graph section.

0151T000003pyC5QAI.png

Step 4:    In the Properties section, provide the following and click Save & Close.

  • Title: BigIPCPUUsage
  • Severity: Critical

0151T000003pyC6QAI.png

NGNIX Insight

Step 5:    Navigate to Beacon>Insights and click Create.

Step 6:    Select the following options and click Run.

  • Source Type: nginx
  • Metric: active
  • Filter: Host <your NGINX hostname>
  • Functions: Mean

Step 7:    View the system status in the graph section.

Step 8:    In the Properties section, provide the following and click Save & Close.

  • Title: AcitveNGINX
  • Severity: Informational

This completes the insight creation for BIG-IP and NGINX


Summary:

This article provided instructions to configure signals from BIG-IP and NGINX to Beacon, and also provided instructions to creating relevant insights.


Next Steps:

This article is part of Visibility and Orchestration with F5 Series. Please check out other articles in this series.

Version history
Last update:
‎11-Dec-2020 10:22
Updated by:
Contributors