F5 Distributed Cloud Telemetry (Metrics) - Prometheus
Scope This article walks through the process of collecting metrics from F5 Distributed Cloud’s (XC) Service Graph API and exposing them in a format that Prometheus can scrape. Prometheus then scrapes these metrics, which can be visualized in Grafana. Introduction Metrics are essential for gaining real-time insight into service performance and behaviour. F5 Distributed Cloud (XC) provides a Service Graph API that captures service-to-service communication data across your infrastructure. Prometheus, a leading open-source monitoring system, can scrape and store time-series metrics — and when paired with Grafana, offers powerful visualization capabilities. This article shows how to integrate a custom Python-based exporter that transforms Service Graph API data into Prometheus-compatible metrics. These metrics are then scraped by Prometheus and visualized in Grafana, all running in Docker for easy deployment. Prerequisites Access to F5 Distributed Cloud (XC) SaaS tenant VM with Python3 installed Running Prometheus instance (If not check "Configuring Prometheus" section below) Running Grafana instance (If not check "Configuring Grafana" section below) Note – In this demo, an AWS VM is used with Python installed and running exporter (port - 8888), Prometheus (host port - 9090) and Grafana (port - 3000) running as docker instance, all in same VM. Architecture Overview F5 XC API → Python Exporter → Prometheus → Grafana Building the Python Exporter To collect metrics from the F5 Distributed Cloud (XC) Service Graph API and expose them in a format Prometheus understands, we created a lightweight Python exporter using Flask. This exporter acts as a transformation layer — it fetches service graph data, parses it, and exposes it through a /metrics endpoint that Prometheus can scrape. Code Link -> exporter.py Key Functions of the Exporter Uses XC-Provided .p12 File for Authentication: To authenticate API requests to F5 Distributed Cloud (XC), the exporter uses a client certificate packaged in a .p12 file. This file must be manually downloaded from the F5 XC console (steps) and stored on the VM where the Python script runs. The script expects the full path to the .p12 file and its associated password to be specified in the configuration section. Fetches Service Graph Metrics: The script pulls service-level metrics such as request rates, error rates, throughput, and latency from the XC API. It supports both aggregated and individual load balancer views. Processes and Structures the Data: The exporter parses the raw API response to extract the latest metric values and converts them into Prometheus exposition format. Each metric is labelled (e.g., by vhost and direction) for flexibility in Grafana queries. Exposes a /metrics Endpoint: A Flask web server runs on port 8888, serving the /metrics endpoint. Prometheus periodically scrapes this endpoint to ingest the latest metrics. Handles Multiple Metric Types: Traffic metrics and health scores are handled and formatted individually. Each metric includes a descriptive name, type declaration, and optional labels for fine-grained monitoring and visualization. Running the Exporter python3 exporter.py > python.log 2>&1 & This command runs exporter.py using Python3 in background and redirects all standard output and error messages to python.log for easier debugging. Configuring Prometheus docker run -d --name=prometheus --network=host -v $(pwd)/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus:latest Prometheus is running as docker instance in host network (port 9090) mode with below configuration (prometheus.yml), scrapping /metrics endpoint exposed from python flask exporter on port 8888 every 60 seconds. Configuring Grafana docker run -d --name=grafana -p 3000:3000 grafana/grafana:latest Private IP of the Prometheus docker instance along with port (9090) is used as data source in Grafana configuration. Once Prometheus is configured under Grafana Data sources, follow below steps: Navigate to Explore menu Select “Prometheus” in data source picker Choose appropriate metric, in this case “f5xc_downstream_http_request_rate” Select desired time range and click “Run query” Observe metrics graph will be displayed Note : Some requests need to be generated for metrics to be visible in Grafana. A broader, high-level view of all metrics can be accessed by navigating to “Drilldown” and selecting “Metrics”, providing a comprehensive snapshot across services. Conclusion F5 Distributed Cloud’s (F5 XC) Service Graph API provides deep visibility into service-to-service communication, and when paired with Prometheus and Grafana, it enables powerful, real-time monitoring without vendor lock-in. This integration highlights F5 XC’s alignment with open-source ecosystems, allowing users to build flexible and scalable observability pipelines. The custom Python exporter bridges the gap between the XC API and Prometheus, offering a lightweight and adaptable solution for transforming and exposing metrics. With Grafana dashboards on top, teams can gain instant insight into service health and performance. This open approach empowers operations teams to respond faster, optimize more effectively, and evolve their observability practices with confidence and control.F5 Distributed Cloud Telemetry (Metrics) - ELK Stack
As we are looking into exporting metrics data to the ELK stack using Python script, let's first get a high-level overview of the same. Metrics are numerical values that provide actionable insights into the performance, health and behavior of systems or applications over time, allowing teams to monitor and improve the reliability, stability and performance of modern distributed systems. ELK Stack (Elasticsearch, Logstash, and Kibana) is a powerful open-source platform. It enables organizations to collect, process, store, and visualize telemetry data such as logs, metrics, and traces from remote systems in real-time.Unlocking Insights: Enhancing Observability in F5 NGINXaaS for Azure for Optimal Operations
Introduction To understand application performance, you need more than just regular health checks. You need to look at the system’s behavior, how users use it, and find possible slowdowns before they become big problems. By using F5 NGINXaaS for Azure, organizations can gain enhanced visibility into their backend applications through extensive metrics, API (access) logs, and operational logs within Azure environments. This proactive approach helps prevent minor issues from developing into major challenges while optimizing resource efficiency. This technical guide highlights advanced observability techniques and demonstrates how organizations can leverage F5 NGINXaaS to create robust, high-performing application delivery solutions that ensure seamless and responsive user experiences. Benefits of F5 NGINX as a Service F5 NGINXaaS for Azure provides robust integration with ecosystem tools designed to monitor and analyze application health and performance. It uses rich telemetry from granular metrics across various protocols, including HTTP, TLS, TCP, and UDP. For technical experts overseeing deployments in Azure, this service delivers valuable insights that facilitate more effective troubleshooting and optimize workflows for streamlined operations. Key advantages of F5 NGINXaaS include access to over 200 detailed health and performance metrics that are critical for ensuring application stability, scalability, and efficiency. Please refer to the documentation for detailed information to learn more about the available metrics. There are two ways to monitor metrics in F5 NGINXaaS for Azure, providing flexibility in how you can track the health and performance of your applications: Azure Monitoring Integration for F5 NGINXaaS: An Azure-native solution delivering detailed analytical reports and customizable alerts. Grafana Dashboard Support: A visualization tool specifically designed to provide real-time, actionable insights into system health and performance. Dive Deep with Azure Monitoring for F5 NGINXaaS Azure Monitoring integration with F5 NGINXaaS provides a comprehensive observability solution tailored to dynamic cloud environments, equipping teams with the tools to enhance application performance and reliability. A crucial aspect of this solution is the integration of F5 NGINXaaS access and error logs, which offers insights essential for troubleshooting and resolving issues effectively. By combining these logs with deep insights into application and performance metrics such as request throughput, latency, error rates, and resource utilization, technical teams can make informed decisions to optimize their applications. Key Features Include: Advanced Analytics: Explore detailed traffic patterns and usage trends to better understand application load dynamics. This allows teams to fine-tune configurations and improve performance based on actual user activity. Customizable Alerts: Set specific thresholds for key performance indicators to receive immediate notifications about anomalies, such as unexpected spikes in 5xx error rates or latency challenges. This proactive approach empowers teams to resolve incidents swiftly and minimize their impact. Detailed Metrics: Utilize comprehensive metrics encompassing connection counts, active connections, and request processing times. These insights facilitate better resource allocation and more efficient traffic management. Logs Integration: Access and analyze F5 NGINXaaS logs alongside performance metrics, providing a holistic view of application behavior. This integration is vital for troubleshooting, enabling teams to correlate log data with observability insights for effective issue identification and resolution. Scalability Insights: Monitor real-time resource allocation and consumption. Predict growth challenges and optimize scaling decisions to ensure your F5 NGINXaaS service deployments can handle variable client load effectively. By integrating Azure Monitoring with F5 NGINXaaS, organizations can significantly enhance their resilience, swiftly tackle performance challenges, and ensure that their services consistently deliver outstanding user experiences. With actionable data at their fingertips, teams are well-positioned to achieve operational excellence and foster greater user satisfaction. Visualize Success with Native Azure Grafana Dashboard Enable the Grafana dashboard and import the F5 NGINXaaS metrics dashboard to take your monitoring capabilities to the next level. This dynamic integration provides a clear view of various performance metrics, allowing teams to make informed decisions backed by insightful data. Together, Azure Monitoring and the Grafana Dashboard form a strong alliance, creating a comprehensive observability solution that amplifies your application’s overall performance and reliability. The Grafana interface allows real-time querying of performance metrics, offering intuitive visual tools like graphs and charts that simplify complex data interpretation. With Azure Monitoring, Grafana builds a robust observability stack, ensuring proactive oversight and reactive diagnostics. Getting Started with NGINXaaS Azure Workshop We have curated self-paced workshops designed to help you effectively leverage the enhanced observability features of F5 NGINXaaS. These workshops provide valuable insights and hands-on experience, empowering you to develop robust observability in a self-directed learning environment. Azure monitoring lab workshop will enhance your skills in creating and analyzing access logs with NGINX. You’ll learn to develop a comprehensive log format, capturing essential details from backend servers. By the end, you'll be equipped to use Azure’s monitoring tools effectively, significantly contributing to your growth and success. In the Native Azure Grafana Dashboard workshop, you'll explore the integration of F5 NGINXaaS for Azure with Grafana for effective service monitoring. You'll create a dashboard to track essential metrics for your backend servers. This hands-on session will equip you with the skills to analyze real-time data and make informed decisions backed by valuable insights. Upon completing this lab exercise, you will have gained practical expertise in leveraging enhanced observability features of F5 NGINXaaS. You will be proficient in creating and analyzing access logs, ensuring you can effectively capture critical data from backend servers. Additionally, you will have developed the skills necessary to integrate F5 NGINXaaS with Grafana, allowing you to build a dynamic dashboard that tracks essential metrics in real-time. This hands-on experience will empower you to make informed decisions based on valuable insights, significantly enhancing your capabilities in monitoring and maintaining your applications. Conclusion By fully utilizing the observability features of F5 NGINXaaS, the organization can gain valuable insights that enhance performance and efficiency. With Azure Monitoring and Grafana working together, teams can manage proactively and make informed, data-driven decisions. This approach leads to smoother web experiences and improves operational performance. Interested in getting started with F5 NGINXaaS for Azure? You can find us on the Azure marketplace.
