Automatic Certificate Management with ACMEv2 in F5 BIG-IP
One of the most anticipated features of F5 BIG-IP is integration with ACMEv2. With the General Availability of BIG-IP 21.1.0 on May/26, this feature came into being. In this tutorial, we are going to configure it, using Let's Encrypt as the CA. The domain for which we are generating/renewing certificates is carlosf5lab.lat. The official docs for this feature are located in SSL Certificate Management | BIG-IP Documentation. Pre-requisite 1: DNS Resolver that can reach the internet (at least the CA endpoints). In this case, we are using the native DNS Resolver that comes with BIG-IP. Pre-requisite 2: The internal proxy that will make the connection with the CA. Pre-requisite 3: a self signed SSL certificate that the ACMEv2 protocol uses as the identifier for a device account. You don't have to fill the Subject Alternative Name. For the Common Name, an e-mail contact is advised. Now, we are going to create the ACME Provider object. Give it a name, and select the internal proxy previously created. For the CA Certificate to enable the secure connection with the Directory URL, you can use the default ca-bundle.crt. The Directory URL is the endpoint for the ACMEv2 protocol. In Let's Encrypt case, it is https://acme-v02.api.letsencrypt.org/directory For the Account Key, choose the previously created self-signed certificate. For the trickier part of all, the field "Contacts" is mandatory, and it must be an URL. That’s why you must use the format mailto:email_address. Check the Terms and Conditions, and the Create Account boxes. After a while, the Account Status must read as "Valid". To prove you own the domain whose certificate Let's Encrypt is going to create/renew, it must be pointing to an IP (A Record) where you must have your Virtual Server listening on Port 80 configured to respond to the ACMEv2 Challenge. (In this specific lab, the domain carlosf5lab.lat points to a Public IP mapped to an internal IP). Now you can order your first certificate via ACMEv2 on BIG-IP: After a while, the Key tab should read something like: Which means your certificate was generated: To track the ACME Provider, you can check its statistics: That's it, my friend! If it helped you, give a thumbs up to this post!
Strengthening your Digital Trust through F5’s Certificate Lifecycle Management partner ecosystem
In modern multi-cloud architectures, the F5 BIG-IP Application Delivery Controller (ADC) sits at the critical intersection of network traffic, application delivery, and security. Whether handling SSL/TLS offloading, inspecting encrypted traffic for threats, or enforcing Zero Trust access policies, a robust public key infrastructure (PKI) is part of nearly every enterprise BIG-IP deployment. However, as encryption levels approach 100% across enterprise traffic and industry standards push for significantly shorter certificate lifespans (such as 47-day validity limits), manually managing digital certificates and protecting private keys is no longer sustainable. A single expired certificate or compromised private key can lead to catastrophic application downtime, lost revenue, and severe compliance violations. The SSL/TLS Management Challenge within Enterprise Architectures As organizations scale their deployments across physical appliances (iSeries, rSeries, VELOS), virtual editions (VE), and public cloud instances, managing cryptographic assets introduces distinct operational challenges: Certificate Expiration & Unplanned Downtime: With hundreds or thousands of Client SSL and Server SSL profiles across multiple BIG-IP clusters, tracking expiration dates manually via spreadsheets inevitably leads to outages. Private Key Sprawl & Exposure: Storing high-value private keys in software files increases exposure to software vulnerabilities, unauthorized access, and side-channel attacks. Compliance Requirements: Industry mandates like PCI DSS, HIPAA, and FIPS 140-2/3 demand stringent controls around key generation, storage, and access auditing. Operational Overhead: Generating Certificate Signing Requests (CSRs), importing intermediate chain bundles, and binding certificates to SSL profiles across large fleets consume valuable engineering time. To address these hurdles, F5 integrates with leading security partners to offer end-to-end protection for keys and automated management for certificates. Each of these partners offer a complete SSL/PKI solution, and have built functionality specific to managing the certificates and keys that reside on the BIG-IP. Although full functionality amongst our partners may differ from vendor to vendor, at the foundation they all provide a way to manage the lifecycle of the certificates that reside on the BIG-IP. By making use of the BIG-IP's REST API, their management frameworks securely discover, catalog, rotate, and provision certificates on the BIG-IP, ensuring that Enterprises escape the challenges mentioned above. There are quite a few vendors offering CLM management for the BIG-IP. In alphabetical order, here are the ones that actively partner with F5 to provide this functionality. The CLM Partners in our ecosystem: AppViewX As an F5 partner for over a decade, AppViewX offers complete BIG-IP and NGINX system automation and management platform that includes CLM through their AVX ADC solution. For those opting for a standalone CLM solution (without the platform management) from AppViewX, AppviewX AVX One fits the bill. Guidance for configuring the AppViewX system to support BIG-IP can be found here. CyberArk Offering their CLM solution for BIG-IP as a self-hosted or SaaS offering, customers have choice in which model to adopt when leveraging CyberArk's Next-Generation Trust Security (NGTS) for BIG-IP. Guidance for configuring the CyberArk solution for BIG-IP can be found here. DigiCert Through DigiCert's "F5 BIG-IP LTM connector", the DigiCert® Trust Lifecycle Manager can not only offer full lifecycle management of BIG-IP, but it also offers certificate discovery and import as well. Guidance for configuring the DigiCert solution for BIG-IP can be found here. Encryption Consulting One of our newer partnerships, Encryption Consulting has extended their CertSecure Manager solution to offer CLM support for F5, focusing on BIG-IP & SSL Orchestrator solutions. Guidance for configuring the Encryption Consulting solution for BIG-IP can be found here. Entrust The Entrust nShield Connect HSMs works with BIG-IP systems to provide FIPS-certified protection of SSL certificates and encryption/decryption keys. The nShield architecture includes a Remote File System (RFS) that stores and manages the encrypted key files, supporting BIG-IP platforms including the Local Traffic Manager (LTM), the Domain Name System (DNS) – formerly Global Traffic Manager (GTM), the VIPRION Series, and the BIG-IP Virtual Edition (VE). Guidance for configuring the Entrust nShield HSM solution for BIG-IP can be found here. KeyFactor With a focus on F5 BIG-IP, BIG-IQ, & WAF, Keyfactor has exended their Certificate Lifecycle Automation platform to support F5 through their "F5 Orchestrator" plugin for Keyfactor Command Guidance for configuring the KeyFactor Orchestrator solution for BIG-IP can be found here. Sectigo Built using F5's Kojot ACME client, Sectigo offers full certificate lifecycle management for BIG-IP ADSP platforms through the Sectigo Certificate Manager (SCM) platform. Guidance for configuring the Sectigo solution for BIG-IP can be found here. Thales Thales Luna HSM solution includes full BIG-IP CLM support that lives within their larger SSL/PKI infrastructure solution. Guidance for configuring the Sectigo solution for BIG-IP can be found here.
Observing F5 BIG-IP CNE CNFs - V2 Metrics Aggregation walkthrough
Introduction Cloud-native distribution solves scale and resilience, but it fragments visibility. When TMM pods run across multiple Kubernetes nodes, logs and metrics scatter with them. The observer pod exists to pull that telemetry back into a single, coherent view. In a CNF deployment, multiple TMM pods run across Kubernetes nodes, each handling separate traffic slices. A coherent view of the entire dataplane requires aggregating those per-pod statistics. CNFs generate stats at high frequency across all TMM pods. Without aggregation, per-pod metric streams multiply quickly and do not compose cleanly into useful dashboard or alerting data. if you upgraded recently to CNF2.2+ you may have encountered new metrics behavior where, that's what we are covering here how V2 metrics change the metric collection and troubleshooting behavior. TODA Architecture: Four Components TODA (Telemetry, Observability, Diagnostics, and Analytics) is the stats collection and aggregation layer for CNFs. The distributed model has four roles: Component Description TMM Scraper Sidecar container in each TMM pod. Replaces tmstatsd. Serves metrics from tmctl over a gRPC response stream when requested by a Receiver. Receiver Runs as a StatefulSet. Scrapes metrics from assigned TMM Scrapers, persists them, and forwards to the Observer over gRPC with mutual TLS (mTLS). Handles metrics from terminated TMM pods so cumulative data is not lost mid-scrape. Observer Runs as a StatefulSet. The aggregation engine. Pulls from Receivers, aggregates metrics across all TMMs per table, and exports to the OTEL collector. Emits internal telemetry covering gRPC call metrics, aggregation performance, and storage state. Operator Runs as a Deployment. Orchestrates lifecycle: discovers TMM Scrapers, Receivers, and Observers; load-balances TMMs across Receivers; applies aggregation mode and collection interval settings via a ConfigMap. V1 vs. V2 CNF Metrics Choose before deploying. V1 and V2 use incompatible metric naming in Prometheus, so PromQL queries written for one will not work on the other. V1 (legacy): tmstatsd runs in each TMM pod and streams metrics directly to OTEL with no aggregation. Metric names look like: virtual_server_stat/spk-app-1-spk-app-tcp-8050-f5ing-testapp-virtual-server/clientside.bytes_out Each metric carries a tmmID attribute identifying the source pod. Six TMM pods means six separate data streams for the same virtual server. Dashboards scale poorly. V2 (current): The Receiver and Observer aggregate before export to OTEL. The equivalent metric: f5.virtual_server.clientside.received.bytes Attributes include f5.virtual_server.name, k8s.namespace.name, and observer.job.mode: aggregated. One metric, unified across all TMMs, with naming aligned to OpenTelemetry semantic conventions. Use V2 for new deployments. V1 remains only for environments not yet migrated. Deploying the Observer with Helm Install the Observer in the same namespace as your F5Ingress. Get the chart version from your CNFs software package: cd cnfinstall ls -1 tar | grep observer # f5-toda-observer-v4.56.4-0.0.15.tgz Create an observer_values.yaml. At minimum, set the image registry and storage class: image: repository: your-registry.example.com persistence: storageClassName: '' accessMode: ReadWriteOnce size: 3Gi platformType: robin fluentbit_sidecar: image: repository: your-registry.example.com fluentbit: tls: enabled: true fluentd: host: f5-toda-fluentd.cnf-gateway.svc.cluster.local. Install: helm install observer f5-toda-observer-<VERSION>.tgz -f observer_values.yaml Note: The Operator and Receivers share a volume. If they run on the same node, any StorageClass works. If Receivers are distributed across multiple nodes, use a ReadWriteMany-compatible StorageClass, NFS is the standard choice. In my lab I'm installing to a cne-core namespace instead of default namespace. Also, Make sure to update BIG-IP Controller ingress values, as below f5-tmm: ... ... observer: enabled: true image: repository: local.registry.com f5-toda-logging: enabled: true type: stdout fluentd: host: f5-toda-fluentd.cne-core.svc.cluster.local. tmstats: enabled: false Once updated upgrade your helm installation helm upgrade f5ingress f5ingress-v15.82.0-0.2.50.tgz -f deployment/values-ingress-v2.yaml -n cnf-fw-01 Now, you have all the components ready, you can reference the below steps for additional integrations with Grafana and Prometheus. Lab notes In my lab there are some commands I had to run to adjust to the openshift deployment, helm upgrade observer f5-toda-observer-5.22.10-0.2.4.tgz -n cne-core --reuse-values --set persistence.storageClassName=openebs-hostpath oc adm policy add-scc-to-user hostmount-anyuid -z f5-observer -n cne-core oc adm policy add-scc-to-user hostmount-anyuid -z f5-observer-operator -n cne-core oc adm policy add-scc-to-user hostmount-anyuid -z f5-observer-receiver -n cne-core oc secrets link f5-observer <secret> --for=pull -n cne-core oc secrets link f5-observer-operator <secret> --for=pull -n cne-core oc secrets link f5-observer-receiver <secret> --for=pull -n cne-core Wiring Prometheus and Grafana to CNF Metrics The OTEL collector exposes a Prometheus-compatible endpoint on TCP port 9090. It requires mTLS, so valid certificates must be in place before the scrape job succeeds. Step 1 — Create a Prometheus namespace and certificate: kubectl create namespace prometheus kubectl apply -f prom-certs.yaml # cert-manager Certificate manifest Step 2 — Configure Prometheus to scrape OTEL with TLS: serverFiles: prometheus.yml: scrape_configs: - job_name: bnk-otel scheme: https static_configs: - targets: - otel-collector-svc.default.svc.cluster.local:9090 tls_config: cert_file: /etc/prometheus/certs/tls.crt key_file: /etc/prometheus/certs/tls.key ca_file: /etc/prometheus/certs/ca.crt insecure_skip_verify: false server: extraVolumes: - name: prometheus-tls secret: secretName: prometheus-client-secret extraVolumeMounts: - name: prometheus-tls mountPath: /etc/prometheus/certs readOnly: true global: scrape_interval: 10s service: type: NodePort nodePort: 31929 persistentVolume: enabled: false Step 3 — Deploy via Helm: helm install prometheus oci://ghcr.io/prometheus-community/charts/prometheus \ -n prometheus --atomic -f values.yaml //Update otel config map and change line 47 to following debug: verbosity: detailed //Then add following at the end of the configmap exporters: - otlp - deb Step 4 — Verify the scrape target is healthy: curl http://<node-ip>:31929/api/v1/targets | jq Step 5 — List all CNF metrics currently ingested: curl http://<node-ip>:31929/api/v1/label/__name__/values | jq Step 6 — Run a quick query to validate data is flowing: curl "http://<node-ip>:31929/api/v1/query?query=f5_tmm_f5_pool_member_serverside_connections_count_total" | jq For Grafana, add Prometheus as a data source. F5 provides a pre-built Observer dashboard JSON on CloudDocs. The dashboard has three sections: gRPC metrics — Communication performance between Observer containers: call latency and request counts. Go Runtime metrics — Pod resource consumption: goroutine counts, heap memory, object allocation rates. Storage/Aggregation metrics — How the Observer handles dead TMM pod data. When a TMM pod terminates, the Observer runs merge operations to consolidate its metrics. This section shows whether those operations are healthy. Note, you need to update your OTEL definition to include the below //Update otel config map debug: verbosity: detailed //Then add following at the end of the configmap exporters: - otlp - deb //Apply updated OTEL configmap Once done, proceed to rollout the otel deployment oc rollout restart deployment otel-collector -n cnf-fw-01 oc logs otel-collector-6b5c9d5f89-qccqv -f | grep profile_tcp -> table: Str(profile_tcp_stat) -> table: Str(profile_tcp_stat) -> table: Str(profile_tcp_stat) -> table: Str(profile_tcp_stat) -> table: Str(profile_tcp_stat) -> table: Str(profile_tcp_stat) -> table: Str(profile_tcp_stat) -> table: Str(profile_tcp_stat) -> Name: f5.profile_tcp.accepts -> f5.profile_tcp.name: Str(tmstat_tcp) -> f5.profile_tcp.vs_name: Str(qkview_for_tmstatsd) -> observer.job.name: Str(cnf-fw-01/default-scrape-template-66456f74c4/default-job-profile_tcp_stat) -> Name: f5.profile_tcp.accepts -> f5.profile_tcp.name: Str(_mcptcp) -> f5.profile_tcp.vs_name: Str(grpc_mt_10:2) -> observer.job.name: Str(cnf-fw-01/default-scrape-template-66456f74c4/default-job-profile_tcp_stat) -> Name: f5.profile_tcp.accepts -> f5.profile_tcp.name: Str(_mcptcp) -> f5.profile_tcp.vs_name: Str(grpc_mt_4:0) -> observer.job.name: Str(cnf-fw-01/default-scrape-template-66456f74c4/default-job-profile_tcp_stat) -> Name: f5.profile_tcp.accepts -> f5.profile_tcp.name: Str(_mcptcp) -> f5.profile_tcp.vs_name: Str(_grpc_tmm_listener_9) -> observer.job.name: Str(cnf-fw-01/default-scrape-template-66456f74c4/default-job-profile_tcp_stat) -> Name: f5.profile_tcp.accepts -> f5.profile_tcp.name: Str(_cgctcp_in) Troubleshooting via Metrics V2 Now, we have better capabilities of actually monitoring traffic across multiple pods and TMMs from single location, [cloud-user@ocp-provisioner f5-cne-2.2.0]$ oc exec sts/f5-observer-receiver -n cne-core -- mdb --list | grep "/cnf-fw-01/" Defaulted container "f5-observer-receiver" out of: f5-observer-receiver, fluentbit 2026/07/15 18:12:09 INFO dialing to observer addr=0.0.0.0:8088 f5-log-ID=0612007a cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/fw_context_stat 2026-07-15 18:11:22 763 cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/virtual_server_stat 2026-07-15 18:11:22 3101 cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/dns_cache_resolver_stat 2026-07-15 18:11:22 5162 cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/profile_dns_stat 2026-07-15 18:11:22 7500 cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/profile_tcp_stat 2026-07-15 18:11:22 972 cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/pool_member_stat 2026-07-15 18:11:22 2156 cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/fw_rule_stat 2026-07-15 18:11:22 1588 cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/dos_stat 2026-07-15 18:11:22 3828 cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/fw_container_stat 2026-07-15 18:11:22 1213 Now, let's have a closer look at one of the segments, below is the FW context [cloud-user@ocp-provisioner f5-cne-2.2.0]$ oc exec sts/f5-observer-receiver -n cne-core -- mdb --segment cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/fw_context_stat Defaulted container "f5-observer-receiver" out of: f5-observer-receiver, fluentbit 2026/07/15 18:12:49 INFO dialing to observer addr=0.0.0.0:8088 f5-log-ID=0612007a -----BEGIN RESOURCE----- Meta: Name: fw_context_stat Unit: Annotations: k8s.namespace.name: cnf-fw-01 Labels: f5.firewall.context.context_name: cnf-fw-01-forwarding-any-virtual-server-SecureContext_vs f5.firewall.context.context_type: virtual f5.firewall.context.policy_type: 1 TTL: 0s Value: [2428 0 0 0] -----END RESOURCE----- Now, let's have a look at the virtual servers stats [cloud-user@ocp-provisioner f5-cne-2.2.0]$ oc exec sts/f5-observer-receiver -n cne-core -- mdb --segment cnf-fw-01/tmm/f5-tmm-fcc888779-w98zh:f5-tmm:ae28ae8bf4ecb2de9f9a1d0674fb47f8acb2eab1bb46727e285e2cbcae87266b/cnf-fw-01/virtual_server_stat Defaulted container "f5-observer-receiver" out of: f5-observer-receiver, fluentbit 2026/07/15 18:14:24 INFO dialing to observer addr=0.0.0.0:8088 f5-log-ID=0612007a -----BEGIN RESOURCE----- Meta: Name: virtual_server_stat Unit: Annotations: k8s.namespace.name: cnf-fw-01 Labels: f5.virtual_server.destination: 0.0.0.0 f5.virtual_server.name: cnf-fw-01-forwarding-any-virtual-server-SecureContext_vs f5.virtual_server.source: 0.0.0.0 TTL: 0s Value: [1844667 108023672 0 4 28282 69732 2542 0 0 0 0 108023672 1844587 0 4 69732 28280 2542] -----END RESOURCE----- -----BEGIN RESOURCE----- Meta: Name: virtual_server_stat Unit: Annotations: k8s.namespace.name: cnf-fw-01 Labels: f5.virtual_server.destination: 10.1.20.100 f5.virtual_server.name: cnf-fw-01-cnf-dohapp-virtual_server f5.virtual_server.source: 0.0.0.0 TTL: 0s Value: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] -----END RESOURCE----- -----BEGIN RESOURCE----- Meta: Name: virtual_server_stat Unit: Annotations: k8s.namespace.name: cnf-fw-01 Labels: f5.virtual_server.destination: 10.1.30.100 f5.virtual_server.name: cnf-fw-01-dnsx-app-listener-virtual_server f5.virtual_server.source: 0.0.0.0 TTL: 0s Value: [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] -----END RESOURCE----- With Observer you have aggregated observer receiver to monitor and observe your CNF deployment. Conclusion As a conclusion, why would you go to V2 metrics vs V1, there are four pain points with V1 in distributed environments: Stream count; One virtual server across six TMM pods gives you six independent metric series in Prometheus, same stat, six rows, differentiated only by tmmID. That's not an observability system. That's a spreadsheet you have to reassemble manually every time you open Grafana. Data loss; A pod gets evicted mid-scrape and its cumulative counters are gone. The Receiver in the TODA pipeline holds that data in a local volume and merges it before export. Your charts stay clean. PromQL tax; In V1, every panel needs a sum() by (virtual_server) wrapper or the numbers are wrong. Not approximately wrong, it's wrong by a factor of N. V2 aggregates inside the pipeline, at the Observer, before the metric ever reaches Prometheus. One series. Use it directly. The naming; V1 embeds the VS name in the metric path. That's not how Prometheus is supposed to work, and everything downstream, like alerting rules, federation, label matchers fight it. V2 puts the VS identity where it belongs: as an attribute, f5.virtual_server.name. Short metric name, proper labels, and PromQL that actually reads like PromQL. The TODA pipeline ( TMM Scraper, Receiver, Observer ) exists specifically to close the visibility gap. The Receiver is the persistence layer. The Observer is the aggregation engine. Together they turn N pod streams into one coherent signal. If you take one thing from this: aggregation has to happen somewhere. In V1, it happens in your head, on every dashboard panel, every time. In V2, it happens in the pipeline, once, before the data leaves the cluster. That's the whole difference. Related Resources Distributed TODA for Stats Aggregation CNFs Event Logs Performance Visualization (Prometheus + Grafana) OTEL Statistics Reference CNF Log Formats Reference Debug Sidecar Overview Troubleshooting Common Errors
How I Did it: Create a Custom SNMP OID
Shamelessly stealing the title lead in from author and colleague Greg_Coward, I thought I'd share a solution for a question I answered in the forum around SNMP OIDs (the TL;DR is there) The request focused on an SNMP OID for the device trust certificate expirations. I walked the mibs and searched through them on the system, but I didn't find any that covers that scenario. However, I had a thought in my brain's wayback machine that it was possible to create custom mibs. This article walks through a brief example on how to accomplish exactly that. Resources 13.1 Implementations Manual - Monitoring BIG-IP System Traffic with SNMP K000151455 - Monitoring Device Trust Certificate Expiry on BIG-IP Using a Shell Script Configure & Test the Expiry Script The script works as is in the K article, so you can just copy it into a file on your BIG-IP. I used /config/check_status.sh in my test but you probably want something more descriptive of what it's checking. The steps I used at the bash prompt: vi /config/check_status.sh :set paste i paste the script from the knowledge article linked above and hit the escape key :wq! If you use an inferior (I said what I said... :) ) editor, these steps will look a little different. Now that the file is in place, you need to set it to executable (for our test here) and you need to change the SELinux context for later to make that file accessible to snmpd to execute. chmod +x /config/check_status.sh chcon -t bin_t /config/check_status.sh Now run the script from the command line: [root@bigip02:Active:In Sync] config # ./check_trust.sh Certificate is valid until Sep 30 21:46:25 2031 GMT (1895 Days remaining) Ok, the script is running and reporting accurately against my device trust certificate. Now let's put it to work. Configure & Test the Custom OIDs First, create your custom_mibs.tcl file in the /config/snmp/ directory: vi /config/snmp/custom_mibs.tcl :set paste i paste the script below these steps and hit the escape key :wq! # /config/snmp/custom_mib.tcl # .1.3.6.1.4.1.3375.2.100.5.0 -> trust cert, full status string # .1.3.6.1.4.1.3375.2.100.6.0 -> trust cert, days remaining (int) register_mib ".5" trust_cert_status string register_mib ".6" trust_cert_days int proc trust_cert_status {} { set status [catch {exec /config/check_trust.sh} result] if {$status != 0 || $result eq ""} { return "ERROR: check_trust.sh failed" } return [string trim $result] } proc trust_cert_days {} { set status [catch {exec /config/check_trust.sh} result] if {$status != 0} { return -1 } if {[regexp {\(([0-9]+) Days remaining\)} $result -> days]} { return $days } return -1 } A couple notes before we test: You only need one of these, but I provided both the full string returned from the KA script and the days remaining numeric value as a standalone as well. Your call to keep both or pick one. If you have other custom mibs already that someone before your time created, you'll need to make sure you avoid overwriting another value. I picked .5.0 and .6.0 here, but you can set it to whatever you want. I am using the default SNMP state on my test box, but you hopefully have something more secure configured for running your tests. Ok, let's restart snmpd and make sure the custom mibs were picked up: [root@bigip02:Active:In Sync] snmp # bigstart restart snmpd [root@bigip02:Active:In Sync] snmp # grep -i "custom mib" /var/log/snmpd.log custom mib initialization completed. total 2 custom mib entry registered Good to go...now the test with snmpget: [root@bigip02:Active:In Sync] snmp # snmpget -v2c -c public localhost .1.3.6.1.4.1.3375.2.100.5.0 F5-BIGIP-COMMON-MIB::bigipTrafficMgmt.100.5.0 = STRING: "Certificate is valid until Sep 30 21:46:25 2031 GMT (1895 Days remaining)" [root@bigip02:Active:In Sync] snmp # snmpget -v2c -c public localhost .1.3.6.1.4.1.3375.2.100.6.0 F5-BIGIP-COMMON-MIB::bigipTrafficMgmt.100.6.0 = STRING: "1895" Huzzah! Now we have functional SNMP OIDs we can target to check status of the device trust certificate expiry. Thank you to Blue_whale for asking the question! It was nice to revisit SNMP after so many years away with APIs and all.
ADC01 – Weak DNS Practices
Introduction DNS is often the unsung hero of application delivery, quietly humming along until something goes wrong. This cornerstone of internet infrastructure translates human readable domain names into machine friendly IP addresses, bridging the gap between users and applications. While vital, DNS frequently gets overlooked, leading to unforeseen performance, availability, scalability, and security issues in application delivery. In today's interconnected world, where seamless application delivery is critical across industries such as finance, healthcare, insurance, telecommunications, hi-tech, energy, government, retail and e-commerce, automotive, and manufacturing, weak DNS practices can wreak havoc. Since stakes are high and user expectations demand near instantaneous responsiveness and reliability, the principles discussed here apply universally. Businesses in any sector must recognize DNS as more than just a basic utility because it is a foundational layer of their application delivery strategy. AI Reference Architecture Use Case example: Enhancing DNS Security and Performance with F5 BIG-IP To better understand how optimized DNS practices strengthen application delivery, let’s consider the following real world use case: Steps in the DNS Optimization Process: Client Initiates DNS Query: A client, located on the external network, initiates a DNS query to resolve a domain name. Query Passes Through F5 BIG-IP: The F5 BIG-IP acts as a secure DNS proxy, validating the query and applying DNSSEC (Domain Name System Security Extensions) for added cryptographic security. DNSSEC ensures that queries are not tampered with and originate from a legitimate source. Authoritative DNS Responds: The validated query is routed securely to the authoritative DNS server cluster within the internal network. The authoritative DNS responds with the appropriate IP address. Response Returned to Client: The optimized, secure response is returned to the client with minimal latency, leveraging DNSSEC and optimized TTL (Time-to-Live) settings for better user experience. This workflow illustrates how incorporating F5 BIG-IP into your DNS architecture can enhance application security, scalability, and performance. Consequences of Weak DNS Practices Impact on Performance DNS inefficiencies are a hidden bottleneck for application performance. In 2023, nearly 27% of user complaints about poor application performance stemmed from DNS related slowdowns (Auvik). When DNS servers aren’t optimized or critical security features like DNS Security Extensions (DNSSEC) are absent, the impact cascades across application performance: Latency increases: Low TTL (Time-to-Live) settings overload DNS servers with repeated queries, slowing down responses for users spread across regions. Vulnerability to hijacking: Without DNSSEC, attackers can intercept or redirect traffic to slower or malicious servers, significantly impacting response times and user experience. Impact on Availability DNS disruptions, either due to attacks or misconfigurations, can lead to major availability issues. For instance: DNS hijacking or cache poisoning: Attackers inject fake records into DNS servers, redirecting users to harmful websites. Without DNSSEC, these vulnerabilities remain exploitable, leading to user mistrust. Low or mismatched TTL settings: These settings exacerbate availability problems by overwhelming DNS servers during failovers or scaling events, making applications inaccessible during critical times. Cloud-based global applications, which rely on dynamic DNS updates to remain accessible, worsen the problem when DNS configurations cannot keep pace with the scaling requirements of infrastructure. Impact on Scalability Scalability is a fundamental goal for any global application. Yet, weak DNS practices create bottlenecks: Dynamic DNS updates: Insecure or improperly authenticated dynamic DNS updates disrupt routing, crippling the application’s ability to handle increased user demand. Unprepared DNS infrastructure: With global audiences and fluctuating traffic volumes, under-provisioned or misconfigured DNS servers can experience elevated latency and outages, hindering business growth and reducing global reach. Impact on Operational Efficiency Operational inefficiencies arise from frequent DNS queries (due to low TTLs) and insecure configurations. IT teams become mired in troubleshooting and responding to incidents like DDoS attacks, instead of focusing on broader strategic goals. Moreover, the resources directed toward mitigating DNS-related issues inflate operational costs unnecessarily, wasting valuable time and money. The Undeniable Need for Best Practices To avoid these pitfalls, organizations need to prioritize robust DNS architectures. Here are key practices to implement: DNSSEC for Security DNSSEC (Domain Name System Security Extensions) operates as a safeguard against cache poisoning and DNS hijacking attacks. Implementing DNSSEC ensures the cryptographic validation of DNS records, mitigating risks from unauthorized changes. This adds a layer of trust and reliability to your DNS infrastructure and by extension to user experiences. DNSSEC is no longer a luxury, but a necessity for securely connecting users across the globe. Optimized TTL Settings for Balance TTL settings determine how long DNS information is cached by resolvers before they query authoritative DNS servers. Too short a value results in frequent DNS lookups, increasing server load and latency. Conversely, excessively long TTL values cause outdated information during dynamic scaling. Organizations should analyze traffic patterns and application behavior to strike the right balance. Regular revisions of TTL settings ensure that user queries are routed efficiently without bottlenecking operations. Secure Dynamic DNS Update Dynamic DNS updates are essential for cloud-based infrastructures where IP addresses frequently change. However, insecure update mechanisms are a gold mine for attackers, providing opportunities to alter DNS records maliciously. By using authentication and encryption mechanisms for DNS updates, organizations can protect records and optimize routing seamlessly. Distributed DNS Architecture Relying on a single DNS service provider or a centralized DNS setup increases the risks of downtime. By spreading DNS across multiple geographically independent providers and locations, organizations ensure better fault tolerance and reliability. Global Implications of Weak DNS In 2023, 90% of organizations faced DNS attacks, with financial losses averaging $1.1 million per incident (EfficientIP). These attacks are not just a cost center but also a real time disruption that tarnishes brand reputation and user trust. Each organization encounters an average of 7.5 DNS attacks annually, underlining the broad spectrum of vulnerabilities across industries. Global operations are particularly susceptible to DNS misconfigurations and weaknesses. Whether through hijacking traffic, launching DDoS campaigns, or exploiting TTL mismanagement, attackers leverage DNS vulnerabilities to cripple applications and compromise sensitive data. To scale securely, organizations need DNS practices aligned with industry standards, steering away from outdated configurations or reliance on "default" infrastructure. Final Thoughts Weak DNS practices are a silent killer of application performance, availability, scalability, and operational efficiency. The good news? These challenges are entirely avoidable. By implementing DNSSEC, optimizing TTL settings, securing dynamic DNS updates, and adopting distributed DNS strategies, organizations can dramatically reduce these risks. In the digital landscape of today, where milliseconds can dictate millions in revenue, DNS cannot be an afterthought. It must be a core pillar of application delivery strategies, ensuring that every user query is answered rapidly, reliably, and securely. DNS influences every aspect of the user experience. It’s the first impression your application makes. Don’t squander it. As your applications scale to meet global demand, strong DNS practices will ensure they deliver consistently, no matter where users click “Buy Now.” Reference Articles Scaling, Securing, and Optimizing DNS Hyperscale and Protect Your DNS While Optimizing Global App Delivery Intelligent DNS Firewall for Service Providers F5 DNS: Global Server Load Balancing The Application Delivery Top 10 ADSP Platform overview AI reference architecture The BIG-IP GTM: Configuring DNSSEC Configuring BIG-IP for Zone Transfer and DNSSECStreamlining F5 WAF for NGINX Telemetry to Splunk via F5 NGINX One Console
Modern application delivery platforms require centralized security management and observability. F5 NGINX One Console addresses this need by offering a unified management plane for distributed NGINX fleets, including a built-in Security Dashboard that provides platform and security teams with instant visibility into WAF activity, threat spikes, and active enforcement policies across instances. However, in enterprise environments, security operations are rarely isolated. Security Operations Center (SOC) teams rely heavily on SIEM platforms like Splunk as their central command center for threat correlation, incident response, and forensic investigations. While the built-in NGINX One Console Security Dashboard works well for platform operators, enterprise SecOps teams need WAF event data integrated seamlessly into their existing SIEM platforms. What was missing was an automated, effortless export mechanism to stream security logs from F5 WAF for NGINX into Splunk—eliminating complex manual log formatting and custom pipeline management. The Traditional Challenge: Log Format Expertise and Configuration Drift SecOps teams live in Splunk. It's where they correlate threats, investigate incidents, and build forensic timelines. But getting WAF security logs into Splunk has traditionally been a pain: **Custom log formats** -- Engineers had to hand-craft key-value or JSON schemas that Splunk could parse without choking on syntax errors. **Manual config edits on every instance** -- Someone had to SSH into each NGINX node to set up log templates and syslog destinations. **Configuration drift at scale** -- Managing logging configs individually across multi-cloud or containerized deployments meant inconsistent profiles and constant maintenance overhead. The result? WAF protection and SOC visibility lived in separate worlds. Security telemetry was harder to set up than the security policy itself. Closing the Gap: GUI-Driven Log Profile Management in NGINX One Console To eliminate this operational friction, F5 NGINX One Console introduces centralized GUI-driven Log Profile Lifecycle Management for F5 WAF for NGINX. Rather than manually authoring log format directives or updating individual instance configuration files, teams can now define, deploy, and manage Splunk-ready logging profiles across distributed NGINX environments in just a few clicks. What's under the hood: Built-in Splunk Template (log_f5_splunk): Pre-configured with an optimized key-value pair schema designed for native ingestion and automatic field extraction in Splunk. Centralized Deployment Engine: Allows administrators to define log profiles (capturing legal, illegal, or all requests) and push them out uniformly across targeted NGINX instances or instance groups. Automated Pipeline Configuration: Generates and validates the required NGINX directives automatically, ensuring seamless, error-free integration with remote syslog collectors. Video Walkthrough & Live Attack Demonstration Watch how NGINX One Console simplifies log profile deployment and enables real-time threat tracing in Splunk: Conclusion Securing modern web applications requires a tight feedback loop between threat protection and threat visibility. While F5 WAF for NGINX provides robust, low-latency defense at the application edge, the F5 NGINX One Console completes the equation by making security telemetry effortless to deploy and standardize. By delivering GUI-driven, Splunk-native log profile management, organizations can eliminate the friction between platform management and security operations—ensuring every blocked attack contributes directly to SOC intelligence and faster incident response. Resources To learn more about configuring log profiles for your NGINX fleet, refer to the official F5 NGINX One Console Log Profile Documentation.Effective Traffic Management: Addressing ADC04's Insufficient Traffic Controls
Applications today face unprecedented demand variability, requiring organizations to prioritize effective traffic management to ensure a seamless user experience. ADC04, one of the key challenges in application delivery per F5's "Application Delivery Top 10," highlights insufficient traffic controls. This issue plagues high-demand environments across industries such as e-commerce, healthcare, high-tech, automotive, insurance, and more, leading to performance bottlenecks, reduced availability, inefficiencies, and scalability challenges. Let's discuss the implications and solutions for this issue, with an added use case to demonstrate practical implementation. AI Reference Architecture The Challenge of Insufficient Traffic Controls Modern digital ecosystems experience fluctuating traffic volumes due to use cases like e-commerce promotions, flash sales, or API-driven workloads. For example, an e-commerce API could go from handling hundreds of requests per second (RPS) to thousands or more during promotional events. Without proper rate limiting, throttling, or caching mechanisms, public APIs become vulnerable to overburdened backend services, Distributed Denial of Service (DDoS) attacks, and inefficient scaling. Automated processes, such as analytics workloads, CI/CD deployments, and backups, exacerbate uneven workloads on backend systems. Furthermore, modern AI applications with data-intensive processing introduce yet another layer of complexity, requiring sophisticated traffic controls. The Consequences of Inefficient Traffic Controls Performance Issues Excessive traffic without sufficient controls leads to backend overload, degrading application response time. For example: User Frustration: Slow response times result in poor customer experiences, where 70% of shoppers abandon purchases due to delays. Critical Application Failures: In AI use cases, such as real-time conversational bots, processing delays impact outcomes and user trust. Reduced Availability API services lacking rate limiting and traffic throttling are especially prone to DDoS attacks or service outages. Additionally, backend systems may face resource starvation due to unoptimized caching mechanisms, reducing availability even during normal traffic conditions. Limited Scalability Applications unable to intelligently manage traffic flows struggle to handle unexpected traffic spikes. Inefficient caching and chaotic workload distribution limit the capacity to scale dynamically. Operational Inefficiencies The absence of automated traffic controls forces manual monitoring and intervention during traffic surges. High operational overhead and inefficient resource usage affect cost-effective management of infrastructure. Use Case example: Implementing Traffic Controls for an E-Commerce Public API The diagram below illustrates a practical implementation of traffic controls using advanced Application Delivery Controller (ADC) features (e.g., BIG-IP Local Traffic Manager) for an e-commerce public API. While tailored for e-commerce, these are equally applicable to other industries, such as finance, insurance, high-tech, and automotive. Key Features for Effective Traffic Management: Global Rate Limiting: Enforces a cap on total requests per second (RPS) to avoid backend overload. Per-API Key Throttling: Limits RPS at an API key level (e.g., 100 RPS per user), ensuring fair usage. CAPTCHA Trigger: Introduces CAPTCHA challenges for clients exceeding predefined limits to mitigate abuse. Circuit Breaker Logic: Detects faults in backend API servers and reroutes traffic to stable instances where possible. Traffic Forwarding or Rejection: Directs healthy requests to backend API servers and rejects problematic traffic. Comprehensive Logging and Metrics: Logs control events, such as rate-limit breaches or circuit-break activations, and streams them into observability platforms like ELK, Prometheus, or Datadog for dashboards and automated alerts. Functionality Overview Clients Send API Requests: Public API clients interact with the ADC (BIG-IP LTM), where all incoming requests are initially processed. Rate Limiting and Key Enforcement: The ADC enforces global rate limits and per-client throttling policies to avoid overloading backend servers. Action on Violations: Requests breaching limits trigger appropriate actions—delays, CAPTCHAs, or outright rejection. Health-Check Monitoring: The ADC regularly monitors backend API servers for faults and dynamically reroutes traffic to healthy nodes using circuit-breaker policies. Logs and Metrics Streaming: All traffic patterns, rule activations, and violations are logged and streamed to observability platforms for operational transparency. Best Practice Recommendations for Traffic Controls Rate Limiting and Throttling Global and per client rate limits are critical for protecting backend services during demand surges. For example: Put a 100 RPS cap per API key. Differentiate limits for premium services or geographies where critical workloads demand higher quality of service. Intelligent Caching Caching reduces backend load by offloading repetitive requests: Use semantic and edge caching for dynamic workloads like AI APIs. Adopt adaptive caching mechanisms to handle variable traffic patterns and reduce latency. Circuit Breaker Logic Circuit breakers prevent cascading failures in the API ecosystem by rerouting traffic to healthy systems: Monitor backend server health dynamically. Use adaptive retry mechanisms to minimize disruptions. Observability and Logging Real-time logging and monitoring tools like ELK or Prometheus provide insights into API performance: Set up custom dashboards to monitor API health, rate-limit violations, and traffic loads. Automate alerts for anomalies (e.g., high latency or recurring limit breaches). Conclusion As demonstrated by the use case, robust traffic controls are essential for managing fluctuating workloads, ensuring availability, and optimizing operational efficiency. Layered controls such as rate limiting, advanced caching, and circuit breaker mechanisms enforce resilience and scalability. Integrating observability tools ensures transparency and rapid issue resolution in real time. With insufficient traffic controls identified as a major challenge, adopting these strategies is crucial for long-term API operational success, particularly for high-traffic environments across industries such as e-commerce, finance, insurance, high-tech, and automotive. Reference Article Managing Traffic with Bandwidth Controllers Intelligent Traffic Management with the F5 BIGIP Platform Mitigating OWASP API Security Risks: Unrestricted Resource Consumption using BIG-IP iRule::ology - Table Based Rate Limiting AI reference architecture ADSP Platform overview The Application Delivery Top 10ADC02 – Lack of Fault Tolerance & Resilience in Enterprise Applications Using A2A Protocol
Introduction In the world of enterprise applications, fault tolerance and resilience play a central role in ensuring uninterrupted service delivery. However, the absence of these critical components can lead to degraded performance, downtime, costly inefficiencies, and dissatisfied users. This article explores fault tolerance challenges using the A2A protocol in enterprise applications, leveraging F5 BIG-IP to resolve primary data center failures, as illustrated in the attached diagram. AI Reference Architecture The Use Case at a glance The architecture for this scenario involves: AI Clients initiating A2A traffic routed via a Primary BIG-IP LTM. The Primary BIG-IP LTM processes the requests and routes intelligently based on A2A protocol inspection. In the event of a Primary BIG-IP failure, a Standby BIG-IP LTM in a high availability (HA) configuration takes over seamlessly. AI Agents (hosted across multiple instances) process user traffic through the Active BIG-IP, ensuring continuous service availability. This structure ensures resilience while avoiding performance bottlenecks caused by load imbalances or failures. Consequences of a Lack of Fault Tolerance and Resilience Impact on Performance Without adequate fault tolerance mechanisms: Failures in a primary system increase the load on remaining servers, causing degraded response times. Systems experience 35% more downtime during high-load scenarios, as indicated by LoadView’s 2024 network performance report. Impact on Availability A lack of redundancy or failover capabilities results in prolonged downtime when a failure occurs, tarnishing organizational reputation and eroding user trust. In complex environments, cascading failures can be triggered, amplifying the chaos. Impact on Scalability Systems lacking fault tolerance cannot scale dynamically to meet changing traffic demands. Rapid traffic surges overwhelm resources, causing bottlenecks. Overprovisioning as a stopgap becomes costly and inefficient. Impact on Operational Efficiency When failures occur, manual interventions become necessary, which increase operational overhead, downtime, and costs. Automated mechanisms for failover and load balancing are critical in reducing reliance on human intervention and ensuring operational efficiency. Solutions to Enable Fault Tolerance and Resilience Using F5 BIG-IP Load Balancing with BIG-IP F5 BIG-IP's iRules dynamically route A2A traffic, ensuring intelligent management even in volatile conditions. A load balancing configuration includes: Active-Standby Configuration: Load balancing redirects traffic to the standby BIG-IP in case of failure. Active-Active Configuration (Optional): For consistently high traffic volumes, active-active HA ensures even traffic distribution, improving both availability and scalability. High Availability (HA) Setup BIG-IP’s HA architecture supports synchronized active and standby systems: Failover Objects and Floating IPs allow seamless rollover during primary system failures. Redundant servers prevent single points of failure, ensuring uninterrupted operations. Comprehensive Health Monitoring Advanced health checks go beyond simple pings to assess the full responsiveness and integrity of applications and supporting infrastructure: Use distributed health checks from geographically disparate locations to simulate actual user experiences. Create application-specific health checks to test backend systems fully. Programmable Infrastructure Programmable infrastructure with F5 BIG-IP allows organizations to: Customize fault-tolerance strategies tailored for specific applications. Adjust traffic dynamically in real-time using programmable application delivery controllers (ADCs). Automation for Instant Response By integrating failover automation, organizations can: Detect and mitigate failures faster, reducing downtime. Lower operational overhead by minimizing manual interventions. Best Practices for Fault Tolerance Optimization Readiness Planning Use resources like "BIG-IP HA - Do it the Proper Way" to correctly implement HA configurations. Synchronize configurations and session data between BIG-IP devices. Tailored Load Balancer Configurations Optimize load balancing policies for real-world traffic patterns. Implement automated traffic redirection during outages. Proactive Monitorin Constantly monitor application performance via distributed health checks described in the "F5 Academy - BIG-IP HA - Do it the Proper Way". Resilience Testing Periodically test failover functionality to ensure system readiness to handle failures under real-world conditions. Resource Scalabilit Leverage the F5 Active-Active HA Configuration for highly scalable environments. Why Fault Tolerance Matters Fault tolerance isn’t just a technical concept; it directly dictates application availability, performance, and scalability. Proactive strategies like HA, programmable infrastructure, and automation enable organizations to build resilient systems capable of handling any disruptions. Conclusion In the ever-evolving digital landscape, resilience and fault tolerance are no longer optional—they are imperative. Leveraging F5 BIG-IP solutions for HA, intelligent load balancing, and failover mechanisms ensures applications remain available, scalable, and efficient, even during disruptions. By building fault-tolerant systems, enterprises not only meet today’s challenges but also position themselves for future growth and stability. Learn More Explore these resources to dive deeper into enabling fault tolerance and resilience: Intro to: BIG-IP HA - Do it the Proper Way High availability on F5 BIG-IP load balancers F5 BIG-IP HA Active Standby Configuration F5 Active-Active HA Configuration F5 Academy - BIG-IP HA - Do it the Proper Way ADSP Platform overview AI reference architecture The Application Delivery Top 10
