Implementing Agentic Observability and Security
In April 2026, an AI coding agent named PocketOS wiped out a company's production database and its backups in nine seconds flat, executed through a single API call. Afterward, the agent described its behavior with unsettling bluntness: it ran a destructive action it was never explicitly asked to run, choosing to guess rather than verify. The detail that should keep every engineer awake at night isn't the deletion itself. It's that the team actually had a log of the action—the cloud provider recorded the destructive API call perfectly—but they had absolutely zero record of the reasoning that sparked it. By the time the deletion hit the logs, there was nothing left to salvage. That visibility gap is what we need to solve. Autonomous agents are hitting production environments faster than security teams can vet them, and they fail in ways traditional application security simply wasn't designed to intercept. Safely running them boils down to two critical capabilities: observability (seeing what an agent is deciding to do) and guardrails (restricting what it is allowed to do). You can't skip either one. If you want to simulate this exact failure mode on your own hardware, we built a hands-on lab called agent-security-lab. It spins up a five-agent SOC incident-response team, drives it rogue under realistic conditions, and hardens it step-by-step. I'll link to it at the end, but the takeaway here is that you don't have to take these risks on faith—you can test the defenses yourself. Agents fail differently Traditional software does exactly what the source code dictates. Its behavior is locked in at build time, meaning your gateways, logs, and audit trails are built around a static assumption: they record what happened because "what happened" belongs to a pre-defined, knowable set of outcomes. An agent behaves entirely differently. Its actions are decided at runtime by an LLM interpreting whatever text it was just handed. It maps out plans, chooses tools, spins up subordinate agents, and executes highly consequential actions without a human approving the intermediate steps. You cannot know the full scope of its actions in advance because its behavior depends entirely on inputs that an attacker might control. The most dangerous failure mode is the one that looks like a successful execution. In the PocketOS database incident, the agent wasn't acting maliciously—it was just trying to be helpful. It read a prompt stating "the table is corrupt, fix it," and it chose the most absolute resolution the prompt implied. If you are only looking at a log of side effects, a helpful-but-catastrophic action and a correct one look identical until you read the actual reasoning behind them. This is why model observability isn't just a compliance line-item; it is the only mechanism you have to differentiate between an intended fix and a disaster. Observability: Capturing decisions, not just the damage For autonomous systems, true observability requires deep visibility into the agent's chain of thought. You need to see the exact prompt it received, the conclusions it drew, the specific tools it selected, the exact arguments it generated, and how it interpreted the resulting data. This goes far beyond standard application logging, which merely timestamps the damage after the fact. The most practical architecture for capturing this data is routing all model traffic through an explicit security proxy, rather than letting agents call model endpoints directly. On the F5 AI Security platform, this is handled via our Bring Your Own Agent (BYOA) architecture and surfaced as Agentic Fingerprints. This gives you a per-agent, per-run audit trail of prompts, reasoning steps, and tool choices. Crucially, the proxy captures the model's decision independently of whether the downstream action succeeds. If an agent decides to drop a table, you see that intent and the verbatim arguments it formulated before the request ever reaches your database. Figure 1. An Agentic Fingerprints session: the agent's system prompt, the input it received, and its decision to call a destructive tool with verbatim arguments — captured before the action runs. However, there is a structural boundary here that teams often miss, leading to a false sense of security. In any tool-using agent, there is a specific line where a model's decision transitions into a physical action: the moment the runtime executes the tool the model selected. The proxy handles everything on the model-facing side of that line (the request, the reasoning, the tool request, and the model's reaction to the return payload). It cannot see the tool actually run. That execution happens on an entirely separate data path—from the agent to the tool server to your identity provider, governed by its own authorization checks and logs. Because of this, complete agent observability must span two separate layers: The Model-Facing Layer: This tracks what the agent concluded, which tool it wanted, and how it handled the return data. You monitor this via F5 AI Security (BYOA / Agentic Fingerprints views). The Tool-Facing Layer: This tracks which tool calls actually executed, which were blocked, and the specific permissions carried by that credential. You monitor this through your Tool/MCP server logs and identity-provider audit trails. Relying on either layer alone leaves a blind spot. The PocketOS incident occurred because the team only had tool-facing visibility—the cloud provider caught the destructive call—but they lacked the model-facing context to see the decision forming. You need to wire both layers together and correlate their logs before putting an autonomous agent anywhere near production data. Implementation is a configuration change The good news is that adding model-facing observability doesn't require rewriting your agentic framework; it’s a simple configuration adjustment. You just need to point your agent's model client to the proxy and tag the outbound traffic with a unique session identifier. Here is how you would configure a standard OpenAI-compatible client in Python: import datetime import uuid from openai import OpenAI # Generate a unique session ID for every single run. # Reusing IDs will merge separate runs into an unreadable trail. # Recommended format: {run_id}-{agent_name}-{timestamp}-{unique_tail} session_id = "-".join([ LAB_RUN_ID, # Shared across all agents in a single execution run AGENT_NAME, # Identifies the specific agent datetime.datetime.now(datetime.timezone.utc).strftime("%Y%m%dT%H%M%SZ"), uuid.uuid4().hex[:6] ]) # Initialize the client, routing traffic through the F5 AI Security proxy client = OpenAI( api_key=CALYPSOAI_TOKEN, # Your F5 platform token, not the raw LLM key base_url=CALYPSOAI_OPENAI_API_BASE, # Redirects requests through the proxy default_headers={ # Tags the traffic to this specific agent's audit trail "x-cai-metadata-session-id": session_id, } ) The first two parameters redirect the outbound calls through the security proxy, while the third converts raw LLM traffic into an organized audit trail. By assigning unique session IDs, the platform groups a single agent's calls into a coherent narrative rather than overlapping traffic from different processes. Without the uniqueness of each call/session, it would be impossible to isolate conversations. In a multi-agent system, this setup allows you to trace an entire workflow as it crosses boundaries: each agent generates its own session ID under a shared run prefix, letting you piece together the entire decision path from end to end. With just a few lines of configuration, you get full visibility across your environments, mapping directly to the telemetry shown in our platform dashboards. Work these standards into your governance frameworks, every agent deployed should have observability baked in. Designing multi-layered guardrails Observability gives you eyes, but it doesn't stop an action mid-flight. For enforcement, you need guardrails. Experience with production failures shows that a single defensive layer will eventually crack. You need independent, decoupled boundaries where each layer catches a different class of failure and fails closed by default. A robust agent defense architecture requires five distinct checkpoints: Content Scanning (Proxy Input/Output Scanners): Blocks poisoned or injected prompts before they hit the model. This is your primary defense against completely novel prompt injection techniques. Capability Scoping (Tool Server Manifests): Restricts the agent's environment so it can only see or select the specific tools required for its role. Authorization Scope (OAuth 2.1 / Per-Call Checks): Evaluates short-lived tokens to ensure that even if an agent requests a tool, its current credential actually possesses the authorization to run it. Agent-to-Agent Control (Signed Agent Cards): Governs multi-agent environments by validating whether one agent is permitted to invoke the skills of another, enforced at the receiving end. Underlying Privilege (Downstream Target Systems): The final backstop where the actual resource (database, identity provider, mail relay) enforces strict least-privilege access on the agent's service account. When designing these layers, focus on the difference between reactive and structural defenses. An authorization scope check is reactive: the agent attempts a destructive call, and the platform intercepts and denies it mid-flight. Capability scoping, on the other hand, is structural: the dangerous tool is completely omitted from the agent's available schema definition. Because the model doesn't even know the tool exists, it can't choose it in the first place, completely removing the attack vector. You should implement both, but structural defenses are far more resilient against novel bypasses because they actively shrink what the agent is capable of doing. It's also worth noting that content scanning is the only layer capable of identifying semantic attacks that authorization policies miss. A prompt injection payload hidden inside a customer support ticket or a scraped web page will easily bypass credential checks because, from an infrastructure perspective, reading that data is perfectly valid. Input scanners inspect the text itself before it reaches the LLM to verify what the agent is being asked to do. When a scanner intercepts an attack, the F5 Logs view isolates the specific rule that triggered the block and records the offending payload—giving you a clean, shareable artifact to explain the security decision to external stakeholders without forcing them to dig through raw system telemetry. Figure 2. The Logs view after a guardrail block: the scanner that fired, its verdict, and the content that triggered it — the explanation artifact for an automated decision. Visibility and enforcement must coexist It is common to view observability and guardrails as separate boxes to check on a security review. In practice, they are two sides of the exact same coin. A guardrail without observability requires blind faith; you can't validate that it's firing correctly, you can't tune it when it blocks legitimate business traffic, and you can't explain its decisions to your team. Conversely, observability without guardrails just gives you a pristine, high-resolution recording of a security breach you did nothing to stop. The deployment order matters here. Always deploy observability first. You cannot confidently evaluate or fine-tune a security policy you can't actively watch. Establish your visibility pipelines, verify that you can accurately map the agent's logic alongside its tool executions, and then begin layering your guardrails on top—using your new telemetry trails to validate each security layer as you turn it on. Next Steps If you are currently deploying or managing autonomous agents in production, you can lock down your environment with four immediate steps: Route all LLM traffic through a dedicated proxy to establish an immutable, server-side record of agent reasoning, rather than relying on standard application logs. Enforce unique session tagging across every agent run to keep your audit trails clean and searchable. Instrument both model-facing and tool-facing logs, ensuring you have a clear correlation key between the two. Layer your enforcement mechanisms sequentially: swap out static API keys for short-lived, scoped credentials; apply tool capability scoping; enforce agent-to-agent verification; and place content scanners in front of every ingress point. The most effective way to understand these interactions is to deliberately break them in a safe sandbox environment. Our agent-security-lab recreates the PocketOS database failure against a sandboxed environment, surfaces the agent's reasoning in real-time, and guides you through configuring OAuth 2.1 scoping, MCP capability scoping, agent cards, and F5 AI Guardrails to stop the exploit. The lab is built entirely for practitioners and provides all the required source code. Note that because the exercises interface with the F5 AI Security platform, you will need access to a platform tenant—if you need to set one up for your team, just reach out to your F5 account representatives to get a lab instance provisioned. Agents are the norm now, you are either planning for agentic AI or have already deployed them, its only a matter of time and observability is the blindspot. For autonomous agent architectures, deep observability forms your foundation, and multi-layered guardrails provide the structure to scale safely. References and further reading agent-security-lab (F5GovSolutions) — hands-on multi-agent security lab: github.com/F5GovSolutions/agent-security-lab F5 AI Security platform documentation: docs.aisecurity.f5.com F5 AI Guardrails: f5.com/company/blog/what-are-ai-guardrails F5 AI Explainability / Agentic Fingerprints: f5.com/company/blog/ai-explainability F5 Labs CASI and ARS leaderboards: f5.com/company/labs/casiProtecting Your MCP Server With F5 BIG-IP Advanced WAF
As AI assistants increasingly interface directly to production databases and APIs, building secure, structured interaction layers has become critical. The Model Context Protocol (MCP) standardizes these connections and eliminates the need for custom integrations (See APIs First: Why AI Systems Are Still API Systems). However, with this comes a new class of vulnerabilities that users must protect their service from if they are considering leveraging an MCP server to handle requests to their service. These vulnerabilities have begun to be identified and classified by the OWASP Foundation as OWASP MCP vulnerabilities and they have defined the Top 10 vulnerabilities. In the below demo, we will show the new MCP Policy Template for the BIG-IP Application Security Module and how it can block an attacker from using an AI Agent to execute Command injection against an MCP server. The vulnerabilities that will be demonstrated are as follows: MCP05:2025 - Command Injection & Execution MCP08:2025 - Lack of Audit and Telemetry This article will also serve as a container for more in-depth articles that will demonstrate how to block other OWASP MCP Top 10 vulnerabilities.
Securing AI with F5 AI Guardrails and Nutanix Enterprise AI
Enterprises are rapidly moving from AI experimentation to production-grade deployments, using Kubernetes to deliver and scale AI services consistently across environments. As AI workloads become embedded in business-critical workflows, new risks emerge—particularly at runtime. Generative AI systems can expose sensitive data, generate responses to restricted topics, or be manipulated through prompt injection and jailbreak techniques during user interactions. The challenge is no longer just how to deploy AI models, but how to control AI behavior in real-time—without adding operational complexity or slowing innovation. F5 and Nutanix address this challenge together by combining Nutanix Enterprise AI (NAI) with F5 AI Guardrails. Nutanix Enterprise AI provides a Kubernetes-based platform to deploy and operate AI models and inference services at scale, while F5 AI Guardrails enforces real-time security and policy controls on AI interactions. 🎬 See the Joint Solution in Action ✨ NEW✨🎬 Watch this demo video to see F5 AI Guardrails work seamlessly together with Nutanix Enterprise AI (NAI) Agent Gateway Unified Endpoint (New in NAI v2.7) to secure LLM inference at scale. 🎬 Watch this demo video to see how F5 and Nutanix work together to deliver secure, trustworthy, enterprise‑ready AI—running side by side on Nutanix Kubernetes Platform (NKP) deployed on Nutanix Infrastructure. F5 AI Guardrails Overview F5 AI Guardrails provides runtime security and governance for AI applications, models, and agents by enforcing policy‑based controls on how AI interacts with users and data. It includes a broad set of out‑of‑the‑box guardrails that enable organizations to immediately begin securing and governing AI interactions from day one. These built‑in guardrails address common AI risks such as prompt injection, jailbreaks, PII exposure, restricted topics, and more. In addition to built‑in guardrails, F5 AI Guardrails supports custom guardrails, allowing teams to tailor policy enforcement to specific use cases. One option for defining custom guardrails is through natural‑language policy creation, guided by an integrated F5 AI Assistant. The AI Assistant reviews and refines guardrail definitions using F5 best practices, ensuring that descriptions are clear, concise, and consistently structured for optimal performance. AI Guardrails simplifies AI observability by providing continuous visibility and traceability across AI interactions, with audit‑ready observability, scanning, and logging to support governance and compliance. As part of the F5 Application Delivery and Security Platform (ADSP), AI Guardrails delivers runtime protection for AI models, agents, and connected data, extending F5’s application delivery, security, and observability capabilities across applications and APIs. https://www.f5.com/products/ai-guardrails Nutanix Enterprise AI (NAI) Overview Nutanix Enterprise AI (NAI) is a cloud‑native, Kubernetes‑based AI platform designed to help organizations run AI inference at scale. NAI is supported on CNCF‑certified Kubernetes platforms, including Nutanix Kubernetes Platform (NKP). It enables enterprises to deploy, manage, and operate AI models, inference endpoints, and AI agents through a single, unified management experience, allowing organizations to run AI consistently wherever they operate Kubernetes while maintaining enterprise control, security, and visibility. https://www.nutanix.com/products/nutanix-enterprise-ai F5 and Nutanix: Better Together Nutanix Enterprise AI provides a Kubernetes‑based platform for deploying, managing, and operating AI models, inference endpoints, and AI agents, while F5 AI Guardrails provides runtime security and governance for AI interactions. F5 AI Guardrails can run side by side with Nutanix Enterprise AI, with both solutions deployed on the same Nutanix Kubernetes Platform (NKP)–managed Kubernetes cluster, enabling them to operate on a common Kubernetes foundation. Together, F5 and Nutanix deliver secure, trustworthy, enterprise‑ready AI deployments by combining an enterprise AI platform with runtime AI security—enabling organizations to move fast with AI without losing control over security, compliance, and governance.
HTTP Load Balancer Routes on F5 Distributed Cloud
Route misconfiguration is one of the most common configuration mistakes we see that can cause incidents on F5 Distributed Cloud (F5 XC). The four route types look deceptively simple in the console, but they have distinct behaviors, ordering rules, and gotchas. This article covers all four types with real field names, decision guidance, and the mistakes that actually happen in production. What Routes Do in F5 XC HTTP Load Balancer An HTTP Load Balancer in F5 XC is a full L7 proxy running at the Regional Edge (RE) and depending on the deployment model, Customer Edge (CE). We will use the Regional Edge as a deployment model for this article. When a request arrives, the RE evaluates the route list in order and applies the first matching route. That route determines what happens to the request: forward it to an origin, redirect the client, return a synthetic response, or apply advanced routing logic. Routes can be configured inside the HTTP Load Balancer configuration which opens a new Route Options window: Multi-Cloud App Connect > Load Balancers > HTTP Load Balancers > [your LB] > Routes > Route Options The four route types map to three underlying route actions: XC Route Type Route Action Traffic Goes To Simple Route route Origin Pool Redirect Route redirect Client (3xx response) Direct Response Route direct_response Client (fixed response body) Custom Route route / redirect / direct_response Depends on configuration Route Matching: How XC Evaluates Routes Route evaluation is sequential, stops on first match, and has no automatic specificity ranking. The order you set is the order XC uses. Evaluation Order The HTTP Load Balancer evaluates routes sequentially, top to bottom. The first route that matches the incoming request wins. No further routes are evaluated. This means: More specific routes must appear before broader ones. A catch-all route (prefix /) at the top will swallow everything. Nothing below it will ever match. Path Match Types Three path match types are available across all route types: Match Type Field Behavior Prefix path_prefix Path must begin with the specified string Exact exact Path must equal the value exactly (query string excluded) Regex regex Entire path (minus query string) must match the regex pattern Prefix matching pitfall: The prefix /api matches /api/v1/users but also /apikeys and /api-internal. If you want to match a path segment boundary, use /api/ (trailing slash) or switch to regex. Additional Matching Criteria Beyond path, routes can match on: HTTP methods: GET, POST, PUT, DELETE, etc. Request headers: presence, exact value, regex Query parameters: Retain, Remove, or Replace Combining criteria (e.g., path prefix + method + header) creates an AND condition: all specified criteria must match. Route Type 1: Simple Routes Simple routes are the workhorse of most HTTP Load Balancer configurations. They match a path (and optionally method/headers) and forward traffic to an Origin Pool. When to Use Standard application traffic forwarding Path-based routing to different backend services API versioning (/v1/ → pool A, /v2/ → pool B) Microservice fanout from a single domain Key Configuration Fields Field Description Path match type Prefix / Exact / Regex HTTP Method Any, GET, POST, PUT, etc... Origin Pool The backend pool receiving the request Host Rewrite Method Disable/ Hostname / Header value: rewrites the Host header sent to origin Query Parameters Retain, Remove, Replace Advanced Options Worth Knowing Path rewriting (under Advanced Options): Disabled: path sent to origin unchanged Prefix Replacement: replaces the matched prefix with a new string (e.g., strip /api/v1 prefix before sending to origin) Regex-based: full regex substitution on the path Retry policy: The default retry policy is 1 retry on 5xx responses. Set this explicitly for your application in every route: Disabled: no retries; required for write operations Default: 1 retry on 5xx Custom: specify retry conditions, count, and interval Per-route WAF override: Each simple route can attach its own WAF App Firewall policy. This completely replaces the load balancer-level WAF for matching requests; it is not additive. Use this to enforce stricter rules on sensitive paths (e.g., /admin/) or to relax inspection on certain paths. Example: API Path Routing Route 1: Prefix /api/v2/ → origin-pool-v2 (exact origin for v2) Route 2: Prefix /api/v1/ → origin-pool-v1 (legacy backend) Route 3: Prefix /api/ → origin-pool-api (catch-all for API paths) Route 4: Prefix / → origin-pool-web (catch-all for everything else) Order matters here. If route 3 or 4 appeared first, routes 1 and 2 would never fire. Route Type 2: Redirect Routes Redirect routes return an HTTP 3xx response directly to the client. No origin pool is involved: the RE handles the response entirely. When to Use HTTP → HTTPS redirect (though XC has a dedicated LB-level toggle for this) Domain canonicalization (www.example.com → example.com) Legacy URL migrations (/old-path/ → /new-path/) Temporary redirects during maintenance or A/B migrations Key Configuration Fields Field Description Protocol HTTP or HTTPS Host Target FQDN; supports non-standard ports Redirect Path / URI Target path; if left unset, original URI is preserved (including query string) Response Code 301 (Permanent), 302 (Temporary), 307, 308 Redirect Behavior: URI Preservation When you leave the redirect path unset, XC preserves the original URI path and query string in the Location header. This is useful for protocol/host-only redirects where you just want to change the scheme or domain without touching the path. Example: Redirect all HTTP traffic to HTTPS on the same host: Protocol: HTTPS Host: same-as-request (leave blank or match domain) Path: (unset — preserve original URI) Code: 301 Redirect Route Limitations Simple redirect routes (defined directly on the LB) do not support custom header manipulation on the redirect response. If you need to inject headers (e.g., Cache-Control: no-store on the redirect response), use a Custom Route object instead. Route Type 3: Direct Response Routes Direct response routes return a fully synthetic HTTP response to the client. The request never reaches an origin pool: the RE generates the response itself. When to Use Health check endpoints that should always return 200 (e.g., /healthz) without touching the app Maintenance mode pages: serve a 503 with a message body while origin is down Blocking specific paths with a meaningful error body (vs. a generic deny) Canary or feature-flag placeholders that return 404 before the feature ships Robots.txt or security.txt served from the edge without an origin Key Configuration Fields Field Description HTTP Status Code Any valid HTTP status code (200, 403, 503, etc.) Response Body Static text or HTML body returned to client Path match Same prefix/exact/regex options as other route types Example: Edge-Served Health Check Path: Exact /healthz Method: GET Action: Direct Response Status: 200 Body: OK This responds to health probes from AWS ALB, Kubernetes ingress controllers, or uptime monitors without any load on the backend. Particularly useful during blue/green deployments when the app might not yet be healthy. Route Type 4: Custom Routes Custom routes reference standalone Route objects created separately in XC and attached to one or more HTTP Load Balancers. Unlike the other three types, they follow a service-mesh model rather than a traditional LB model. When to Use Custom Routes Weighted traffic splitting between origin pools (canary releases, blue/green) Request/response header manipulation not available on simple routes Advanced retry policies with specific conditions and intervals Traffic mirroring (shadow traffic to a secondary backend for testing) Reusable route logic shared across multiple load balancers Architecture: Route Objects vs. Inline Routes Inline routes (simple, redirect, direct response) are defined directly on the HTTP Load Balancer. Custom route objects are: Created as standalone objects under Multi-Cloud App Connect Referenced by the HTTP Load Balancer Reusable: multiple LBs can reference the same route object Weighted Clusters Custom routes enable weighted traffic splitting across multiple upstream clusters, equivalent to BIG-IP pool ratio weights or AWS ALB weighted target groups. Route: Prefix /api/ Cluster A (origin-pool-v2): weight 90 Cluster B (origin-pool-v1): weight 10 This is the correct mechanism for canary deployments and gradual traffic shifts on F5 XC. The weights are percentage-based and must sum to 100. Header Manipulation Custom routes support header operations at the route level, applied before forwarding to origin: Operation Direction Example Add header Request X-Forwarded-For: {client-ip} Remove header Request Strip Authorization before certain paths Add header Response Strict-Transport-Security: max-age=31536000 Remove header Response Strip Server header from responses Header manipulation runs in order: route-level → virtual host-level → route configuration-level. Retry Policies Route retry policies take complete precedence over any virtual host-level retry policy. Configure explicitly: Field Description Retry on Conditions: 5xx, gateway-error, reset, connect-failure, retriable-4xx Number of retries Integer Per-try timeout Timeout applied to each individual retry attempt Retry interval Base interval between retries Traffic Mirroring Mirror policies shadow a copy of each request to a secondary cluster. The mirrored request is fire-and-forget. Example: Testing a new backend version against live traffic without affecting users Security analysis pipelines Route Ordering and Priority Route order is the most common source of routing bugs in XC deployments. There is no automatic specificity ranking; you own the order. The Rules Routes evaluate top to bottom. First match wins. Evaluation stops. Disabling a route (via the Route Activation Status toggle) does not remove it: traffic falls through to the next matching route. Recommended Ordering Pattern Order routes from most specific to least specific: Exact paths first Exact /api/v2/auth/token 2. Specific prefixes next Prefix /api/v2/ 3. Broader prefixes after Prefix /api/ 4. Path-specific exceptions Exact /healthz 5. Catch-all last Prefix / Common Ordering Mistakes Mistake Symptom Fix Catch-all prefix / first All traffic hits one origin; other routes never fire Move catch-all to last position /api prefix before /api/v2/ V2 traffic hits wrong origin Reverse the order Disabled/Unused route above active route Traffic silently hits next route with different behavior Remove disabled routes; don't rely on toggle for permanent changes Redirect route below a prefix match Redirect never fires Move redirect above the prefix that would match it first Common Mistakes and Gotchas Prefix /api matches /apikeys. The prefix match does not anchor to path segment boundaries. /api matches /api/, /api/v1/, and also /apikeys, /api-docs. Use /api/ (trailing slash) or regex if segment boundary matters. Per-route WAF is a full replacement, not additive. Attaching a WAF policy to a route does not stack with the LB-level WAF. It replaces it entirely for that route. If your LB WAF is in blocking mode and you attach a route-level WAF in monitoring mode, that route is now in monitoring mode only. Custom routes enforce TLS: test before production. If your origin uses a self-signed certificate and you switch from a simple route to a custom route without uploading the Root CA, connections will fail. Test in a staging environment first. Header manipulation on redirect requires a custom route object. Simple redirect routes in XC do not support response header injection. If you need Cache-Control or Vary headers on your redirects, you must use a standalone custom route object with redirect action. Regex route performance at scale. Regex routes require full path evaluation on every request. At high request volumes, a large number of regex routes adds measurable CPU overhead compared to prefix or exact routes. Use regex only where prefix or exact matching is insufficient. FAQ Q: What is the difference between a Simple Route and a Custom Route? A: Simple routes are inline on the HTTP Load Balancer and forward traffic directly to an Origin Pool. Custom routes are standalone objects that use an Endpoints → Clusters → Routes model, support weighted traffic splits, header manipulation, and mirroring, but cannot reference Origin Pools directly. Q: Why is my catch-all route matching everything instead of the specific routes below it? A: Route evaluation stops at the first match. If your catch-all prefix (/) is above more specific routes, it wins every time. Move the catch-all to the last position in the list. Q: Can I use a Custom Route to send traffic directly to an F5 XC Origin Pool? A: No. Custom routes do not support Origin Pools directly. They use an Endpoints → Clusters → Routes abstraction. If you need weighted splitting with Origin Pool support, custom routes are not the right fit; simple routes forward to Origin Pools but do not support weighted clusters. Q: My POST requests are creating duplicate records and I traced it to XC retries. What is happening? A: The default retry policy on simple routes is "1 retry on 5xx." A POST that hits a 500 gets retried once, potentially double-writing. Set the retry policy to Disabled on any route handling non-idempotent operations (POST, PUT, PATCH, DELETE). Q: Does attaching a WAF policy to a route add rules on top of my LB-level WAF? A: No. Per-route WAF replaces the LB-level WAF entirely for requests matching that route. If your LB WAF is in blocking mode and the route WAF is in monitoring mode, those requests are evaluated in monitoring mode only. Q: My Custom Route TLS connections to origin are failing but the same origin works fine on a Simple Route. Why? A: Custom routes enforce strict TLS with no skip-verify option. Simple routes do not have this requirement. For custom routes, upload the Root CA certificate for your origin, or use the use_volterra_trusted_ca_url flag via the API for public CAs. Self-signed certs without the Root CA uploaded will fail silently. Q: I disabled a route in the console but traffic behavior changed unexpectedly. What happened? A: Disabling a route via the Route Activation toggle does not remove it from evaluation. Traffic falls through to the next matching route in the list. If that route is a broad catch-all, the behavior shift may look correct until something that depends on specific routing breaks. Remove routes you no longer need rather than toggling them off.API Discovery and Enforcement with API Security Local Edition
API Security Local Edition is a self-hosted platform that discovers APIs from BIG-IP traffic insights, builds and maintains an inventory with risk scoring, and pushes enforcement back to BIG-IP. This article covers the architecture, the data flows between components, and the operator workflow from discovery to enforcement.
Automating ACMEv2 Certificate Management on BIG-IP
While we often associate and confuse Let's Encrypt with ACMEv2, the former is ultimately a consumer of the latter. The "Automated Certificate Management Environment" (ACME) protocol describes a system for automating the renewal of PKI certificates. The ACME protocol can be used with public services like Let's Encrypt, but also with internal certificate management services. In this article we explore the more generic support of ACME (version 2) on the F5 BIG-IP.Automatic Certificate Management with ACMEv2 in F5 BIG-IP
One of the most anticipated features of F5 BIG-IP is integration with ACMEv2. With the General Availability of BIG-IP 21.1.0 on May/26, this feature came into being. In this tutorial, we are going to configure it, using Let's Encrypt as the CA. The domain for which we are generating/renewing certificates is carlosf5lab.lat. The official docs for this feature are located in SSL Certificate Management | BIG-IP Documentation. Pre-requisite 1: DNS Resolver that can reach the internet (at least the CA endpoints). In this case, we are using the native DNS Resolver that comes with BIG-IP. Pre-requisite 2: The internal proxy that will make the connection with the CA. Pre-requisite 3: a self signed SSL certificate that the ACMEv2 protocol uses as the identifier for a device account. You don't have to fill the Subject Alternative Name. For the Common Name, an e-mail contact is advised. Now, we are going to create the ACME Provider object. Give it a name, and select the internal proxy previously created. For the CA Certificate to enable the secure connection with the Directory URL, you can use the default ca-bundle.crt. The Directory URL is the endpoint for the ACMEv2 protocol. In Let's Encrypt case, it is https://acme-v02.api.letsencrypt.org/directory For the Account Key, choose the previously created self-signed certificate. For the trickier part of all, the field "Contacts" is mandatory, and it must be an URL. That’s why you must use the format mailto:email_address. Check the Terms and Conditions, and the Create Account boxes. After a while, the Account Status must read as "Valid". To prove you own the domain whose certificate Let's Encrypt is going to create/renew, it must be pointing to an IP (A Record) where you must have your Virtual Server listening on Port 80 configured to respond to the ACMEv2 Challenge. (In this specific lab, the domain carlosf5lab.lat points to a Public IP mapped to an internal IP). Now you can order your first certificate via ACMEv2 on BIG-IP: After a while, the Key tab should read something like: Which means your certificate was generated: To track the ACME Provider, you can check its statistics: That's it, my friend! If it helped you, give a thumbs up to this post!NGINX Gateway Fabric - Data Plane Programmability with NGINX JavaScript
This post walks through a pattern for injecting NGINX JavaScript logic into NGINX Gateway Fabric using Kubernetes-native extension points to enable data plane programmability, with an F5 AI Guardrails integration as a worked example.Forwarding Logs to SIEM Tools via HTTP Proxy for F5 Distributed Cloud Global Log Receiver
Purpose This guide provides a solution for forwarding logs to SIEM tools that support syslog but lack HTTP/HTTPS ingestion capabilities. It covers the deployment and tuning of an HTTP Proxy log receiver configured to work with F5 Distributed Cloud (XC) Global Log Receiver settings. Audience: This guide is intended for technical professionals, including SecOps teams and Solution Architects, who are responsible for integrating SIEM tools with F5 XC Global Log Receiver. Readers should have a solid understanding of HTTP communication (methods, request body, reverse proxy), syslog, and data center network architecture. Familiarity with F5 XC concepts such as namespaces, log types, events, and XC-GLR is also required. Introduction: Problem Statement: SIEM tools often support syslog ingestion but lack HTTP/HTTPS log reception capabilities. Objective: Explain how to deploy and configure an HTTP Proxy to forward logs to F5 Distributed Cloud Global Log Receiver. Solution Overview: Architecture Diagram and workflow: Configuration Steps: Configure Global Log Receiver in F5 Distributed Cloud Console Navigate to: Home → Shared Configuration → Global Log Receiver Create or edit the Global Log Receiver settings for HTTP receiver Ensure the Global Log Receiver batch size is based on the payload size expected from F5 NGINX. Example configuration snap: Set Up NGINX as an HTTPs Log Receiver Install NGINX on your designated server. Configure log_format Configure NGINX to accept HTTP POST requests only and forward access logs to syslog Example configuration snippet: log_format custom_log_format_1 escape=json $request_body; # Example: include request body only server { listen 443 ssl; server_name <logreceiver_server_name>; ssl_certificate /etc/ssl/<logreceiver_server_cert>; ssl_certificate_key /etc/ssl/<logreceiver_server_key>; # Other SSL/TLS configurations (e.g., protocols, ciphers) ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers HIGH:!aNULL:!MD5; client_body_in_single_buffer on; # The directive is recommended when using the $request_body variable, to save the number of copy operations involved client_body_in_file_only off; #default client_max_body_size 32M; # based on tuning gzip on; location /log_endpoint { # Allow only POST requests for sending log data limit_except POST { deny all; } # Configure access_log to write incoming data to a file # access_log /var/log/nginx/log_receiver.log custom_log_format_1; access_log syslog:server=127.0.0.1:514,facility=local7,tag=nginx,severity=info custom_log_format_1; proxy_pass http://localhost:8091/; # This dummy Internal server required to collect request_body variable. } } # dummy internal server to respond back 200 ok server { listen 8091; server_name localhost; location / { return 200 "Log received successfully."; } } Set Up rsyslog server Install/configure rsyslog on your designated server. Configure 60-nginx.conf file in /etc/rsyslog.d/ directory Sample 60-nginx.conf file #nginx.* @@127.0.0.1:514 :syslogtag, isequal, "[nginx]" /var/log/nginx-syslog/nginx-access-log.log OR Set Up BIG-IP as an HTTPs Log Receiver Syslog pool ltm pool glr_syslog_pool { members { <syslog-servr-ip>:514 { address <syslog-server-ip-address> session monitor-enabled state up } } monitor tcp } iRule for hsl logging when CLIENT_ACCEPTED priority 500 { set hsl_handle "" set request_body "" } when HTTP_REQUEST priority 500 { if { [HTTP::method] eq "POST" } { HTTP::collect [HTTP::header Content-Length] } else { HTTP::respond 200 content "data received" } } when HTTP_REQUEST_DATA priority 500 { set request_body [HTTP::payload] if { ! [info exists hsl_handle] || $hsl_handle eq "" } { set hsl_handle [HSL::open -proto UDP -pool "/Common/glr_syslog_pool"] } HSL::send $hsl_handle "<134> XC_LOG: $request_body" HTTP::respond 200 content "data received" HTTP::release } Virtual Server configuration ltm virtual vs-3 { destination <vs-ip>:443 ip-protocol tcp mask 255.255.255.255 profiles { demo-ent-merja-in { context clientside } http { } tcp { } } rules { <iRule-created-in-earlier-step> } serverssl-use-sni disabled source 0.0.0.0/0 source-address-translation { type automap } translate-address enabled translate-port enabled } rsyslog server config inputs #### The following modules are enabled in rsyslogd.conf module(load="omfile") module(load="imtcp") input(type="imtcp" port="514") ### /etc/rsyslog.d/70-bigip.conf template(name="F5_Format" type="string" string="%TIMESTAMP% %msg%\n") if ($msg contains "XC_LOG:") then { action( type="omfile" File="/var/log/10.1.20.177/f5_hsl_custom.log" Template="F5_Format" ) stop } References: F5 Distributed Cloud Global log receiver supports many log receivers natively: F5 Distributed Cloud Technical Knowledge page on "Configure Global Log receiver" Prerequisites: An external log collection system reachable publicly. The following IP address ranges are required to be added to your firewall's allowlist: 193.16.236.64/29 185.160.8.152/29What’s New in Zero Trust Access v10?
Introduction F5 BIG-IP Zero Trust Access, a key component of the F5 Application Delivery and Security Platform (ADSP), helps teams secure apps that are spread across hybrid, multi-cloud and AI environments. In this article, I’ll highlight some of the key Access features available in F5 BIG-IP v21.1. F5 BIG-IP v21.1 was released in May of 2026. This release included an updated Zero Trust Access version (v10) with new features. Demo Video New Features in Zero Trust Access IPsec VPN Support Added support for Access IPsec VPN Tunnels, to meet global security standards and enable the transition from SSL/TLS-VPNs to IPsec VPNs. Clients can now connect to BIG-IP using the Windows Edge Client or F5 Access for macOS, establish an IPsec tunnel, and securely access the backend network. To enable IPsec support, navigate to Access > Connectivity / VPN > Connectivity > Profiles Click on a VPN Profile, “VPN-Profile1” in this example Scroll down and click Edit Profile Change the VPN Profile Type to IPsec and click OK When you set the VPN Profile Type to IPsec, the system automatically generates an Access IPsec Policy. HTTP Connector Support Added to Per-Session Policies in APM Support for the HTTP Connector in per-session policies is now available in F5 BIG-IP Access Policy Manager (APM). This feature enables administrators to send HTTP requests to external services during session establishment and use the response for authentication, authorization, and access control decisions. To use the HTTP Connector, navigate to Access > Profiles / Policies > Access Profiles Edit the Per-Session Policy of any Profile you wish to add an HTTP Connector to, “Test“ in this example Click the plus to add an item Go to the General Purpose tab, select HTTP Connector and click Add Item Select the HTTP Connector Profile then click Save Dynamic Client Registration (DCR) support This release adds support for OAuth 2.0 Dynamic Client Registration (RFC 7591). Administrators can enable DCR on OAuth profiles to allow authorized clients to dynamically register using an Initial Access Token (IAT). The feature includes support for the Client Credentials grant type, configurable client authentication settings, client secret expiration, and enhanced logging. To enable DCR, navigate to Federation > OAuth Authorization Server > OAuth Profile Click on the name of the profile you want to edit, oauth in this example Check the box to enable Dynamic Client Registration Click Update at the bottom Custom Logging Preferences for Windows Edge Client The Windows Edge Client now offers custom logging preferences, giving you enhanced control over log verbosity to improve both security and flexibility. You can select the required log level from the APM Client Log Level drop-down in General Settings while creating a Connectivity Profile. To change the Logging Preferences, navigate to Access > Connectivity / VPN > Connectivity > Profiles Click on a VPN Profile, “VPN-Profile1” in this example Scroll down and click Edit Profile Change the APM Clients Log Level and click OK Native Support for SAML Authentication for Windows APM clients now support native SAML authentication, significantly improving user experience, maintainability, and overall supportability. Edge Client on macOS and Windows can leverage the system’s default browser to authenticate users with identity providers (IdPs), enabling modern authentication mechanisms such as FIDO2 and Microsoft Entra ID device authentication. To enable this feature, navigate to Access > Connectivity / VPN > Connectivity > Profiles Click on a VPN Profile, “VPN-Profile1” in this example Scroll down and click Edit Profile Select the Desktop Client Settings, check the box to Enable System Browser and click OK Auto-Upgrade Machine Tunnel Service Windows Edge Clients can now automatically upgrade the F5 Machine Tunnel Service when a newer version is available on BIG-IP, and the auto-upgrade feature is enabled. Additionally, if the Machine Tunnel service is running before the upgrade, it continues to run after the upgrade completes without affecting existing VPN configuration settings. Endpoint Inspection Support on Ubuntu with ARM64 Endpoint Inspection is now supported on Ubuntu with ARM64, allowing seamless management and inspection of endpoints on Linux ARM64 platforms. Conclusion F5 BIG-IP Zero Trust Access, a key component of the F5 Application Delivery and Security Platform (ADSP), helps teams secure apps that are spread across hybrid, multi-cloud and AI environments. The latest version of F5 BIG-IP is packed with new Zero Trust Access features. Related Content F5 BIG-IP Zero Trust Access Zero Trust Solution Overview Secure Corporate Apps with a Zero Trust Security Model F5 BIG-IP APM Identity Aware Proxy (IAP): The Gateway to a Zero Trust Architecture Zero Trust Application Access for Federal Agencies BIG-IP APM Configuration for Compliance Retrieval Service
