How to Optimize AI Inference with F5 NGINX Gateway Fabric
If you’re managing Kubernetes clusters right now, you already know the drill: standard Layer 7 load balancing works flawlessly for web APIs where requests resolve in milliseconds. But the moment you start hosting Large Language Models (LLMs), that traditional routing logic falls apart. AI inference workloads are a completely different beast. You have to account for GPU memory, active inference queues, and KV-caches. If you rely on basic proxying, you usually end up with incredibly expensive GPUs tied up handling lightweight tasks, while your developers are forced to write custom middleware just to orchestrate traffic. We needed a way to bring intent-driven networking directly to the AI edge. That’s exactly what the Gateway API Inference Extension does. By pairing this with F5 NGINX Gateway FabricWe can transform a standard Kubernetes Gateway into a dedicated Inference Gateway. Let's look at how this changes the game for platform teams. The Two-Stage Architecture To make intelligent routing decisions, we use a two-stage architecture that separates high-level routing intent from real-time endpoint selection. Stage 1 (Intent): We use standard Kubernetes HTTPRoutes to define exactly where traffic should go based on paths, headers, or weights. Stage 2 (Real-Time Selection): Instead of routing blindly to backend pods, we target an InferencePool CRD. This pool uses an Endpoint Picker to evaluate real-time node telemetry (like queue depth) and picks the absolute best pod for the job. To prove this is running under the hood, we can describe our GPU and CPU InferencePools. Notice how each pool has a dedicated Endpoint Picker attached and ready to route traffic based on real-time node health. GPU Pool Endpoint Picker: kubectl describe inferencepool ollama-inferencepool -n ollama | grep -A 10 "Endpoint Picker" Endpoint Picker Ref: Failure Mode: FailClose Group: Kind: Service Name: ollama-inferencepool-epp Port: Number: 9002 Selector: Match Labels: App: ollama --> GPU Target Ports: CPU Pool Endpoint Picker: kubectl describe inferencepool ollama-cpu-pool -n ollama | grep -A 10 "Endpoint Picker" Endpoint Picker Ref: Failure Mode: FailOpen Group: Kind: Service Name: ollama-cpu-pool-epp Port: Number: 9002 Selector: Match Labels: App: ollama-cpu --> CPU Target Ports: This separation of routing intent from real-time endpoint selection allows platform engineers to solve three critical AI infrastructure challenges without requiring developers to write custom middleware. A quick note on resilience: Notice the Failure Mode in those outputs? This defines what happens if the Endpoint Picker itself goes offline. For our expensive GPU pool, we set it to FailClose (rejecting traffic so we don't overwhelm premium hardware blindly). For our CPU efficiency tier, we set it to FailOpen (falling back to standard round-robin load balancing to keep the application alive). Let’s look at three practical ways to use this setup to optimize your hardware. 1. Model-Aware Routing: Stop Using GPUs for Everything Treating all AI traffic equally is the fastest way to exhaust a hardware budget. High-performance GPUs should be strictly reserved for complex generation tasks, while efficiency tiers (such as CPUs) should handle lightweight tasks, such as audio transcription or basic summarization. apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: model-aware-httproute namespace: ollama spec: parentRefs: - name: inference-gateway namespace: nginx-gateway sectionName: http rules: # --- UI Preservation Rules --- - matches: - path: { type: PathPrefix, value: /ui } filters: - type: URLRewrite urlRewrite: path: { type: ReplacePrefixMatch, replacePrefixMatch: / } backendRefs: - name: chatbot port: 8501 - matches: - path: { type: PathPrefix, value: /static } - path: { type: Exact, value: /favicon.png } backendRefs: - name: chatbot port: 8501 # --- AI Routing Rules --- - matches: - path: { type: PathPrefix, value: /v1/audio } backendRefs: - group: inference.networking.k8s.io kind: InferencePool name: ollama-cpu-pool. # CPU Pool Fallback port: 11434 - matches: - path: { type: PathPrefix, value: /v1/chat } backendRefs: - group: inference.networking.k8s.io kind: InferencePool name: ollama-inferencepool. # GPU Pool Fallback port: 11434 Using path-based routing, the Gateway acts as an intelligent traffic cop. In the configuration below, we map the /v1/audio path directly to a CPU InferencePool. If a request hits this endpoint, the Gateway seamlessly offloads it to the efficiency tier, protecting our premium GPUs from trivial workloads. 2. Canary Deployments In MLOps, rolling out a new LLM is inherently risky. Performance regressions, latency spikes, and hallucinations are real threats to the user experience. You cannot simply cut over all production traffic to a new model overnight. Native traffic splitting provides a safety net for model validation. By configuring a deterministic weight at the Gateway level, 90% of users continue to be served by the stable production GPU pool, while 10% of traffic is routed to the efficiency tier for real-time validation. apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: canary-httproute namespace: ollama spec: parentRefs: - name: inference-gateway namespace: nginx-gateway sectionName: http rules: # --- UI Preservation Rules --- - matches: - path: { type: PathPrefix, value: /ui } filters: - type: URLRewrite urlRewrite: path: { type: ReplacePrefixMatch, replacePrefixMatch: / } backendRefs: - name: chatbot port: 8501 - matches: - path: { type: PathPrefix, value: /static } - path: { type: Exact, value: /favicon.png } backendRefs: - name: chatbot port: 8501 # --- Use Case 2: 90/10 Canary Split --- - matches: - path: { type: PathPrefix, value: /v1/chat } backendRefs: - group: inference.networking.k8s.io kind: InferencePool name: ollama-inferencepool. # GPU Pool Fallback weight: 90 port: 11434 - group: inference.networking.k8s.io kind: InferencePool name: ollama-cpu-pool. # CPU Pool Fallback weight: 10 port: 11434 Because this is handled purely via declarative YAML, platform teams can execute risk-free canary tests without altering any application code. 3. Cost Optimization (Header-Based): When an AI cluster is under maximum pressure, standard load balancers process requests on a first-in, first-out basis. In an enterprise environment, this is unacceptable. Mission-critical workflows and premium users must have guaranteed access to the best compute resources. By utilizing custom HTTP headers, client applications can signal their importance to the Gateway. The NGINX semantic engine reads these headers in real-time. If the x-query-complexity: high header is present, the request is immediately fast-tracked to the premium GPU pool. Every other request falls back to the CPU tier. apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: header-based-httproute namespace: ollama spec: parentRefs: - name: inference-gateway namespace: nginx-gateway sectionName: http rules: # --- UI Preservation Rules --- - matches: - path: { type: PathPrefix, value: /ui } filters: - type: URLRewrite urlRewrite: path: { type: ReplacePrefixMatch, replacePrefixMatch: / } backendRefs: - name: chatbot port: 8501 - matches: - path: { type: PathPrefix, value: /static } - path: { type: Exact, value: /favicon.png } backendRefs: - name: chatbot port: 8501 # --- Use Case 3: Priority Steering --- - matches: - headers: - type: Exact name: x-query-complexity value: high path: { type: PathPrefix, value: /v1/chat } backendRefs: - group: inference.networking.k8s.io kind: InferencePool name: ollama-inferencepool # GPU Pool port: 11434 - matches: - path: { type: PathPrefix, value: /v1/chat } backendRefs: - group: inference.networking.k8s.io kind: InferencePool name: ollama-cpu-pool # CPU Pool Fallback port: 11434 This enforces strict SLAs regardless of overall cluster load, so your most valuable transactions stay running. See the Architecture in Action Managing complex AI traffic across heterogeneous hardware doesn't have to be a headache. By utilizing F5 NGINX Gateway Fabric, you can optimize compute, validate models safely, and prioritize critical traffic—all through declarative, intent-driven configurations. To see exactly how these routing rules are executed, check out my full technical walk-through below: Resources : [NGINX Community Blog] Read the official announcement and dive deeper into how NGF supports this new extension. [NGINX Gateway Fabric Documentation] The official documentation for deploying and configuring NGINX Gateway Fabric in your Kubernetes environment. [NGINX Gateway Fabric (GitHub)] Explore the upstream development, CRD definitions, and architecture of the Inference Extension project.BIG-IP Cloud-Native Network Functions 2.3: What’s New in CNF and BNK
Introduction F5 BIG-IP continues to advance BIG-IP Next for Kubernetes (BNK) and Cloud-Native Network Functions (CNFs) to meet the growing demands of service providers and modern application environments. F5 provides the full stack required to make cloud-native networking work in a service provider environment. CNFs alone are not enough; you need functions, control, infrastructure, and observability, working together as one system. What is new in BIG-IP Cloud-Native Edition 2.3 for BNK and CNF? Release 2.3 adds MPLS provider edge support (early access), native UDP/TCP load balancing, DPU-accelerated data plane offload on NVIDIA BlueField, and subscriber-aware Policy Enforcement Manager (PEM) with Gx interface integration. It also introduces VRF-aware AFM policies, GSLB with Sync Groups for multi-region deployments, BBRv2 congestion control, and crash diagnostics that operate without host-level Kubernetes access. This release targets service providers and telecom operators who need cloud-native networking without sacrificing the protocol support and policy control of traditional infrastructure. Cloud-Native Network Functions (CNFs): What Changed in 2.3? In release 2.3, CNF capabilities focus on strengthening the underlying network functions required for service provider deployments. How does BIG-IP CNF 2.3 handle crash diagnostics in restricted Kubernetes clusters? Operating CNFs in production environments requires strong observability, even in restricted clusters. Release 2.3 introduces improvements to the crash agent that allow core files to be collected directly from pods without requiring host-level access. This enables deployments in more secure Kubernetes environments and simplifies troubleshooting when issues occur. How does BIG-IP CNF 2.3 operate in multi-tenant and multi-VRF environments? Multi-tenant environments demand precise control over traffic behavior. Release 2.3, Advanced Firewall Manager (AFM), introduces VRF-aware ACL and NAT policies, allowing operators to apply firewall and translation rules within specific routing contexts. This enables better segmentation and supports overlapping address spaces while maintaining consistent policy enforcement. It aligns CNFs more closely with how service provider networks are designed and operated. Can BIG-IP CNF 2.3 operate at edge? One of the most significant additions to this release is MPLS support within CNFs. This is currently an early-access feature and is expected to reach general availability in a future release. CNFs can now operate as provider edge nodes, supporting label-based forwarding and applying policies based on MPLS labels. This allows service providers to extend existing MPLS architectures into Kubernetes environments without requiring major redesigns. It also provides a path for replacing legacy systems with cloud-native alternatives while maintaining familiar networking constructs. UDP and TCP Application Load Balancing Release 2.3 introduces UDP and TCP application load balancing, expanding support beyond HTTP-based traffic management. This capability enables CNFs to handle a broader range of applications and telco protocols, including workloads that rely on Layer 4 traffic patterns. Traffic can be balanced across services both inside and outside the Kubernetes cluster, which is critical for hybrid deployments and incremental modernization efforts. This enhancement is especially important for service providers and large enterprises that operate in mixed environments. They allow existing applications to continue functioning while new cloud-native components are introduced, without requiring immediate architectural changes. Subscriber-Aware Policy Enforcement Subscriber awareness remains a core requirement for service providers. Release 2.3 enhances Policy Enforcement Manager (PEM) with GX interface integration, enabling real-time policy enforcement based on subscriber data. This allows traffic to be classified and controlled dynamically, supporting use cases such as QoS enforcement, traffic shaping, and content filtering. It also enables compliance with regulatory requirements and opens new opportunities for service differentiation. Improved Observability and Aggregated Insights As CNFs scale, visibility becomes more complex. Earlier approaches relied on per-pod metrics, which made it difficult to build a unified view of the system. Release 2.3 enhances PEM by introducing aggregation through TODA, allowing statistics and session data to be collected and presented as a single entity. Enhancements to MRFDB and PEM reporting further improve visibility into subscriber sessions and traffic behavior, giving operators a more complete and centralized view of network activity. Building on this foundation, release 2.3 expands PEM capabilities with subscriber-aware policy enforcement. By integrating external policy systems and classification services, CNFs can now correlate traffic with subscriber identity and apply policies dynamically. This provides deeper insight into how individual subscribers and applications behave on the network, enabling more precise control and improved operational awareness. Additional DNS visibility enhancements, such as adding dig support into netkvest, further strengthen troubleshooting capabilities. By enabling more detailed DNS query inspection and response analysis, operators can quickly diagnose resolution issues and better understand traffic patterns tied to application behavior. Together, these enhancements move CNFs beyond basic monitoring. They provide a richer, more contextual understanding of traffic, subscribers, and services, which simplifies operations and enables faster troubleshooting in large-scale environments. DNS and Traffic Behavior Enhancements Release 2.3 includes improvements that address real-world network behavior, particularly in how DNS and transport protocols operate at scale. One example is the handling of DNS requests during certain scenarios. Instead of silently dropping traffic, CNFs can now return NXDOMAIN responses, preventing upstream systems from interpreting the lack of response as a service failure. This improves reliability and ensures better interoperability with external DNS resolvers in distributed environments. In addition, support for BBRv2 congestion control improves TCP performance in challenging conditions. It provides better fairness across flows and adapts more effectively to latency and packet loss, improving overall user experience in mobile and distributed networks. Extending to Multi-Region Traffic Management Release 2.3 continues to expand DNS capabilities with early access support for Global Server Load Balancing, enabling traffic distribution across multiple locations such as data centers and cloud environments. This represents an important step toward multi-region and hybrid architectures, where applications are no longer tied to a single cluster or deployment location. Building on this, the introduction of GSLB Sync Groups improves how configurations are managed across distributed deployments. Within a sync group, one instance is designated as the sync agent and is responsible for propagating configuration changes to other members. This approach ensures consistency across environments while preventing conflicting updates and reducing the risk of synchronization issues. Release 2.3 also begins to introduce more intelligent traffic steering with topology-based load balancing. This capability allows traffic to be directed based on user-defined parameters such as location or network proximity. As a result, operators can optimize application delivery by sending users to the most appropriate endpoint, improving latency and overall service quality. Together, these enhancements move CNFs closer to providing a fully cloud-native, globally distributed traffic management solution that aligns with modern application deployment patterns. BNK on NVIDIA BlueField DPU: What Performance Does Hardware Offload Deliver? As Kubernetes environments scale, the limitations of CPU-based packet processing become more visible. Networking workloads compete directly with applications for resources, which can impact both performance and cost. Release 2.3 continues the expansion of BNK on NVIDIA BlueField DPUs, allowing key data plane functions to be offloaded from the host CPU. This change improves throughput and reduces latency while freeing compute resources for applications that generate business value. The benefit is not just raw performance. It also brings predictability. With networking and security processing handled on the DPU, operators can achieve more consistent performance across distributed environments. This is especially important for AI infrastructure and high-throughput telco deployments, where even small inefficiencies can scale into significant costs. From an operational perspective, this also simplifies infrastructure design. Separating application workloads from networking functions reduces contention and allows for more efficient scaling strategies. CNFs begin to behave less like shared software components and more like purpose-built networking systems, while still retaining the flexibility of Kubernetes. BNK for Telco and Modern Applications Modern environments rarely consist of purely cloud-native applications. Most organizations are running a mix of legacy protocols, telco workloads, and newer microservices. Release 2.3 addresses this reality directly. Can BNK 2.3 load balance non-HTTP protocols? One of the most important additions to this release is TCP and UDP load balancing. This extends BNK beyond HTTP-based traffic management and enables support for telco protocols and other non-HTTP workloads. It also allows traffic to be balanced both inside and outside the Kubernetes cluster, which is critical for hybrid architectures and phased migrations This capability reflects a broader shift in BNK. It is no longer just an ingress layer. It is evolving into a unified traffic management platform that can handle diverse protocols and application types without forcing architectural changes. For service providers, this means they can modernize incrementally. Existing applications can continue to operate while new components are introduced in Kubernetes. For enterprise environments, it provides a consistent way to manage traffic across distributed services without introducing additional tools or complexity. Frequently asked questions These questions represent the most common queries architects and operators ask when evaluating BIG-IP Cloud-Native Edition 2.3. Q: What is new in BIG-IP Cloud-Native Edition 2.3? A: BIG-IP CNF 2.3 adds MPLS provider edge support (early access), native UDP/TCP load balancing, DPU-accelerated data plane offload on NVIDIA BlueField, subscriber-aware PEM with Gx integration, VRF-aware AFM policies, GSLB with Sync Groups, congestion control, and crash diagnostics that operate without host-level Kubernetes access. Q: What CPU savings does BNK on NVIDIA BlueField DPU deliver? A: Validated testing (Tolly Report #226104, February 2026) showed approximately 80% host CPU reduction, 40% more output tokens per second versus HAProxy on Llama 70B, and 61% faster time to first token (TTFT). These results reflect BNK offloading data plane processing from the host CPU to the BlueField DPU, freeing the host compute for application workloads. Q: Does BIG-IP CNF 2.3 support protocols beyond HTTP and HTTPS? A: Yes. Release 2.3 adds native UDP and TCP load balancing to both CNFs and BNK, extending traffic management beyond HTTP. This supports telco protocols such as GTP-U, Diameter, and RADIUS, with the ability to balance traffic across services inside and outside the Kubernetes cluster.Automating F5 Application Delivery and Security Platform Deployments
The F5 ADSP Architecture Automation Project The F5 ADSP reduces the complexity of modern applications by integrating operations, traffic management, performance optimization, and security controls into a single platform with multiple deployment options. This series outlines practical steps anyone can take to put these ideas into practice using the F5 ADSP Architectures GitHub repo. Each article highlights different deployment examples, which can be run locally or integrated into CI/CD pipelines following DevSecOps practices. The repository is community-supported and provides reference code that can be used for demos, workshops, or as a stepping stone for your own F5 ADSP deployments. If you find any bugs or have any enhancement requests, open an issue, or better yet, contribute. The F5 Application Delivery and Security Platform (F5 ADSP) The F5 ADSP addresses four core areas: how you operate day to day, how you deploy at scale, how you secure against evolving threats, and how you deliver reliably across environments. Each comes with its own challenges, but together they define the foundation for keeping systems fast, stable, and safe. Each architecture deployment example is designed to cover at least two of the four core areas: xOps, Deployment, Delivery, and Security. This ensures the examples demonstrate how multiple components of the platform work together in practice. DevSecOps: Integrating security into the software delivery lifecycle is a necessary part of building and maintaining secure applications. This project incorporates DevSecOps practices by using supported APIs and tooling, with each use case including a GitHub repository containing IaC code, CI/CD integration examples, and telemetry options. Demo: Use-Case 1: F5 Distributed Cloud WAF and BIG-IP Advanced WAF Resources: F5 Application Delivery and Security Platform GitHub Repo and Automation Guide ADSP Architecture Article Series: Automating F5 Application Delivery and Security Platform Deployments (Intro) F5 Hybrid Security Architectures (Part 1 - F5's Distributed Cloud WAF and BIG-IP Advanced WAF) F5 Hybrid Security Architectures (Part 2 - F5's Distributed Cloud WAF and NGINX App Protect WAF) F5 Hybrid Security Architectures (Part 3 - F5 XC API Protection and NGINX Ingress Controller) F5 Hybrid Security Architectures (Part 4 - F5 XC BOT and DDoS Defense and BIG-IP Advanced WAF) F5 Hybrid Security Architectures (Part 5 - F5 XC, BIG-IP APM, CIS, and NGINX Ingress Controller) Minimizing Security Complexity: Managing Distributed WAF Policies
You Don't Have to Have Played to Understand the Game
Andy Reid barely played football. He was a community college tackle who transferred to BYU and then rode the bench for most of his time in Provo. Teammates remember him as the guy in the film room, not the guy on the field. He spent his Saturdays watching, taking notes, and pestering head coach LaVell Edwards with so many questions about strategy that Edwards eventually told him: Kid, you should coach. That’s the origin story of three Super Bowl wins for my local Kansas City Chiefs, six appearances across the Chiefs and Eagles, and one of the winningest coaches in NFL history. Not a stud player who worked his way down to the sideline, but a guy who asked a lot of questions, kept asking them, and turned that into a career. Richard Williams had never picked up a tennis racket in his life when he saw Virginia Ruzici win a tournament on TV in 1978 and decided his daughters were going to be world champions. He taught himself the sport from books and instructional videos and then wrote and implemented a 78-page plan for coaching Venus and Serena on the public courts in Compton when they were very young. Thirty Grand Slam singles titles between them later and the “you have to have played at the highest level to coach at the highest level” theory was looking pretty thin. Nobody looks at Reid's three Super Bowl rings and says “yeah, but did you really understand it without playing in the league?” Nobody tells Richard Williams his daughters’ Grand Slam titles have an asterisk because he learned the game from a VHS tape. We accept, in sports, that there’s more than one way to know a thing. Somehow that grace evaporates the second AI enters the conversation. Doing isn't understanding There’s a flavor of pushback on AI use that goes something like: “you have to do it manually first to really understand it.” Sometimes that’s gatekeeping in a wise-elder’s costume. Sometimes it’s a genuine concern. An experienced person who built their intuition the hard way, watching newer folks skip the grind, and worrying (not unreasonably) that the intuition won’t form. But “doing it manually” and “understanding it” aren’t the same thing. They overlap, but they’re not the same thing. You can grind through a problem manually for years and still not understand the system around it. And you can understand a system deeply without having implemented every piece of it yourself, if you’re willing to ask enough questions. The questioning is the work Here’s the part I think people miss when they’re worried about AI making us dumber: A lot of what an expert does for you, when you’re lucky enough to have one, is answer questions patiently. Over and over. Sometimes the same question is phrased three different ways because you didn’t quite get it the first time. Sometimes a dumb question that you’d be embarrassed to ask on a Slack channel. Good mentors don’t get tired of this. But there are very few good mentors. They’re busy, and you only get so many of them in a career. I’ve been at this for thirty years now and I can count the great mentors I’ve had on one hand. LLMs don’t get tired. They don’t sigh. They don’t make you feel stupid for asking why something works the way it does for the fourth time. And the act of formulating the question, asking "what exactly am I confused about?" and "what do I need to know to clear the fog?” That’s a huge chunk of where understanding actually comes from. The model is a sparring partner for your own thinking, if you let it be one. Use it as a vending machine and you’ll get exactly that: answers, not understanding. The tragic version of LLM use is the one where someone pastes the problem, takes the answer, ships it, and walks away no smarter than they started. Then does it again the next day. And the next. Building a career out of outputs they couldn’t reproduce or defend if you took the tool away. That's the version the skeptics are right about. It just isn’t the only version. Andy Reid didn’t need to have been a pro-bowl tackle to understand offensive football. He needed to watch carefully, ask the right questions, and think rigorously about what he was seeing. Richard Williams didn’t need to have been on tour. He needed books, tapes, and the willingness to do the homework. Playing at the highest level is one path to understanding. But for systemic thinking, tactical thinking, architectural thinking, it might not always be the best one. Two things I learned this week First: I’m working on a side project where the FastAPI Cloud backend runs as a two-instance replica deployment. I started on SQLite, which worked fine until I realized writes were landing in whichever instance happened to handle the request, leaving me with two file-based databases with immediate data drift. I moved to a serverless Postgres database (Neon) to give both instances a single source of truth, and once I was there, realized I could just point dev and prod at the same data. Yes, in a real production system this is an anti-pattern and I’d never recommend it. But for a small project where I’m iterating fast and the bottleneck is my own understanding of the problem, not having to migrate data back and forth every time I want to test a frontend change or hunt down a bug? Game changer. I got there by talking the tradeoffs through with my good friend Claude. What breaks, when it breaks, what the actual risk surface looks like at my scale. Nobody handed me a "here's when to break the rule" tutorial. I asked questions until I understood the rule well enough to break it on purpose. Second: I'm building an on-box tool for BIG-IP (article coming soon), and I hit the HA problem. How do I keep state synced across boxes? My first instinct was file-based storage on the host, which, it turns out, is exactly where AS3 and SSL Orchestrator started. SSLO went a step further and built a dedicated sync layer called gossip to keep those files coordinated across the cluster. Over time, both products converged on a different approach: data-groups for metadata and iFiles for larger payloads, both of which ride along with standard config sync. That's a much smaller surface area to maintain, and it leans on infrastructure the platform already guarantees. So I'm following the same path: metadata in data-groups, data blobs in iFiles. I figured this out by interrogating Claude about how those products were architected, why they made the choices they did, and what the failure modes were. I could have read the source, and I could have tried to track down the developers and architects (and I should have over dinner to get the inside scoop). But the speed of “ask, get an answer, ask the next question, get an answer” let me sketch the whole design space in an afternoon. That's not skipping the understanding. That’s building it. Get off whose lawn? I get the resistance. Some of it is "get off my lawn." Some of it is genuine expertise feeling devalued. Some of it is real fear about what this technology means for the people who come up behind us. None of those concerns are stupid. The people who built their understanding the hard way, by tinkering, by breaking things, by reading source code under duress at 2am because there was no other way to get the system back online? They are not wrong about the value of that path. They earned something real in all that trial by fire. Some of them are the best engineers I know. The intuition that comes from years of manual struggle is a kind of literacy that doesn’t have a shortcut, and the people who have it are the ones I most want in the room when something goes sideways. But I’d push back on the specific claim that you must do every step manually to understand the thing. You don’t. Engage with it seriously. Ask real questions and chase the answers until they hold together. Be willing to be wrong. Notice when you’re wrong, and update accordingly. Used well, an LLM doesn’t dull that loop. It tightens it. The design decision, the tradeoff, the bet, I’m getting to that part of the problem sooner than I would have otherwise. Reid had Edwards. Williams had the library. The skeptics aren’t wrong that some understanding only comes from doing. They’re wrong that this is one of them.VMware VKS integration with F5 BIG-IP and CIS
Introduction vSphere Kubernetes Service (VKS) is the Kubernetes runtime built directly into VMware Cloud Foundation (VCF). With CNCF certified Kubernetes, VKS enables platform engineers to deploy and manage Kubernetes clusters while leveraging a comprehensive set of cloud services in VCF. Cloud admins benefit from the support for N-2 Kubernetes versions, enterprise-grade security, and simplified lifecycle management for modern apps adoption. Alike with other Kubernetes platforms, the integration with BIG-IP is done through the use of the Container Ingress Services (CIS) component, which is hosted in the Kubernetes platform and allows to configure the BIG-IP using the Kubernetes API. Under the hood, it uses the F5 AS3 declarative API. Note from the picture that BIG-IP integration with VKS is not limited to BIG-IP´s load balancing capabilities and that most BIG-IP features can be configured using this integration. These features include: Advanced TLS encryption, including safe key storage with Hardware Security Module (HSM) or Network & Cloud HSM support. Advanced WAF, L7 bot and API protection. L3-L4 High-performance firewall with IPS for protocol conformance. Behavioral DDoS protection with cloud scrubbing support. Visibility into TLS traffic for inspection with 3 rd party solutions. Identity-aware ingress with Federated SSO and integration with leading MFAs. AI inference and agentic support thanks to JSON and MCP protocol support. Planning the deployment of CIS for VMware VKS The installation of CIS in VMware VKS is performed through the standard Helm charts facility. The platform owner needs to determine beforehand: Whether the deployment is hosted on a vSphere (VDS) network or an NSX network. It has to be taken into account that on an NSX network, VKS doesn´t currently allow to place the load balancers in the same segment as the VKS cluster. No special considerations have to be taken when hosting BIG-IP in a vSphere (VDS) network. Whether this is a single-cluster or a multi-cluster deployment. When using the multi-cluster option and clusterIP mode (only possible with Calico in VKS), it has to be taken into account that the POD networks of the clusters cannot have overlapping prefixes. What Kubernetes networking (CNI) is desired to be used. CIS supports both VKS supported CNIs: Antrea (default) and Calico. From the CIS point of view, the CNI is only relevant when sending traffic directly to the PODs. See next. What integration with the CNI is desired between the BIG-IP and VKS NodePort mode This is done by making applications discoverable using Services of type NodePort. From the BIG-IP, the traffic is sent to the Node´s IPs where it is redistributed to the POD depending on the TrafficPolicies of the Service. This is CNI agnostic. Any CNI can be used. Direct-to-POD mode This is done by making applications discoverable using the Services of type ClusterIP. Note that the CIS integration with Antrea uses Antrea´s nodePortLocal mechanism, which requires an additional annotation in the Service declaration. See the CIS VKS page in F5 CloudDocs for details. This Antrea nodePortLocal mechanism allows to send the traffic directly to the POD without actually using the POD IP address. This is especially relevant for NSX because it allows to access the PODs without actually re-distributing the PODs IPs across the NSX network, which is not allowed. When using vSphere (VDS) networking, either Antrea’s nodePortLocal or clusterIP with Calico can be used. Another way (but not frequent) is the use of hostNetwork POD networking because it requires privileges for the application PODs or ingress controllers. Network-wise, this would have a similar behavior to nodePortLocal, but without the automatic allocation of ports. Whether the deployment is a single-tier or a two-tier deployment. A single-tier deployment is a deployment where the BIG-IP sends the traffic directly to the application PODs. This has a simpler traffic flow and easier persistence and end-to-end monitoring. A two-tier deployment sends the traffic to an ingress controller POD instead of the application PODs. This ingress controller could be Contour, NGINX Gateway Fabric, Istio or an API gateway. This type of deployment offers the ultimate scalability and provides additional segregation between the BIG-IPs (typically owned by NetOps) and the Kubernetes cluster (typically owned by DevOps). Once CIS is deployed, applications can be published either using the Kubernetes standard Ingress resource or F5’s Custom Resources. This latter is the recommended way because it allows to expose most of the BIG-IPs capabilities. Details on the Ingress resource and F5 custom annotations can be found here. Details on the F5 CRDs can be found here. Please note that at time of this writing Antrea nodePortLocal doesn´t support the TransportServer CRD. Please consult your F5 representative for its availability. Detailed instructions on how to deploy CIS for VKS can be found on this CIS VKS page in F5 CloudDocs. Application-aware MultiCluster support MultiCluster allows to expose applications that are hosted in multiple VKS clusters and publish them in a single VIP. BIG-IP & CIS are in charge of: Discover where the PODs of the applications are hosted. Note that a given application doesn´t need to be available in all clusters. Upon receiving the request for a given application, decide to which cluster and Node/Pod the request has to be sent. This decision is based on the weight of each cluster, the application availability and the load balancing algorithm being applied. Single-tier or Two-tier architectures are possible. NodePort and ClusterIP modes are possible as well. Note that at the time of this writing, Antrea in ClusterIP mode (nodePortLocal) is not supported currently. Please consult your F5 representative for availability of this feature. Considerations for NSX Load Balancers cannot be placed in the same VPC segment where the VMware VKS cluster is. These can be placed in a separate VPC segment of the same VPC gateway as shown in the next diagram. In this arrangement the BIG-IP can be configured as either 1NIC mode or as a regular deployment, in which case the MGMT interface is typically configured through an infrastructure VLAN instead of an NSX segment. The data segment is only required to have enough prefixes to host the self-IPs of the BIG-IP units. The prefixes of the VIPs might not belong to the Data Segment´s subnet. These additional prefixes have to be configured as static routes in the VPC Gateway and Route Redistribution for these must be enabled. Given that the Load Balancers are not in line with the traffic flow towards the VKS Cluster, it is required to use SNAT. When using SNAT pools, the prefixes of these can optionally be configured as additional prefixes of the Data Segment, like the VIPs. Specifically for Calico, clusterIP mode cannot be used in NSX because this would require the BIG-IP to be in the same VPC segment as VMware VKS. Note also that BGP multi-hop is not feasible either because it would require the POD cluster network prefixes to be redistributed through NSX, which is not possible either. Conclusion and final remarks F5 BIG-IPs provides unmatched deployment options and features for VMware VKS; these include: Support for all VKS CNIs, which allows sending the traffic directly instead of using hostNetwork (which implies a security risk) or using the common NodePort, which can incur an additional kube-proxy indirection. Both 1-tier or 2-tier arrangements (or both types simultaneously) are possible. F5´s Container Ingress Services provides the ability to handle multiple VMware VKS clusters with application-aware VIPs. This is a unique feature in the industry. Securing applications with the wide range of L3 to L7 security features provided by BIG-IP, including Advanced WAF and Application Access. To complete the circle, this integration also provides IP address management (IPAM) which provides great flexibility to DevOps teams. All these are available regardless of the form factor of the BIG-IP: Virtual Edition, appliance or chassis, allowing great scalability and multi-tenancy options. In NSX deployments, the recommended form-factor is Virtual Edition in order to connect to the NSX segments. We look forward to hearing your experience and feedback on this article.Future-Proofing Kubernetes Routing: From Standard Ingress to Role-Based CRDs
Standard Kubernetes Ingress is a "lowest common denominator" model that enforces rigid, monolithic configurations, limiting agility. F5 NGINX Ingress Controller breaks this mold with role-oriented Custom Resource Definitions (CRDs)—VirtualServer and VirtualServerRoute—enabling a modern system of delegated authority that is a sharp contrast to standard monolithic configurations. This approach provides platforms with central security control while granting application teams the essential freedom to manage their own routes without risk.Where SASE Ends and ADSP Begins, The Dual-Plane Zero Trust Model
Introduction Zero Trust Architecture (ZTA) mandates “never trust, always verify”, explicit policy enforcement across every user, device, network, application, and data flow, regardless of location. The challenge is that ZTA isn’t a single product. It’s a model that requires enforcement at multiple planes. Two converged platforms cover those planes: SASE at the access edge, and F5 ADSP at the application edge. This article explains what each platform does, where the boundary sits, and why both are necessary. Two Planes, One Architecture SASE and F5 ADSP are both converged networking and security platforms. Both deploy across hardware, software, and SaaS. Both serve NetOps, SecOps, and PlatformOps through unified consoles. But they enforce ZTA at different layers, and at different scales. SASE secures the user/access plane: it governs who reaches the network and under what conditions, using ZTNA (Zero Trust Network Access), SWG, CASB, and DLP. F5 ADSP secures the application plane: it governs what authenticated sessions can actually do once traffic arrives, using WAAP, bot management, API security, and ZTAA (Zero Trust Application Access). The NIST SP 800-207 distinction is useful here: SASE houses the Policy Decision Point for network access; ADSP houses the Policy Enforcement Point at the application layer. Neither alone satisfies the full ZTA model. The Forward/Reverse Proxy Split The architectural difference comes down to proxy direction. SASE is a forward proxy. Employee traffic terminates at an SSE PoP, where identity and device posture are checked before content is retrieved on the user’s behalf. SD-WAN steers traffic intelligently across MPLS, broadband, 5G, or satellite based on real-time path quality. SSE enforces CASB, RBI, and DLP policies before delivery. F5 ADSP is a reverse proxy. Traffic destined for an application terminates at ADSP first, where L4–7 inspection, load balancing, and policy enforcement happen before the request reaches the backend. ADSP understands application protocols, session behavior, and traffic patterns, enabling health monitoring, TLS termination, connection multiplexing, and granular authorization across BIG-IP (hardware, virtual, cloud), NGINX, BIG-IP Next for Kubernetes (BNK), and BIG-IP CNE. The scale difference matters: ADSP handles consumer-facing traffic at orders of magnitude higher volume than SASE handles employee access. This is why full platform convergence only makes sense at the SMB scale, enterprise organizations operate them as distinct, specialized systems owned by different teams. ZTA Principles Mapped to Each Platform ZTA requires continuous policy evaluation, not just at initial authentication, but throughout every session. The table below maps NIST SP 800-207 principles to how each platform implements them. ZTA Principle SASE F5 ADSP Verify explicitly Identity + device posture evaluated per session at SSE PoP L7 authz per request: token validation, API key checks, behavioral scoring Least privilege ZTNA grants per-application, per-session access, no implicit lateral movement API gateway enforces method/endpoint/scope, no over-permissive routes Assume breach CASB + DLP monitors post-access behavior, continuous posture re-evaluation WAF + bot mitigation inspects every payload; micro-segmentation at service boundaries Continuous validation Real-time endpoint compliance; access revoked on posture drift ML behavioral baselines detect anomalous request patterns mid-session Use Case Breakdown Secure Remote Access SASE enforces ZTNA, validating identity, MFA, and endpoint compliance before granting access. F5 ADSP picks up from there, enforcing L7 authorization continuity: token inspection, API gateway policy, and traffic steering to protected backends. A compromised identity that passes ZTNA still faces ADSP’s per-request behavioral inspection. Web Application and API Protection (WAAP) SASE pre-filters known malicious IPs and provides initial TLS inspection, reducing volumetric noise. F5 ADSP delivers full-spectrum WAAP in-path, signature, ML, and behavioral WAF models simultaneously, where application context is fully visible. SASE cannot inspect REST API schemas, GraphQL mutation intent, or session-layer business logic. ADSP can. Bot Management SASE blocks bot C2 communications and applies rate limits at the network edge. F5 ADSP handles what gets through: JavaScript telemetry challenges, ML-based device fingerprinting, and human-behavior scoring that distinguishes legitimate automation (CI/CD, partner APIs) from credential stuffing and scraping, regardless of source IP reputation. AI Security SASE applies CASB and DLP policies to block sensitive data uploads to external AI services and discover shadow AI usage across the workforce. F5 ADSP protects custom AI inference endpoints: prompt injection filtering, per-model, rate limiting, request schema validation, and encrypted traffic inspection. The Handoff Gap, and How to Close It The most common zero trust failure in hybrid architectures isn’t within either platform. It’s the handoff between them. ZTNA grants access, but session context (identity claims, device posture score, risk level) doesn’t automatically propagate to the application plane. The fix is explicit context propagation: SASE injects headers carrying identity and posture signals; ADSP policy engines consume them for L7 authorization decisions. This closes the gap between “who is allowed to connect” and “what that specific session is permitted to do.” Conclusion SASE and F5 ADSP are not competing platforms. They are complementary enforcement planes. SASE answers: can this user reach the application? ADSP answers: What can this session do once it arrives? Organizations that deploy only one leave systematic gaps. Together, with explicit context propagation at the handoff, they deliver the end-to-end zero trust coverage that NIST SP 800-207 actually requires. Related Content Why SASE and ADSP are complementary platform
Infrastructure as Code: Using Git to deploy F5 iRules Automagically
Many approaches within DevOps take the view that infrastructure must be treated like code to realize true continuous deployment. The TL;DR on the concept is simply this: infrastructure configuration and related code (like that created to use data path programmability) should be treated like, well, code. That is, it should be stored in a repository, versioned, and automatically pulled as part of the continuous deployment process. This is one of the foundational concepts that enables immutable infrastructure, particularly for infrastructure tasked with providing application services like load balancing, web application security, and optimization. Getting there requires that you not only have per-application partitioning of configuration and related artifacts (templates, code, etc…) but a means to push those artifacts to the infrastructure for deployment. In other words, an API. A BIG-IP, whether appliance, virtual, cloud, or some combination thereof, provides the necessary per-application partitioning required to support treating its app services (load balancing, web app security, caching, etc..) as “code”. A whole lot of apps being delivered today take advantage of the programmability available (iRules) to customize and control everything from scalability to monitoring to supporting new protocols. It’s code, so you know that means it’s pretty flexible. So it’s not only code, but it’s application-specific code, and that means in the big scheme of continuous deployment, it should be treated like code. It should be versioned, managed, and integrated into the (automated) deployment process. And if you’re standardized on Git, you’d probably like the definition of your scalability service (the load balancing) and any associated code artifacts required (like some API version management, perhaps) to be stored in Git and integrated into the CD pipeline. Cause, automation is good. Well have I got news for you! I wish I’d coded this up (but I don’t do as much of that as I used to) but that credit goes to DevCentral community member Saverio. He wasn’t the only one working on this type of solution, but he was the one who coded it up and shared it on Git (and here on DevCentral) for all to see and use. The basic premise is that the system uses Git as a repository for iRules (BIG-IP code artifacts) and then sets up a trigger such that whenever that iRule is committed, it’s automagically pushed back into production. Now being aware that DevOps isn’t just about automagically pushing code around (especially in production) there’s certain to be more actual steps here in terms of process. You know, like code reviews because we are talking about code here and commits as part of a larger process, not just because you can. That caveat aside, the bigger takeaway is that the future of infrastructure relies as much on programmability – APIs, templates, and code – as it does on the actual services it provides. Infrastructure as Code, whether we call it that or not, is going to continue to shift left into production. The operational process management we generally like to call “orchestration” and “data center automation" , like its forerunner, business process management, will start requiring a high degree of programmability and integratability (is too a word, I just made it up) to ensure the infrastructure isn’t impeding the efficiency of the deployment process. Code on, my friends. Code on.Leveraging BGP and ECMP for F5 Distributed Cloud Customer Edge, Part Two
Introduction This is the second part of our series on leveraging BGP and ECMP for F5 Distributed Cloud Customer Edge deployments. In Part One, we explored the high-level concepts, architecture decisions, and design principles that make BGP and ECMP such a powerful combination for Customer Edge high availability and maintenance operations. This article provides step-by-step implementation guidance, including: High-level and low-level architecture diagrams Complete BGP peering and routing policy configuration in F5 Distributed Cloud Console Practical configuration examples for Fortinet FortiGate and Palo Alto Networks firewalls By the end of this article, you'll have everything you need to implement BGP-based high availability for your Customer Edge deployment. Architecture Overview Before diving into configuration, let’s establish a clear picture of the architecture we’re implementing. We’ll examine this from two perspectives: a high-level logical view and a detailed low-level view showing specific IP addressing and AS numbers. High-Level Architecture The high-level architecture illustrates the fundamental traffic flow and BGP relationships in our deployment: Key Components: Component Role Internet External connectivity to the network Next-Generation Firewall Acts as the BGP peer and performs ECMP distribution to Customer Edge nodes Customer Edge Virtual Site Two or more CE nodes advertising identical VIP prefixes via BGP The architecture follows a straightforward principle: the upstream firewall establishes BGP peering with each CE node. Each CE advertises its VIP addresses as /32 routes. The firewall, seeing multiple equal-cost paths to the same destination, distributes incoming traffic across all available CE nodes using ECMP. Low-Level Architecture with IP Addressing The low-level diagram provides the specific details needed for implementation, including IP addresses and AS numbers: Network Details: Component IP Address Role Firewall (Inside) 10.154.4.119/24 BGP Peer, ECMP Router CE1 (Outside) 10.154.4.160/24 Customer Edge Node 1 CE2 (Outside) 10.154.4.33/24 Customer Edge Node 2 Global VIP 192.168.100.10/32 Load Balancer VIP BGP Configuration: Parameter Firewall Customer Edge AS Number 65001 65002 Router ID 10.154.4.119 Auto-assigned based on interface IP Advertised Prefix None 192.168.100.0/24 le 32 This configuration uses eBGP (External BGP) between the firewall and CE nodes, with different AS numbers for each. The CE nodes share the same AS number (65002), which is the standard approach for multi-node CE deployments advertising the same VIP prefixes. Configuring BGP in F5 Distributed Cloud Console The F5 Distributed Cloud Console provides a centralized interface for configuring BGP peering and routing policies on your Customer Edge nodes. This section walks you through the complete configuration process. Step 1: Configure the BGP peering Go to: Multi-Cloud Network Connect --> Manage --> Networking --> External Connectivity --> BGP Peers & Policies Click on Add BGP Peer Then add the following information: Object name Site where to apply this BGP configuration ASN Router ID Here is an example of the required parameters. Then click on Peers --> Add Item And filled the relevant fields like below by adapting the parameters for your requirements. Step 2: Configure the BGP routing policies Go to: Multi-Cloud Network Connect --> Manage --> Networking --> External Connectivity --> BGP Peers & Policies --> BGP Routing Policies Click on Add BGP Routing Policy Add a name for your BGP routing policy object and click on Configure to add the rules. Click on Add Item to add a rule. Here we are going to allow the /32 prefixes from our VIP subnet (192.168.100.0/24). Save the BGP Routing Policy Repeat the action to create another BGP routing policy with the exact same parameters except the Action Type, which should be of type Deny. Now we have two BGP routing policies: One to allow the VIP prefixes (for normal operations) One to deny the VIP prefixes (for maintenance mode) We still need to a a third and final BGP routing policy, in order to deny any prefixes on the CE. For that, create a third BGP routing policy with this match. Step 3: Apply the BGP routing policies To apply the BGP routing policies in your BGP peer object, edit the Peer and: Enable the BGP routing policy Apply the BGP routing policy objects created before for Inbound and Outbound Fortinet FortiGate Configuration FortiGate firewalls are widely deployed as network security appliances and support robust BGP capabilities. This section provides the minimum configuration for establishing BGP peering with Customer Edge nodes and enabling ECMP load distribution. Step 1: Configure the Router ID and AS Number Configure the basic BGP settings: config router bgp set as 65001 set router-id 10.154.4.119 set ebgp-multipath enable Step 2: Configure BGP Neighbors Add each CE node as a BGP neighbor: config neighbor edit "10.154.4.160" set remote-as 65002 set route-map-in "ACCEPT-CE-VIPS" set route-map-out "DENY-ALL" set soft-reconfiguration enable next edit "10.154.4.33" set remote-as 65002 set route-map-in "ACCEPT-CE-VIPS" set route-map-out "DENY-ALL" set soft-reconfiguration enable next end end Step 3: Create Prefix List for VIP Range Define the prefix list that matches the CE VIP range: config router prefix-list edit "CE-VIP-PREFIXES" config rule edit 1 set prefix 192.168.100.0 255.255.255.0 set ge 32 set le 32 next end next end Important: The ge 32 and le 32 parameters ensure we only match /32 prefixes within the 192.168.100.0/24 range, which is exactly what CE nodes advertise for their VIPs. Step 4: Create Route Maps Configure route maps to implement the filtering policies: Inbound Route Map (Accept VIP prefixes): config router route-map edit "ACCEPT-CE-VIPS" config rule edit 1 set match-ip-address "CE-VIP-PREFIXES" next end next end Outbound Route Map (Deny all advertisements): config router route-map edit "DENY-ALL" config rule edit 1 set action deny next end next end Step 5: Verify BGP Configuration After applying the configuration, verify the BGP sessions and routes: Check BGP neighbor status: get router info bgp summary VRF 0 BGP router identifier 10.154.4.119, local AS number 65001 BGP table version is 4 1 BGP AS-PATH entries 0 BGP community entries Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 10.154.4.33 4 65002 2092 2365 0 0 0 00:05:33 1 10.154.4.160 4 65002 2074 2346 0 0 0 00:14:14 1 Total number of neighbors 2 Verify ECMP routes: get router info routing-table bgp Routing table for VRF=0 B 192.168.100.10/32 [20/255] via 10.154.4.160 (recursive is directly connected, port2), 00:00:11, [1/0] [20/255] via 10.154.4.33 (recursive is directly connected, port2), 00:00:11, [1/0] Palo Alto Networks Configuration Palo Alto Networks firewalls provide enterprise-grade security with comprehensive routing capabilities. This section covers the minimum BGP configuration for peering with Customer Edge nodes. Note: This part is assuming that Palo Alto firewall is configured in the new "Advanced Routing Engine" mode. And we will use the logical-router named "default". Step 1: Configure ECMP parameters set network logical-router default vrf default ecmp enable yes set network logical-router default vrf default ecmp max-path 4 set network logical-router default vrf default ecmp algorithm ip-hash Step 2: Configure objects IPs and firewall rules for BGP peering set address CE1 ip-netmask 10.154.4.160/32 set address CE2 ip-netmask 10.154.4.33/32 set address-group BGP_PEERS static [ CE1 CE2 ] set address LOCAL_BGP_IP ip-netmask 10.154.4.119/32 set rulebase security rules ALLOW_BGP from service set rulebase security rules ALLOW_BGP to service set rulebase security rules ALLOW_BGP source LOCAL_BGP_IP set rulebase security rules ALLOW_BGP destination BGP_PEERS set rulebase security rules ALLOW_BGP application bgp set rulebase security rules ALLOW_BGP service application-default set rulebase security rules ALLOW_BGP action allow Step 3: Palo Alto Configuration Summary (CLI Format) set network routing-profile filters prefix-list ALLOWED_PREFIXES type ipv4 ipv4-entry 1 prefix entry network 192.168.100.0/24 set network routing-profile filters prefix-list ALLOWED_PREFIXES type ipv4 ipv4-entry 1 prefix entry greater-than-or-equal 32 set network routing-profile filters prefix-list ALLOWED_PREFIXES type ipv4 ipv4-entry 1 prefix entry less-than-or-equal 32 set network routing-profile filters prefix-list ALLOWED_PREFIXES type ipv4 ipv4-entry 1 action permit set network routing-profile filters prefix-list ALLOWED_PREFIXES description "Allow only m32 inside 192.168.100.0m24" set network routing-profile filters prefix-list DENY_ALL type ipv4 ipv4-entry 1 prefix entry network 0.0.0.0/0 set network routing-profile filters prefix-list DENY_ALL type ipv4 ipv4-entry 1 prefix entry greater-than-or-equal 0 set network routing-profile filters prefix-list DENY_ALL type ipv4 ipv4-entry 1 prefix entry less-than-or-equal 32 set network routing-profile filters prefix-list DENY_ALL type ipv4 ipv4-entry 1 action deny set network routing-profile filters prefix-list DENY_ALL description "Deny all prefixes" set network routing-profile bgp filtering-profile FILTER_INBOUND ipv4 unicast inbound-network-filters prefix-list ALLOWED_PREFIXES set network routing-profile bgp filtering-profile FILTER_OUTBOUND ipv4 unicast inbound-network-filters prefix-list DENY_ALL set network logical-router default vrf default bgp router-id 10.154.4.119 set network logical-router default vrf default bgp local-as 65001 set network logical-router default vrf default bgp install-route yes set network logical-router default vrf default bgp enable yes set network logical-router default vrf default bgp peer-group BGP_PEERS type ebgp set network logical-router default vrf default bgp peer-group BGP_PEERS address-family ipv4 ipv4-unicast-default set network logical-router default vrf default bgp peer-group BGP_PEERS filtering-profile ipv4 FILTER_INBOUND set network logical-router default vrf default bgp peer-group BGP_PEERS filtering-profile ipv4 FILTER_OUTBOUND set network logical-router default vrf default bgp peer-group BGP_PEERS peer CE1 peer-as 65002 set network logical-router default vrf default bgp peer-group BGP_PEERS peer CE1 local-address interface ethernet1/2 set network logical-router default vrf default bgp peer-group BGP_PEERS peer CE1 local-address ip svc-intf-ip set network logical-router default vrf default bgp peer-group BGP_PEERS peer CE1 peer-address ip 10.154.4.160 set network logical-router default vrf default bgp peer-group BGP_PEERS peer CE2 peer-as 65002 set network logical-router default vrf default bgp peer-group BGP_PEERS peer CE2 local-address interface ethernet1/2 set network logical-router default vrf default bgp peer-group BGP_PEERS peer CE2 local-address ip svc-intf-ip set network logical-router default vrf default bgp peer-group BGP_PEERS peer CE2 peer-address ip 10.154.4.33 Step 4: Verify BGP Configuration After committing the configuration, verify the BGP sessions and routes: Check BGP neighbor status: run show advanced-routing bgp peer status logical-router default Logical Router: default ============== Peer Name: CE2 BGP State: Established, up for 00:01:55 Peer Name: CE1 BGP State: Established, up for 00:00:44 Verify ECMP routes: run show advanced-routing route logical-router default Logical Router: default ========================== flags: A:active, E:ecmp, R:recursive, Oi:ospf intra-area, Oo:ospf inter-area, O1:ospf ext 1, O2:ospf ext 2 destination protocol nexthop distance metric flag tag age interface 0.0.0.0/0 static 10.154.1.1 10 10 A 01:47:33 ethernet1/1 10.154.1.0/24 connected 0 0 A 01:47:37 ethernet1/1 10.154.1.99/32 local 0 0 A 01:47:37 ethernet1/1 10.154.4.0/24 connected 0 0 A 01:47:37 ethernet1/2 10.154.4.119/32 local 0 0 A 01:47:37 ethernet1/2 192.168.100.10/32 bgp 10.154.4.33 20 255 A E 00:01:03 ethernet1/2 192.168.100.10/32 bgp 10.154.4.160 20 255 A E 00:01:03 ethernet1/2 total route shown: 7 Implementing CE Isolation for Maintenance As discussed in Part One, one of the key advantages of BGP-based deployments is the ability to gracefully isolate CE nodes for maintenance. Here’s how to implement this in practice. Isolation via F5 Distributed Cloud Console To isolate a CE node from receiving traffic, in your BGP peer object, edit the Peer and: Change the Outbound BGP routing policy from the one that is allowing the VIP prefixes to the one that is denying the VIP prefixes The CE will stop advertising its VIP routes, and within seconds (based on BGP timers), the upstream firewall will remove this CE from its ECMP paths. Verification During Maintenance On your firewall, verify the route withdrawal (in this case we are using a Fortigate firewall): get router info bgp summary VRF 0 BGP router identifier 10.154.4.119, local AS number 65001 BGP table version is 4 1 BGP AS-PATH entries 0 BGP community entries Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 10.154.4.33 4 65002 2070 2345 0 0 0 00:04:05 0 10.154.4.160 4 65002 2057 2326 0 0 0 00:12:46 1 Total number of neighbors 2 We are not receiving any prefixes anymore for the 10.154.4.33 peer. get router info routing-table bgp Routing table for VRF=0 B 192.168.100.10/32 [20/255] via 10.154.4.160 (recursive is directly connected, port2), 00:06:34, [1/0] End we have now only one path. Restoring the CE in the data path After maintenance is complete: Return to the BGP Peer configuration in the F5XC Console Restore the original export policy (permit VIP prefixes) Save the configuration On the upstream firewall, confirm that CE prefixes are received again and that ECMP paths are restored Conclusion This article has provided the complete implementation details for deploying BGP and ECMP with F5 Distributed Cloud Customer Edge nodes. You now have: A clear understanding of the architecture at both high and low levels Step-by-step instructions for configuring BGP in F5 Distributed Cloud Console Ready-to-use configurations for both Fortinet FortiGate and Palo Alto Networks firewalls Practical guidance for implementing graceful CE isolation for maintenance By combining the concepts from the first article with the practical configurations in this article, you can build a robust, highly available application delivery infrastructure that maximizes resource utilization, provides automatic failover, and enables zero-downtime maintenance operations. The BGP-based approach transforms your Customer Edge deployment from a traditional Active/Standby model into a full active topology where every node contributes to handling traffic, and any node can be gracefully removed for maintenance without impacting your users.Using ExternalDNS with F5 CIS to Automate DNS on Non-F5 DNS Servers
Overview F5 Container Ingress Services (CIS) is a powerful way to manage BIG-IP configuration directly from Kubernetes. Using CIS Custom Resource Definitions (CRDs) like VirtualServer and TransportServer, you can define rich traffic management policies in native Kubernetes manifests and have CIS automatically create and update Virtual IPs (VIPs) on BIG-IP. One common question that comes up: “What if I want DNS records created automatically when a VirtualServer comes up, but I’m not using F5 DNS?” This article answers exactly that question. We’ll walk through how to combine CIS VirtualServer resources with the community project ExternalDNS to automatically register DNS records on external DNS providers like AWS Route 53, Infoblox, CoreDNS, Azure DNS, and others — all without touching a zone file by hand. Background: How DNS Automation Typically Works in Kubernetes Before diving into the solution, it’s worth grounding ourselves in how DNS automation normally works in Kubernetes. The Standard Pattern: Services of Type LoadBalancer The most common pattern is: Create a Service of type LoadBalancer. A cloud controller (or a bare-metal equivalent like MetalLB) assigns an external IP and updates the status.loadBalancer.ingress field of the Service object. ExternalDNS watches for Services of type LoadBalancer with specific annotations, reads the IP from the status field, and creates a DNS A record on your external DNS server. This is clean, well-understood, and widely supported. ExternalDNS can also watch Ingress objects or Services of type ClusterIP and NodePort, but the LoadBalancer pattern is the most common integration point. Where F5 CIS Fits In CIS supports creating VIPs on BIG-IP in multiple ways: VirtualServer / TransportServer CRDs — Most customers prefer to use VS or TS CRDs because they expose rich BIG-IP capabilities: iRules, custom persistence profiles, health monitors, TLS termination policies, and more. This is where the DNS automation story gets more nuanced and is the focus of this article. Service of type LoadBalancer — CIS watches for Services of type LoadBalancer. Typically an IPAM controller or a custom annotation will be used to configure an IP address. CIS allocates a VIP on BIG-IP. This is not the focus of this article. Other — CIS can also use Ingress or ConfigMap resources, but these are more historical approaches, not recommended for new deployments, and out of scope for this article. The Gap: F5 CRDs and Non-F5 DNS CIS does include its own ExternalDNS CRD (not to be confused with the community project of the same name). However, F5’s built-in ExternalDNS CRD only supports F5 DNS (BIG-IP DNS / GTM). If you’re using Route 53, Infoblox, PowerDNS, or any other DNS provider, you need a different approach. That’s where the community ExternalDNS project comes in. The Solution: VirtualServer + Service of Type LoadBalancer + ExternalDNS The trick is straightforward once you see it: CIS can manage a VIP on BIG-IP via a VirtualServer CRD while simultaneously updating the status field of a Service of type LoadBalancer. ExternalDNS then reads that status field and creates DNS records. Let’s walk through the manifests. Step-by-Step Walkthrough Step 1: Deploy Your Application A standard Deployment — nothing special here. apiVersion: apps/v1 kind: Deployment metadata: name: my-app namespace: my-namespace spec: replicas: 2 selector: matchLabels: app: my-app template: metadata: labels: app: my-app spec: containers: - name: my-app image: my-app:latest ports: - containerPort: 8080 Step 2: Create the Service of Type LoadBalancer This Service is the linchpin of the whole solution. It serves three purposes: It acts as a target for the CIS VirtualServer pool (either via NodePort or directly to pod IPs in cluster mode). CIS updates its status.loadBalancer.ingress field with the BIG-IP VIP address. ExternalDNS reads its status and annotations to create a DNS record. apiVersion: v1 kind: Service metadata: name: my-app-svc namespace: my-namespace annotations: # ExternalDNS annotation — tells ExternalDNS what hostname to register external-dns.alpha.kubernetes.io/hostname: myapp.example.com # Optional: set a custom TTL external-dns.alpha.kubernetes.io/ttl: "60" spec: selector: app: my-app ports: - port: 80 targetPort: 8080 protocol: TCP type: LoadBalancer # Prevent other LB controllers from acting on this Service loadBalancerClass: f5.com/bigip # Do not allocate NodePort endpoints — more on this below allocateLoadBalancerNodePorts: false Two fields here deserve extra explanation. loadBalancerClass: f5.com/bigip In a typical cluster, multiple controllers may be watching for Services of type LoadBalancer — MetalLB, the cloud provider controller, etc. If you’re using CIS VirtualServer CRDs to manage the VIP (rather than having CIS act directly as a LoadBalancer controller for this Service), you likely don’t want any of those other controllers touching this Service. Setting loadBalancerClass to a value that no other running controller claims means this Service will be ignored by all LB controllers except the one that explicitly handles that class. In this pattern, you want CIS to "see" this service, but not other controllers. So use the CIS argument --load-balancer-class=f5.com/bigip here. Note: The value of loadBalancerClass in your Service should match the value of load-balancer-class in your CIS deployment. The goal is to prevent unintended controllers from assigning IPs or creating cloud load balancers for this Service. allocateLoadBalancerNodePorts: false By default, LoadBalancer Services in Kubernetes allocate NodePort endpoints. This means traffic could reach your pods directly via : — bypassing BIG-IP, security policies, and your iRules. Setting allocateLoadBalancerNodePorts: false prevents this. The Service effectively behaves like a ClusterIP service in terms of access — the only way to reach it from outside the cluster is via the BIG-IP VIP. This is the right posture when: Your CIS deployment uses --pool-member-type=cluster , sending traffic directly to pod IPs. You want BIG-IP to be the sole external entry point for policy enforcement. Step 3: Create the VirtualServer CRD Now we define the VirtualServer. Note how it references the Service by name in the pool configuration: apiVersion: cis.f5.com/v1 kind: VirtualServer metadata: name: my-app-vs namespace: my-namespace labels: f5cr: "true" spec: host: myapp.example.com ipamLabel: prod # Optional: use F5 IPAM Controller for IP allocation # virtualServerAddress: "10.1.10.50" # Or specify IP directly pools: - path: / service: my-app-svc servicePort: 80 monitor: type: http send: "GET / HTTP/1.1\r\nHost: myapp.example.com\r\n\r\n" recv: "" interval: 10 timeout: 10 When CIS processes this VirtualServer, it: Creates a VIP on BIG-IP Configures the BIG-IP pool with the backends from my-app-svc Writes the VIP IP address back into my-app-svc’s status.loadBalancer.ingress field. That last step is what makes the whole chain work. IP Address: Specify Directly or Use F5 IPAM Controller You have two options for IP allocation: Option A — Specify the IP directly in the VirtualServer manifest: spec: virtualServerAddress: "10.1.10.50" This is simple and predictable. Good for static, well-planned deployments. Option B — Use the F5 IPAM Controller: spec: ipamLabel: prod The F5 IPAM Controller watches for CIS resources with ipamLabel annotations and allocates IPs from a configured range. CIS then picks up the allocated IP automatically. This is ideal when you want full automation without managing IP addresses in YAML files. Step 4: Verify CIS Updates the Service Status After CIS processes the VirtualServer, check the Service: kubectl get svc my-app-svc -n my-namespace -o jsonpath='{.status.loadBalancer.ingress}' You should see output like: [{"ip":"10.1.10.50"}] This is the IP that ExternalDNS will use to create the DNS record. Step 5: ExternalDNS Does Its Job With ExternalDNS deployed and configured for your DNS provider (Route 53, Infoblox, etc.), it will: Discover my-app-svc because it’s of type LoadBalancer with an external-dns.alpha.kubernetes.io/hostname annotation. Read 10.1.10.50 from status.loadBalancer.ingress. Create an A record: myapp.example.com → 10.1.10.50. ExternalDNS handles the rest automatically, including updates if the IP changes. A minimal ExternalDNS deployment for Route 53 would look like: apiVersion: apps/v1 kind: Deployment metadata: name: external-dns namespace: external-dns spec: replicas: 1 selector: matchLabels: app: external-dns template: metadata: labels: app: external-dns spec: serviceAccountName: external-dns containers: - name: external-dns image: registry.k8s.io/external-dns/external-dns:v0.14.0 args: - --source=service - --domain-filter=example.com - --provider=aws - --aws-zone-type=public - --registry=txt - --txt-owner-id=my-cluster Refer to the ExternalDNS documentation for provider-specific configuration (IAM roles for Route 53, credentials for Infoblox, etc.). Putting It All Together: Summary of the Architecture Key Considerations and Design Choices When to Use This Pattern vs. CIS as a LoadBalancer Controller CIS can act directly as a LoadBalancer controller — watching Services of type LoadBalancer and creating VIPs on BIG-IP without any VirtualServer CRD involvement. If that’s sufficient for your needs, it’s simpler. ExternalDNS works with that mode natively, since CIS updates status.loadBalancer.ingress in both cases. Use the VirtualServer CRD approach when you need: Custom iRules or iApps on the VIP Advanced persistence profiles Fine-grained TLS termination control Traffic splitting or A/B routing policies Any BIG-IP capability that doesn’t map directly to Kubernetes Service semantics allocateLoadBalancerNodePorts: false — When It Applies This setting is appropriate when your CIS deployment uses --pool-member-type=cluster . In cluster mode, BIG-IP sends traffic directly to pod IPs, not through NodePort endpoints. Disabling NodePort allocation: Prevents back-door access to your application via : Reduces iptables rule sprawl on your nodes Aligns with a clean security boundary where BIG-IP is the sole ingress If your CIS deployment uses --pool-member-type=nodeport , you should not set allocateLoadBalancerNodePorts: false, as CIS will need those NodePorts to forward traffic. F5 IPAM Controller Integration The F5 IPAM Controller pairs particularly well with this pattern. Rather than managing VIP IP addresses in your VirtualServer manifests, IPAM handles allocation from a configured pool. This means: Platform teams manage IP ranges in the IPAM controller config. Application teams simply specify an ipamLabel in their VirtualServer manifest. CIS picks up the IPAM-assigned IP and writes it to the Service status automatically. The ExternalDNS chain remains identical regardless of whether the IP comes from IPAM or is statically assigned. Frequently Asked Questions Q: Can I use this pattern with TransportServer CRDs instead of VirtualServer? Yes. CIS similarly updates the status of a referenced Service when using TransportServer. The same approach applies. Q: What if I want ExternalDNS to also create a CNAME instead of an A record? Use the external-dns.alpha.kubernetes.io/target annotation on the Service to override the IP with a hostname, causing ExternalDNS to create a CNAME. Refer to ExternalDNS documentation for specifics. Q: Can I use multiple hostnames for the same VirtualServer? Add multiple external-dns.alpha.kubernetes.io/hostname annotations (comma-separated values are supported by ExternalDNS) or create additional Services pointing to the same pods. Conclusion Combining F5 CIS VirtualServer CRDs with the community ExternalDNS project gives you the best of both worlds: rich BIG-IP traffic management via CIS, and flexible, provider-agnostic DNS automation via ExternalDNS. The core insight is simple — CIS writes the BIG-IP VIP IP address back into the Kubernetes Service status field, and ExternalDNS reads from that same field. By using loadBalancerClass and allocateLoadBalancerNodePorts: false, you ensure the Service is a clean “status carrier” that doesn’t accidentally expose your application through unintended paths. Whether you assign VIP IPs statically in your manifests or use the F5 IPAM Controller for full automation, this pattern integrates naturally into any Kubernetes-native GitOps workflow. Additional Resources F5 CIS Documentation F5 CIS VirtualServer CRD Reference F5 IPAM Controller on GitHub ExternalDNS on GitHub ExternalDNS: Service Source Documentation Kubernetes: LoadBalancer Service specification
