The Ingress NGINX Alternative: F5 NGINX Ingress Controller for the Long Term
The Kubernetes community recently announced that Ingress NGINX will be retired in March 2026. After that date, there won’t be any more new updates, bugfixes, or security patches. ingress-nginx is no longer a viable enterprise solution for the long-term, and organizations using it in production should move quickly to explore alternatives and plan to shift their workloads to Kubernetes ingress solutions that are continuing development. Your Options (And Why We Hope You’ll Consider NGINX) There are several good Ingress controllers available—Traefik, HAProxy, Kong, Envoy-based options, and Gateway API implementations. The Kubernetes docs list many of them, and they all have their strengths. Security start-up Chainguard is maintaining a status-quo version of ingress-nginx and applying basic safety patches as part of their EmeritOSS program. But this program is designed as a stopgap to keep users safe while they transition to a different ingress solution. F5 maintains an OSS permissively licensed NGINX Ingress Controller. The project is open source, Apache 2.0 licensed, and will stay that way. There is a team of dedicated engineers working on it with a slate of upcoming upgrades. If you’re already comfortable with NGINX and just want something that works without a significant learning curve, we believe that the F5 NGINX Ingress Controller for Kubernetes is your smoothest path forward. The benefits of adopting NGINX Ingress Controller open source include: Genuinely open source: Apache 2.0 licensed with 150+ contributors from diverse organizations, not just F5. All development happens publicly on GitHub, and F5 has committed to keeping it open source forever. Plus community calls every 2 weeks. Minimal learning curve: Uses the same NGINX engine you already know. Most Ingress NGINX annotations have direct equivalents, and the migration guide provides clear mappings for your existing configurations. Supported annotations include popular ones such as nginx.org/client-body-buffer-size mirrors nginx.ingress.kubernetes.io/client-body-buffer-size (sets the maximum size of the client request body buffer). Also available in VirtualServer and ConfigMap. nginx.org/rewrite-target mirrors nginx.ingress.kubernetes.io/rewrite-target (sets a replacement path for URI rewrites) nginx.org/ssl-ciphers mirrors nginx.ingress.kubernetes.io/ssl-ciphers (configures enabled TLS cipher suites) nginx.org/ssl-prefer-server-cipher mirrors nginx.ingress.kubernetes.io/ssl-prefer-server-ciphers (controls server-side cipher preference during the TLS handshake) Optional enterprise-grade capabilities: While the OSS version is robust, NGINX Plus integration is available for enterprises needing high availability, authentication and authorization, session persistence, advanced security and commercial support Sustainable maintenance: A dedicated full-time team at F5 ensures regular security updates, bug fixes, and feature development. Production-tested at scale: NGINX Ingress Controller powers approximately 40% of Kubernetes Ingress deployments with over 10 million downloads. It’s battle-tested in real production environments. Kubernetes-native design: Custom Resource Definitions (VirtualServer, Policy, TransportServer) provide cleaner configuration than annotation overload, with built-in validation to prevent errors. Advanced capabilities when you need them: Support for canary deployments, A/B testing, traffic splitting, JWT validation, rate limiting, mTLS, and more—available in the open source version. Future-proof architecture: Active development of NGINX Gateway Fabric provides a clear migration path when you’re ready to move to Gateway API. NGINX Gateway Fabric is a conformant Gateway API solution under CNCF conformance criteria and it is one of the most widely used open source Gateway API solutions. Moving to NGINX Ingress Controller Here’s a rough migration guide. You can also check our more detailed migration guide on our documentation site. Phase 1: Take Stock See what you have: Document your current Ingress resources, annotations, and ConfigMaps Check for snippets: Identify any annotations like: nginx.ingress.kubernetes.io/configuration-snippet Confirm you're using it: Run kubectl get pods --all-namespaces --selector app.kubernetes.io/name=ingress-nginx Set it up alongside: Install NGINX Ingress Controller in a separate namespace while keeping your current setup running Phase 2: Translate Your Config Convert annotations: Most of your existing annotations have equivalents in NGINX Ingress Controller - there's a comprehensive migration guide that maps them out Consider VirtualServer resources: These custom resources are cleaner than annotation-heavy Ingress, and give you more control, but it's your choice Or keep using Ingress: If you want minimal changes, it works fine with standard Kubernetes Ingress resources Handle edge cases: For anything that doesn't map directly, you can use snippets or Policy resources Phase 3: Test Everything Try it with test apps: Create some test Ingress rules pointing to NGINX Ingress Controller Run both side-by-side: Keep both controllers running and route test traffic through the new one Verify functionality: Check routing, SSL, rate limiting, CORS, auth—whatever you're using Check performance: Verify it handles your traffic the way you need Phase 4: Move Over Gradually Start small: Migrate your less-critical applications first Shift traffic slowly: Update DNS/routing bit by bit Watch closely: Keep an eye on logs and metrics as you go Keep an escape hatch: Make sure you can roll back if something goes wrong Phase 5: Finish Up Complete the migration: Move your remaining workloads Clean up the old controller: Uninstall community Ingress NGINX once everything's moved Tidy up: Remove old ConfigMaps and resources you don't need anymore Enterprise-grade capabilities and support Once an ingress layer becomes mission-critical, enterprise features become necessary. High availability, predictable failover, and supportability matter as much as features. Enterprise-grade capabilities available for NGINX Ingress Controller Plus include high availability, authentication and authorization, commercial support, and more. These ensure production traffic remains fast, secure, and reliable. Capabilities include: Commercial support Backed by vendor commercial support (SLAs, escalation paths) for production incidents Access to tested releases, patches, and security fixes suitable for regulated/enterprise environments Guidance for production architecture (HA patterns, upgrade strategies, performance tuning) Helps organizations standardize on a supported ingress layer for platform engineering at scale Dynamic Reconfiguration Upstream configuration updates via API without process reloads Eliminates memory bloat and connection timeouts as upstream server lists and variables are updated in real time when pods scale or configurations change Authentication & Authorization Built-in authentication support for OAuth 2.0 / OIDC, JWT validation, and basic auth External identity provider integration (e.g., Okta, Azure AD, Keycloak) via auth request patterns JWT validation at the edge, including signature verification, claims inspection, and token expiry enforcement Fine-grained access control based on headers, claims, paths, methods, or user identity Optional Web Application Firewall Native integration with F5 WAF for NGINX for OWASP Top 10 protection, gRPC schema validation, and OpenAPI enforcement DDoS mitigation capabilities when combined with F5 security solutions Centralized policy enforcement across multiple ingress resources High availability (HA) Designed to run as multiple Ingress Controller replicas in Kubernetes for redundancy and scale State sharing: Maintains session persistence, rate limits, and key-value stores for seamless uptime. Here’s the full list of differences between NGINX Open Source and NGINX One – a package that includes NGINX Plus Ingress Controller, NGINX Gateway Fabric, F5 WAF for NGINX, and NGINX One Console for managing NGINX Plus Ingress Controllers at scale. Get Started Today Ready to begin your migration? Here's what you need: 📚 Read the full documentation: NGINX Ingress Controller Docs 💻 Clone the repository: github.com/nginx/kubernetes-ingress 🐳 Pull the image: Docker Hub - nginx/nginx-ingress 🔄 Follow the migration guide: Migrate from Ingress-NGINX to NGINX Ingress Controller Interested in the enterprise version? Try NGINX One for free and give it a whirl The NGINX Ingress Controller community is responsive and full of passionate builders -- join the conversation in the GitHub Discussions or the NGINX Community Forum. You’ve got time to plan this migration right, but don’t wait until March 2026 to start.F5 Distributed Cloud Kubernetes Integration: Securing Services with Direct Pod Connectivity
Introduction As organizations embrace Kubernetes for container orchestration, they face critical challenges in exposing services securely to external consumers while maintaining granular control over traffic management and security policies. Traditional approaches using NodePort services or basic ingress controllers often fall short in providing the advanced application delivery and security features required for production workloads. F5 Distributed Cloud (F5 XC) addresses these challenges by offering enterprise-grade application delivery and security services through its Customer Edge (CE) nodes. By establishing direct connectivity to Kubernetes pods, F5 XC can provide sophisticated load balancing, WAF protection, API security, and multi-cloud connectivity without the limitations of NodePort-based architectures. This article demonstrates how to architect and implement F5 XC CE integration with Kubernetes clusters to expose and secure services effectively, covering both managed Kubernetes platforms (AWS EKS, Azure AKS, Google GKE) and self-managed clusters using K3S with Cilium CNI. Understanding F5 XC Kubernetes Service Discovery F5 Distributed Cloud includes a native Kubernetes service discovery feature that communicates directly with Kubernetes API servers to retrieve information about services and their associated pods. This capability operates in two distinct modes: Isolated Mode In this mode, F5 XC CE nodes are isolated from the Kubernetes cluster pods and can only reach services exposed as NodePort services. While the discovery mechanism can retrieve all services, connectivity is limited to NodePort-exposed endpoints with the inherent NodePort limitations: Port Range Restrictions: Limited to ports 30000-32767 Security Concerns: Exposes services on all node IPs Performance Overhead: Additional network hops through kube-proxy Limited Load Balancing: Basic round-robin without advanced health checks Non-Isolated Mode, Direct Pod Connectivity (and why it matters) This is the focus of our implementation. In non-isolated mode, F5 XC CE nodes can reach Kubernetes pods directly using their pod IP addresses. This provides several advantages: Simplified Architecture: Eliminate NodePort complexity and port management limitation Enhanced Security: Apply WAF, DDoS protection, and API security directly at the pod level Advanced Load Balancing: Sophisticated algorithms, circuit breaking, and retry logic Architectural Patterns for Pod IP Accessibility To enable direct pod connectivity from external components like F5 XC CEs, the pod IP addresses must be routable outside the Kubernetes cluster. The implementation approach varies based on your infrastructure: Cloud Provider Managed Kubernetes Cloud providers typically handle pod IP routing through their native Container Network Interfaces (CNIs): Figure 1: Cloud providers' K8S CNI routes PODs IPs to the Cloud Provider Private Cloud Routing Table AWS EKS: Uses Amazon VPC CNI, which assigns VPC IP addresses directly to pods Azure AKS: Traditional CNI mode allocates Azure VNET IPs to pods Google GKE: VPC-native clusters provide direct pod IP routing In these environments, the cloud provider's CNI automatically updates routing tables to make pod IPs accessible within the VPC/VNET. Self-Managed Kubernetes Clusters For self-managed clusters, you need an advanced CNI that can expose the Kubernetes overlay network. The most common solutions are: Cilium: Provides eBPF-based networking with BGP support Calico: Offers flexible networking policies with BGP peering capabilities and eBPF support as well These CNIs typically use BGP to advertise pod subnets to external routers, making them accessible from outside the cluster. Figure 2: Self-managed K8S clusters use advanced CNI with BGP to expose the overlay subnet Cloud Provider Implementations AWS EKS Architecture Figure 3: AWS EKS with F5 XC CE integration using VPC CNI With AWS EKS, the VPC CNI plugin assigns real VPC IP addresses to pods, making them directly routable within the VPC without additional configuration. Azure AKS Traditional CNI Figure 4: Azure AKS with traditional CNI mode for direct pod connectivity Azure's traditional CNI mode allocates IP addresses from the VNET subnet directly to pods, enabling native Azure networking features. Google GKE VPC-Native Figure 5: Google GKE VPC-native clusters with alias IP ranges for pods GKE's VPC-native mode uses alias IP ranges to provide pods with routable IP addresses within the Google Cloud VPC. Deeper dive into the implementation Implementation Example 1: AWS EKS Integration Let's walk through a complete implementation using AWS EKS as our Kubernetes platform. Prerequisites and Architecture Network Configuration: VPC CIDR: 10.154.0.0/16 Three private subnets (one per availability zone) F5 XC CE deployed in Private Subnet 1 EKS worker nodes distributed across all three subnets Figure 6: Complete EKS implementation architecture with F5 XC CE integration Kubernetes Configuration: EKS cluster with AWS VPC CNI Sample application: microbot (simple HTTP service) Three replicas distributed across nodes What is running inside the K8S cluster? The PODs We have three PODs in the default namespace. Figure 7: The running PODs in the EKS cluster One running with POD IP 10.154.125.116, another one with POD IP 10.154.76.183 and one running with POD IP 10.154.69.183. microbot POD is a simple HTTP application that is returning the full name of the POD and an image. Figure 8: The microbot app The services Figure 9: The services running in the EKS cluster Configure F5 XC Kubernetes Service Discovery Create a K8S service discovery object. Figure 10: Kubernetes service discovery configuration In the “Access Credentials” activate the “Show Advanced Fields” slider. This is the key! Figure 11: The "advanced fields" slider Then provide the Kubeconfig file of the K8S cluster and select “Kubernetes POD reachable”. Figure 12: Kubernetes POD network reachability Then the K8S should be displayed in the “Service Discoveries”. Figure 13: The discovered PODs IPs One can see that the services are discovered by the F5 XC node and more interestingly, the PODs IPs. Are the pods reachable from the F5XC CE? Figure 14: Testing connectivity to pod 10.154.125.116 Figure 15: Testing connectivity to pod 10.154.76.183 Figure 16: Testing connectivity to pod 10.154.69.183 Yes, they are! Create Origin Pool with K8S Service Create an origin pool that references your Kubernetes service: Figure 17: Creating origin pool with Kubernetes service type Create an HTTPS Load-Balancer and test the service Just create a regular F5 XC HTTPS Load-Balancer and use the origin pool created above. Figure 18: Traffic load-balanced across the three PODs The result shows traffic being load-balanced across all EKS pods. Implementation Example 2: Self-Managed K3S with Cilium CNI One infrastructure subnet (10.154.1.0/24) in which the following components are going to be deployed: F5 XC CE single node (10.154.1.100) Two Linux Ubuntu nodes (10.154.1.10 & 10.154.1.11) On the Linux Ubuntu nodes, a Kubernetes cluster is going to be deployed using K3S (www.k3s.io) with the following specifications: PODs overlay subnet: 10.160.0.0/16 Services overlay subnet: 10.161.0.0/16 Default K3S CNI (flannel) will be disabled K3S CNI will be replaced by Cilium CNI to expose directly the PODs overlay subnet to the “external world” Figure 19: Self-managed K3S cluster with Cilium CNI and BGP peering to F5 XC CE What is running inside the K8S cluster? The PODs We have two PODs in the default namespace. Figure 20: The running PODs in the K8S cluster One running on node “k3s-1” with POD IP 10.160.0.203 and the other one running on node “k3s-2” with POD IP 10.160.1.208. microbot POD is a simple HTTP application that is returning the full name of the POD and an image. The services Figure 21: The services running in the K8S cluster Different Kubernetes services are created to expose the microbot PODs, one of type Cluster IP and the other one of type LoadBalancer. The type of service doesn’t really matter for F5XC because we are working in a full routed mode between the CE and the K8S cluster. F5XC only needs to “know” the PODs IPs, which will be discovered through the services. Configure F5 XC Kubernetes Service Discovery Steps are identical regarding what we did for EKS. And once done, services and PODs IPs are discovered by F5XC. Figure 22: The discovered PODs IPs Configure the BGP peering on F5XC CE In this example topology, BGP peerings are established directly between the K8S nodes and the F5 XC CE. Other implementations are possible, for instance, with an intermediate router. Figure 23: BGP peerings Check if the peerings are established. Figure 24: Verification of the BGP peerings Are the pods reachable from the F5XC CE? Figure 25: PODs reachability test They are! Create Origin Pool with K8S Service As we did for the EKS configuration, create an origin pool that references your Kubernetes service. Create an HTTPS Load-Balancer and test the service Just create a regular F5 XC HTTPS Load-Balancer and use the origin pool created above. Figure 26: Traffic load-balanced across the two PODs Scaling up? Let’s add another POD to the deployment to see how F5XC will handle the load-balancing after. Figure 27: Scaling up the Microbot PODs And it’s working! Load is spread automatically as soon as new PODs instances are available for the given service. Figure 28: Traffic load-balanced across the three PODs Appendix - K3S and Cilium deployment example Step 1: Install K3S without Default CNI On the master node: curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" \ INSTALL_K3S_EXEC="--flannel-backend=none \ --disable-network-policy \ --disable=traefik \ --disable servicelb \ --cluster-cidr=10.160.0.0/16 \ --service-cidr=10.161.0.0/16" sh - # Export kubeconfig export KUBECONFIG=/etc/rancher/k3s/k3s.yaml # Get token for worker nodes sudo cat /var/lib/rancher/k3s/server/node-token On worker nodes: IP_MASTER=10.154.1.10 K3S_TOKEN=<token-from-master> curl -sfL https://get.k3s.io | K3S_URL=https://${IP_MASTER}:6443 K3S_TOKEN=${K3S_TOKEN} sh - Step 2: Install and Configure Cilium On the K3S master node, please perform the following: Install Helm and Cilium CLI: # Install Helm sudo snap install helm --classic # Download Cilium CLI CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt) CLI_ARCH=amd64 curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum} sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin Install Cilium with BGP support: helm repo add cilium https://helm.cilium.io/ helm install cilium cilium/cilium --version 1.16.5 \ --set=ipam.operator.clusterPoolIPv4PodCIDRList="10.160.0.0/16" \ --set kubeProxyReplacement=true \ --set k8sServiceHost=10.154.1.10 \ --set k8sServicePort=6443 \ --set bgpControlPlane.enabled=true \ --namespace kube-system \ --set bpf.hostLegacyRouting=false \ --set bpf.masquerade=true # Monitor installation cilium status --wait Step 3: Configure BGP Peering Label nodes for BGP: kubectl label nodes k3s-1 bgp=true kubectl label nodes k3s-2 bgp=true Create BGP configuration: # BGP Cluster Config apiVersion: cilium.io/v2alpha1 kind: CiliumBGPClusterConfig metadata: name: cilium-bgp spec: nodeSelector: matchLabels: bgp: "true" bgpInstances: - name: "k3s-instance" localASN: 65001 peers: - name: "f5xc-ce" peerASN: 65002 peerAddress: 10.154.1.100 peerConfigRef: name: "cilium-peer" --- # BGP Peer Config apiVersion: cilium.io/v2alpha1 kind: CiliumBGPPeerConfig metadata: name: cilium-peer spec: timers: holdTimeSeconds: 9 keepAliveTimeSeconds: 3 gracefulRestart: enabled: true restartTimeSeconds: 15 families: - afi: ipv4 safi: unicast advertisements: matchLabels: advertise: "bgp" --- # BGP Advertisement apiVersion: cilium.io/v2alpha1 kind: CiliumBGPAdvertisement metadata: name: bgp-advertisements labels: advertise: bgp spec: advertisements: - advertisementType: "PodCIDR"Announcing F5 NGINX Gateway Fabric 2.0.0 with a New Distributed Architecture
Gateway Fabric 2.0 marks our transition into a distributed architecture that is highly scalable, secure, and flexible. This architecture also easily enables more advanced capabilities and prepares us for integration for observability and fleet management with F5 NGINX One. Here are the big highlights for this major release: Control and Data Plane Separation Multiple Gateway Support HTTP Request Mirror Listener Isolation As always, bug fixesAnnouncing F5 NGINX Gateway Fabric 2.0.0 with a New Distributed Architecture
Today, F5 NGINX Gateway Fabric is reaching an important milestone. The release of NGINX Gateway Fabric 2.0 marks our transition into a distributed architecture that is highly scalable, secure, and flexible. This architecture also easily enables more advanced capabilities and prepares us for integration for observability and fleet management with F5 NGINX One. Here are the big highlights for this major release: Control and Data Plane Separation Multiple Gateway Support HTTP Request Mirror Listener Isolation As always, bug fixes Data and Control Plane Separation Before 2.0, NGINX Gateway Fabric contained both the “control plane,” the container responsible for reading and applying configuration, and the “data plane,” the NGINX container where all traffic flows through, within the same pod. This meant if you scaled the NGINX Gateway Fabric pod, you would be forced to scale the data and control plane together. Now with our distributed architecture, the data plane deploys in a separate pod from the control plane. This allows us to enable more flexible use cases such as multiple gateways per control plane, a highly available control plane, and directly scaling NGINX replicas per Gateway. This makes NGINX Gateway Fabric much more resource efficient at scale. This change also improves NGINX Gateway Fabric’s security posture by limiting how much is accessible if a single pod is compromised. While NGINX Gateway Fabric and NGINX have always been secure by default, this architecture enables a two-tier defense against potential attacks: any security intrusion on the control plane has no way to directly access traffic, nor can the data plane directly access control plane interfaces. Multiple Gateways NGINX Gateway Fabric has historically been limited to a single Gateway object. We first chose this architecture for the short term because Routes are a good way to separate routing and access control for developers and cluster operators. As our product matures, we know that more advanced use cases require the isolation of infrastructure for separate teams, customers, or SLAs. Our new architecture enables you to do just that: In NGINX Gateway Fabric 2.0, every time you define a Gateway object, the control plane will provision an NGINX deployment, which can then be independently scaled by adding more replicas if needed. With this pattern, it’s easy to create a second, third, or many more Gateways. You can choose to give each customer or team in your cluster their own Gateway so they can each own their own infrastructure. Or you may want to separate infrastructure to apply separate policies based on hostname or product group. Each Gateway can also be scaled independently for varying levels of traffic, all managed by a single control plane. HTTP Request Mirrors Request mirrors are often useful when you want to mirror traffic to analyze traffic for security issues or to test a new version of an application with production traffic without impacting current users. All request responses to a mirrored location are ignored, so request mirrors serve as another useful tool for testing and analysis. These mirrors are added using a filter on the route rule. They will only affect a small group of traffic you choose. If you need more, you can add as many as you need. Listener Isolation We decided to also include the concept of listener isolation in this release to make advanced configurations more intuitive to work with and guard against accidental misconfiguration. Listener isolation means that any request should match at most one Listener within a Gateway. The Gateway API lists this example below: If Listeners are defined for "foo.example.com" and "*.example.com", a request to "foo.example.com" SHOULD only be routed using routes attached to the "foo.example.com" Listener and NOT the "*.example.com" Listener. The alternative is that request may match against multiple listeners unintentionally, which can become a problem when different policies are applied to the other listeners, or the request is routed to the wrong location. Now, NGINX Gateway Fabric will ensure that a Route will only match the most specific Listener on the Gateway it is attached to. What’s Next Our next release will be primarily focused around delivering more extended features from the Gateway API in an effort to support all extended features at the extended support level. If you are unfamiliar, the Gateway API has three separate support levels for every feature in the specification: Core: Portable features that all implementations should support. Extended: Portable features that are not universally supported across implementations. Implementations that support the feature should have the same behavior and semantics. Some extended features may eventually move to core. Implementation-specific: These features are not portable and vendor-specific. These features are far less defined in API and schema. You can see the full description of support levels here. So far, NGINX Gateway Fabric has supported all core features since 1.0 and has slowly been adding extended features over the past few releases. For 2.1, we will have a greater focus on these features to enable more advanced use cases. We plan to have all extended features available in NGINX Gateway Fabric by 2.2. For F5 NGINX Plus users, 2.1 will bring connection to the F5 NGINX One Console with basic fleet management capabilities. We plan on expanding these capabilities to include Observability for all NGINX deployments in your organization in the near future. 2.1 will also include support for F5 NGINX App Protect Web Application Firewall to protect your applications from any incoming malicious traffic. Specifically, NGINX Gateway Fabric will be implementing support for v5, with a new configuration experience coming later this year. Resources For the complete changelog for NGINX Gateway Fabric 2.0.0, see the Release Notes. To try NGINX Gateway Fabric for Kubernetes with NGINX Plus, start your free 30-day trial today or contact us to discuss your use cases. If you would like to get involved, see what is coming next, or see the source code for NGINX Gateway Fabric, check out our repository on GitHub! We have weekly community meetings on Tuesdays at 9:30AM Pacific/12:30PM Eastern/5:30PM GMT. Meeting links, updates, agenda, and notes are on the NGINX Gateway Fabric Meeting Calendar. Links are also always available from our GitHub readme.Kubernetes architecture options with F5 Distributed Cloud Services
Summary F5 Distributed Cloud Services (F5 XC) can both integrate with your existing Kubernetes (K8s) clusters and/or host a K8s workload itself. Within these distinctions, we have multiple architecture options. This article explores four major architectures in ascending order of sophistication and advantages. Architecture #1: External Load Balancer (Secure K8s Gateway) Architecture #2: CE as a pod (K8s site) Architecture #3: Managed Namespace (vK8s) Architecture #4: Managed K8s (mK8s) Kubernetes Architecture Options As K8s continues to grow, options for how we run K8s and integrate with existing K8s platforms continue to grow. F5 XC can both integrate with your existing K8s clusters and/or run a managed K8s platform itself. Multiple architectures exist within these offerings too, so I was thoroughly confused when I first heard about these possibilities. A colleague recently laid it out for me in a conversation: "Michael, listen up: XC can either integrate with your K8s platform, run inside your K8s platform, host virtual K8s (Namespace-aaS), or run a K8s platform in your environment." I replied, "That's great. Now I have a mental model for differentiating between architecture options." This article will overview these architectures and provide 101-level context: when, how, and why would you implement these options? Side note 1: F5 XC concepts and terms F5 XC is a global platform that can provide networking and app delivery services, as well as compute (K8s workloads). We call each of our global PoP's a Regional Edge (RE). RE's are highly meshed to form the backbone of the global platform. They connect your sites, they can expose your services to the Internet, and they can run workloads. This platform is extensible into your data center by running one or more XC Nodes in your network, also called a Customer Edge (CE). A CE is a compute node in your network that registers to our global control plane and is then managed by a customer as SaaS. The registration of one or more CE's creates a customer site in F5 XC. A CE can run on a hypervisor (VMWare/KVM/Etc), a Hyperscaler (AWS, Azure, GCP, etc), baremetal, or even as a k8s pod, and can be deployed in HA clusters. XC Mesh functionality provides connectivity between sites, security services, and observability. Optionally, in addition, XC App Stack functionality allows a large and arbitrary number of managed clusters to be logically grouped into a virtual site with a single K8s mgmt interface. So where Mesh services provide the networking, App Stack services provide the Kubernetes compute mgmt. Our first 2 architectures require Mesh services only, and our last two require App Stack. Side note 2: Service-to-service communication I'm often asked how to allow services between clusters to communicate with each other. This is possible and easy with XC. Each site can publish services to every other site, including K8s sites. This means that any K8s service can be reachable from other sites you choose. And this can be true in any of the architectures below, although more granular controls are possible with the more sophisticated architectures. I'll explore this common question more in a separate article. Architecture 1: External Load Balancer (Secure K8s Gateway) In a Secure Kubernetes Gateway architecture, you have integration with your existing K8s platform, using the XC node as the external load balancer for your K8s cluster. In this scenario, you create a ServiceAccount and kubeconfig file to configure XC. The XC node then performs service discovery against your K8s API server. I've covered this process in a previous article, but the advantage is that you can integrate with existing K8s platforms. This allows exposing both NodePort and ClusterIP services via the XC node. XC is not hosting any workloads in this architecture, but it is exposing your services to your local network, or remote sites, or the Internet. In the diagram above, I show a web application being accesssed from a remote site (and/or the Internet) where the origin pool is a NodePort service discovered in a K8s cluster. Architecture 2: Run a site within a K8s cluster (K8s site type) Creating a K8s site is easy - just deploy a single manifest found here. This file deploys multiple resources in your cluster, and together these resources work to provide the services of a CE, and create a customer site. I've heard this referred to as "running a CE inside of K8s" or "running your CE as a pod". However, when I say "CE node" I'm usually referring to a discreet compute node like a VM or piece of hardware; this architecture is actually a group of pods and related resources that run within K8s to create a XC customer site. With XC running inside your existing cluster, you can expose services within the cluster by DNS name because the site will resolve these from within the cluster. Your service can then be exposed anywhere by the F5 XC platform. This is similar to Architecture 1 above, but with this model, your site is simply a group of pods within K8s. An advantage here is the ability to expose services of other types (e.g. ClusterIP). A site deployed into a K8s cluster will only support Mesh functionality and does not support AppStack functionality (i.e., you cannot run a cluster within your cluster). In this architecture, XC acts as a K8s ingress controller with built-in application security. It also enables Mesh features, such as publishing of other sites' services on this site, and publishing of this site's discovered services on other sites. Architecture 3: vK8s (Namespace-as-a-Service) If the services you use include AppStack capabilities, then architectures #3 and #4 are possible for you. In these scenarios, our XC node actually runs your K8s on your workloads. We are no longer integrating XC with your existing K8s platform. XC is the platform. A simple way to run K8s workloads is to use a virtual k8s (vK8s) architecture. This could be referred to as a "managed Namespace" because by creating a vK8s object in XC you get a single namespace in a virtual cluster. Your Namespace can be fully hosted (deployed to RE's) or run on your VM's (CE's), or both. Your kubeconfig file will allow access to your Namespace via the hosted API server. Via your regular kubectl CLI (or via the web console) you can create/delete/manage K8s resources (Deployments, Services, Secrets, ServiceAccounts, etc) and view application resource metrics. This is great if you have workloads that you want to deploy to remote regions where you do not have infrastructure and would prefer to run in F5's RE's, or if you have disparate clusters across multiple sites and you'd like to manage multiple K8s clusters via a single centralized, virtual cluster. Best practice guard rails for vK8s With a vK8s architecture, you don't have your own cluster, but rather a managed Namespace. So there are some restrictions (for example, you cannot run a container as root, bind to a privileged port, or to the Host network). You cannot create CRD's, ClusterRoles, PodSecurityPolicies, or Namespaces, so K8s operators are not supported. In short, you don't have a managed cluster, but a managed Namespace on a virtual cluster. Architecture 4: mK8s (Managed K8s) In managed k8s (mk8s, also known as physical K8s or pk8s) deployment, we have an enterprise-level K8s distribution that is run at your site. This means you can use XC to deploy/manage/upgrade K8s infrastructure, but you manage the Kubernetes resources. The benefits include what is typical for 3rd-party K8s mgmt solutions, but also some key differentiators: multi-cloud, with automation for Azure, AWS, and GCP environments consumed by you as SaaS enterprise-level traffic control natively allows a large and arbitrary number of managed clusters to be logically managed with a single K8s mgmt interface You can enable kubectl access against your local cluster and disable the hosted API server, so your kubeconfig file can point to a global URL or a local endpoint on-prem. Another benefit of mK8s is that you are running a full K8s cluster at your site, not just a Namespace in a virtual cluster. The restrictions that apply to vK8s (see above) do not apply to mK8s, so you could run privileged pods if required, use Operators that make use of ClusterRoles and CRDs, and perform other tasks that require cluster-wide access. Traffic management controls with mK8s Because your workloads run in a cluster managed by XC, we can apply more sophisticated and native policies to K8s traffic than non-managed clusters in earlier architectures: Service isolation can be enforced within the cluster, so that pods in a given namespace cannot communicate with services outside of that namespace, by default. More service-to-service controls exist so that you can decide which services can reach with other services with more granularity. Egress control can be natively enforced for outbound traffic from the cluster, by namespace, labels, IP ranges, or other methods. E.g.: Svc A can reach myapi.example.com but no other Internet service. WAF policies, bot defense, L3/4 policies, etc—all of these policies that you have typically applied with network firewalls, WAF's, etc—can be applied natively within the platform. This architecture took me a long time to understand, and longer to fully appreciate. But once you have run your workloads natively on a managed K8s platform that is connected to a global backbone and capable of performing network and application delivery within the platform, the security and traffic mgmt benefits become very compelling. Conclusion: As K8s continues to expand, management solutions of your clusters make it possible to secure your K8s services, whether they are managed by XC or exist in disparate clusters. With F5 XC as a global platform consumed as a service—not a discreet installation managed by you—the available architectures here are unique and therefore can accommodate the diverse (and changing!) ways we see K8s run today. Related Articles Securely connecting Kubernetes Microservices with F5 Distributed Cloud Multi-cluster Multi-cloud Networking for K8s with F5 Distributed Cloud - Architecture Pattern Multiple Kubernetes Clusters and Path-Based Routing with F5 Distributed CloudAnnouncing F5 NGINX Gateway Fabric 1.5.0 with NGINX Code Snippets
Today we are announcing the release of F5 NGINX Gateway Fabric 1.5.0 which comes with one of the most prominent features from F5 NGINX Ingress Controller: code snippets. As always, we have made this feature available through an extension of the Gateway API, a new resource called a SnippetFilter. With it, you can define custom NGINX configuration that can be applied to any Route rule! In addition to SnippetFilters, some other highlights of this release include: Ability to retain client IP information via Proxy Pass or XForwardedFor header Configurable NGINX error log level and reduced “info” log verbosity A new Getting Started guide for installing NGF for the first time F5 NGINX Plus R33 support Bug fixes! You can see the full changelog with details here! NGINX Code Snippets via SnippetFilters With a 20 year old legacy, there is a massive range of features that F5 NGINX provides that are not yet explicitly exposed through NGINX Gateway Fabric via the Gateway API. If you are familiar with NGINX directives and configuration and just want to use a native NGINX feature, you were unable to because our control plane would overwrite any configuration you wrote. We needed a way for your directives to merge with the configuration we pushed. With SnippetFilters, we have not only enabled you to append directives to whatever context you need them in but also built them as an extension to the Gateway API. Once you define a SnippetFilter, any route rule can then use that snippet filter to apply NGINX configuration for the context you supplied. You can even change higher level contexts using these snippets, such as main, to do things like importing modules – just make sure you coordinate with everyone when making these changes! As SnippetFilters do open the opportunity for misconfiguration in a way that may impact other application teams using the same Gateway, they do have to be explicitly enabled. But if you find yourself needing a feature from NGINX that is not yet available in NGINX Gateway Fabric, or you want to include your own NGINX logic, the SnippetFilter will fill that gap between our current first-class Gateway API features and the full power of NGINX. What’s Next Next release we will be focusing on a big architectural shift in NGINX Gateway Fabric that we have been planning for a while: separating our data and control planes. Currently, both our control plane (what configures NGINX for us) and our data plane (NGINX itself) are contained within the same pod. Ideally, these should be separate for security, performance, and scalability reasons. As we are also looking at supporting multiple Gateway resources for a single NGINX Gateway Fabric installation, we thought this was a good time to make that change, prompting our next release to be 2.0! In 2.0, you’ll see the control plane start as its own pod. As you create Gateways, or as you scale existing ones, the control plane will spin up new instances of NGINX in their own pods. This translates into less overhead, better RBAC security, and the ability to provision multiple Gateway resources. For more information, check out our design on the separation! Resources For the complete changelog for NGINX Gateway Fabric 1.5.0, see the Release Notes. To try NGINX Gateway Fabric for Kubernetes with NGINX Plus, start your free 30-day trial today or contact us to discuss your use cases. If you would like to get involved, see what is coming next, or see the source code for NGINX Gateway Fabric, check out our repository on GitHub! We have weekly community meetings on Tuesdays at 9:30AM Pacific/12:30PM Eastern/5:30PM GMT. The meeting link, updates, agenda, and notes are on the NGINX Gateway Fabric Meeting Calendar. Links are also always available from our GitHub README.5 Technical Sessions That Should Be Great: F5 AppWorld 2025
These F5 Academy sessions explore modern app delivery, security, and operations. The full list of sessions is on the F5 AppWorld 2025 Academy page - if you haven't yet registered you can do so here: Register for F5 AppWorld 2025 LAB - F5 Distributed Cloud: Discovering & Securing APIs API security has never been more critical, and this lab dives straight into the tough stuff. Learn how to find hidden endpoints, detect sensitive data and authentication states, and apply integrated API security measures to keep your environment locked tight. TECHNICAL BRIEFING - LLM Security and Delivery with F5’s Distributed Cloud Security Ecosystem AI is fueling the next wave of applications—but it’s also introducing new security blind spots. This briefing explores how to secure LLMs and integrate the right solutions to ensure your AI-driven workloads remain fast, cost-effective, and protected. LAB - F5 NGINX Plus Ingress as an API Gateway for Kubernetes Containerized environments and microservices are here to stay, and this lab helps you navigate the complexity. Configure NGINX Plus Ingress as a powerful API gateway for your Kubernetes workloads, enabling schema enforcement, authorization, and rate-limiting all in one streamlined solution. LAB - Zero Trust at Scale With F5 NGINX Zero trust principles become a whole lot more meaningful when you can scale them. Get hands-on with NGINX Plus and BIG-IP GTM to build a robust, scalable zero trust architecture, ensuring secure and seamless app access across enterprises and multi-cluster Kubernetes environments. LAB - F5 Distributed Cloud: Security Automation & Zero Day Mitigation In this lab, you’ll learn how to leverage advanced matching criteria and custom rules to quickly respond to emerging threats. Shore up your defenses with automated policies that deliver frictionless security and agile zero-day mitigation. Session Updates Coming in January 🚨 AppWorld's Breakout Sessions officially drop in January 2025 but here is a sneak preview! Check back in January to add these to your agenda. Global App Delivery With a Global Network How Generative AI Breaks Traditional Application Security and What You Can Do About It The New Wave of Bots: A Deep Dive into Residential IP Proxy Networks From ZTNA to Universal ZTNA: Expanding Your App Security Strategy --- See you at F5 AppWorld 2025! #AppWorld25NGINX Unit running in Distributed Cloud vK8s
F5 Distributed Cloud provides a mechanism to easily deploy applications using virtual Kubernetes (vK8s) across a global network. This results in applications running closer to the end user. This article will demonstrate creating a NGINX Unit container to run a simple server-side WebAssembly (Wasm) application and deliver it via Distributed Cloud Regional Edge (RE) sites. Distributed Cloud provides a vK8s infrastructure for deploying modern apps as well as load balancing and security services to securely and reliably deliver applications at the edge. NGINX Unit Pod A Kubernetes pod is the smallest execution unit that can be deployed in Kubernetes. A pod is a single instance of an application and may contain single or multiple containers. For this article, a single NGINX Unit container will make up the application pod and replicas of the pod will be deployed to each of the configured Distributed Cloud Points of Presence (PoPs). Building the NGINX Unit Container The first step is to build a server-side WebAssemby (Wasm) application that Unit can execute. Instructions on creating a simple Hello World Wasm application are available here. After the Hello World Wasm component is built, a NGINX Unit Container needs to be created to execute the application. To build the NGINX Unit container, download the Unit Wasm Dockerfile that is available on the NGINX Unit github repo. The Dockerfile needs to be modified to allow the container to run as a non-root user in the Distributed Cloud virtual Kubernetes (vK8s) environment. Below is the Dockerfile that I used for this demonstration. FROM debian:bullseye-slim LABEL org.opencontainers.image.title="Unit (wasm)" LABEL org.opencontainers.image.description="Official build of Unit for Docker." LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" LABEL org.opencontainers.image.version="1.32.1" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates git build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ && mkdir -p /usr/src/unit \ && mkdir -p /unit/var/lib/unit \ && mkdir -p /unit/var/run \ && mkdir -p /unit/var/log \ && mkdir -p /unit/var/tmp \ && mkdir /app \ && chmod -R 777 /unit/var/lib/unit \ && chmod -R 777 /unit/var/run \ && chmod -R 777 /unit/var/log \ && chmod -R 777 /unit/var/tmp \ && chmod 777 /app \ && cd /usr/src/unit \ && git clone --depth 1 -b 1.32.1-1 https://github.com/nginx/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ && CONFIGURE_ARGS_MODULES="--prefix=/usr \ --statedir=/unit/var/lib/unit \ --debug \ --control=unix:/unit/var/run/control.unit.sock \ --runstatedir=/unit/var/run \ --pid=/unit/var/run/unit.pid \ --logdir=/unit/var/log \ --log=/unit/var/log/unit.log \ --tmpdir=/unit/var/tmp \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ --njs" \ && make -j $NCPU -C pkg/contrib .njs \ && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ && apt-get install --no-install-recommends --no-install-suggests -y libclang-dev \ && export RUST_VERSION=1.76.0 \ && export RUSTUP_HOME=/usr/src/unit/rustup \ && export CARGO_HOME=/usr/src/unit/cargo \ && export PATH=/usr/src/unit/cargo/bin:$PATH \ && dpkgArch="$(dpkg --print-architecture)" \ && case "${dpkgArch##*-}" in \ amd64) rustArch="x86_64-unknown-linux-gnu"; rustupSha256="0b2f6c8f85a3d02fde2efc0ced4657869d73fccfce59defb4e8d29233116e6db" ;; \ arm64) rustArch="aarch64-unknown-linux-gnu"; rustupSha256="673e336c81c65e6b16dcdede33f4cc9ed0f08bde1dbe7a935f113605292dc800" ;; \ *) echo >&2 "unsupported architecture: ${dpkgArch}"; exit 1 ;; \ esac \ && url="https://static.rust-lang.org/rustup/archive/1.26.0/${rustArch}/rustup-init" \ && curl -L -O "$url" \ && echo "${rustupSha256} *rustup-init" | sha256sum -c - \ && chmod +x rustup-init \ && ./rustup-init -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION --default-host ${rustArch} \ && rm rustup-init \ && rustup --version \ && cargo --version \ && rustc --version \ && make -C pkg/contrib .wasmtime \ && install -pm 755 pkg/contrib/wasmtime/target/release/libwasmtime.so /usr/lib/$(dpkg-architecture -q DEB_HOST_MULTIARCH)/ \ && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure wasm --include-path=`pwd`/pkg/contrib/wasmtime/crates/c-api/include --lib-path=/usr/lib/$(dpkg-architecture -q DEB_HOST_MULTIARCH)/ && ./configure wasm-wasi-component \ && make -j $NCPU wasm-install wasm-wasi-component-install \ && make clean \ && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure wasm --include-path=`pwd`/pkg/contrib/wasmtime/crates/c-api/include --lib-path=/usr/lib/$(dpkg-architecture -q DEB_HOST_MULTIARCH)/ && ./configure wasm-wasi-component \ && make -j $NCPU wasm-install wasm-wasi-component-install \ && cd \ && rm -rf /usr/src/unit \ && for f in /usr/sbin/unitd /usr/lib/unit/modules/*.unit.so; do \ ldd $f | awk '/=>/{print $(NF-1)}' | while read n; do dpkg-query -S $n; done | sed 's/^\([^:]\+\):.*$/\1/' | sort | uniq >> /requirements.apt; \ done \ && apt-mark showmanual | xargs apt-mark auto > /dev/null \ && { [ -z "$savedAptMark" ] || apt-mark manual $savedAptMark; } \ && /bin/true \ && mkdir -p /var/lib/unit/ \ && mkdir -p /docker-entrypoint.d/ \ && apt-get update \ && apt-get --no-install-recommends --no-install-suggests -y install curl $(cat /requirements.apt) \ && apt-get purge -y --auto-remove build-essential \ && rm -rf /var/lib/apt/lists/* \ && rm -f /requirements.apt \ && ln -sf /dev/stderr /var/log/unit.log WORKDIR /app COPY --chmod=755 hello_wasi_http.wasm /app COPY ./*.json /docker-entrypoint.d/ COPY --chmod=755 docker-entrypoint.sh /usr/local/bin/ COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] EXPOSE 8000 CMD ["unitd", "--no-daemon", "--control", "unix:/unit/var/run/control.unit.sock"] The changes to the Dockerfile are due to the non-root user not being able to write to the /var directory. Rather than modify the permissions on /var and its subdirectories, this docker file creates a /unit directory that is used for storing process information and logs. The docker-entrypoint.sh script also needed to be modified to use the /unit directory structure. The updated docker-entrypoint.sh script is shown below. #!/bin/sh set -e WAITLOOPS=5 SLEEPSEC=1 curl_put() { RET=$(/usr/bin/curl -s -w '%{http_code}' -X PUT --data-binary @$1 --unix-socket /unit/var/run/control.unit.sock http://localhost/$2) RET_BODY=$(echo $RET | /bin/sed '$ s/...$//') RET_STATUS=$(echo $RET | /usr/bin/tail -c 4) if [ "$RET_STATUS" -ne "200" ]; then echo "$0: Error: HTTP response status code is '$RET_STATUS'" echo "$RET_BODY" return 1 else echo "$0: OK: HTTP response status code is '$RET_STATUS'" echo "$RET_BODY" fi return 0 } if [ "$1" = "unitd" ] || [ "$1" = "unitd-debug" ]; then if /usr/bin/find "/unit/var/lib/unit/" -mindepth 1 -print -quit 2>/dev/null | /bin/grep -q .; then echo "$0: /unit/var/lib/unit/ is not empty, skipping initial configuration..." else echo "$0: Launching Unit daemon to perform initial configuration..." /usr/sbin/$1 --control unix:/unit/var/run/control.unit.sock for i in $(/usr/bin/seq $WAITLOOPS); do if [ ! -S /unit/var/run/control.unit.sock ]; then echo "$0: Waiting for control socket to be created..." /bin/sleep $SLEEPSEC else break fi done # even when the control socket exists, it does not mean unit has finished initialisation # this curl call will get a reply once unit is fully launched /usr/bin/curl -s -X GET --unix-socket /unit/var/run/control.unit.sock http://localhost/ if /usr/bin/find "/docker-entrypoint.d/" -mindepth 1 -print -quit 2>/dev/null | /bin/grep -q .; then echo "$0: /docker-entrypoint.d/ is not empty, applying initial configuration..." echo "$0: Looking for certificate bundles in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.pem"); do echo "$0: Uploading certificates bundle: $f" curl_put $f "certificates/$(basename $f .pem)" done echo "$0: Looking for JavaScript modules in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.js"); do echo "$0: Uploading JavaScript module: $f" curl_put $f "js_modules/$(basename $f .js)" done echo "$0: Looking for configuration snippets in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.json"); do echo "$0: Applying configuration $f"; curl_put $f "config" done echo "$0: Looking for shell scripts in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.sh"); do echo "$0: Launching $f"; "$f" done # warn on filetypes we don't know what to do with for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -not -name "*.sh" -not -name "*.json" -not -name "*.pem" -not -name "*.js"); do echo "$0: Ignoring $f"; done else echo "$0: /docker-entrypoint.d/ is empty, creating 'welcome' configuration..." curl_put /usr/share/unit/welcome/welcome.json "config" fi echo "$0: Stopping Unit daemon after initial configuration..." kill -TERM $(/bin/cat /unit/var/run/unit.pid) for i in $(/usr/bin/seq $WAITLOOPS); do if [ -S /unit/var/run/control.unit.sock ]; then echo "$0: Waiting for control socket to be removed..." /bin/sleep $SLEEPSEC else break fi done if [ -S /unit/var/run/control.unit.sock ]; then kill -KILL $(/bin/cat /unit/var/run/unit.pid) rm -f /unit/var/run/control.unit.sock fi echo echo "$0: Unit initial configuration complete; ready for start up..." echo fi fi exec "$@" The Dockerfile copies all files ending in .json to a directory named /docker-entrypoint.d. The docker-entrypoint.sh script checks the /docker-entrypoint.d directory for configuration files that are used to configure NGINX Unit. The following config.json file was created in the same directory as the Dockerfile and docker-entrypoint.sh script. { "listeners": { "*:8000": { "pass": "applications/wasm" } }, "applications": { "wasm": { "type": "wasm-wasi-component", "component": "/app/hello_wasi_http.wasm" } } } This config file instructs NGINX unit to listen on port 8000 and pass any client request to the Wasm application component located at /app/hello_wasi_http.wasm. The last step before building the container is to copy the Wasm application component into the same directory as the Dockerfile, docker-entrypoint.sh, and config.json files. With these four files in the same directory, the docker build command is used to create a container: docker build --no-cache --platform linux/amd64 -t hello-wasi-xc:1.0 . Here I use the --platform option to specify that the container will run on a linux/amd64 platform. I also use -t to give the image a name. The portion of the name after the ":" can be used for versioning. For example, if I added a new feature to my application, I could create a new image with -t hello-wasi-xc:1.1 to represent version 1.1 of the hello-wasi-xc application. Once the image is built, it can be uploaded to the container registry of your choice. I have access to an Azure Container Registry, so that is what I chose to use. Distributed Cloud supports pulling images from both public and private container registries. To push the image to my registry, I had to first tag the image with the registry location ({{registry_name}} should be replaced with the name of your registry): docker tag hello-wasi-xc:1.0 {{registry_name}}/examples/hello-wasi-xc:1.0 Next, I logged into my Azure registry: az login az acr login --name {{registry_name}} I then pushed the image to the registry: docker push {{registry_name}}/examples/hello-wasi-xc:1.0 F5 Distributed Cloud Virtual Kubernetes F5 Distributed Cloud Services support a Kubernetes compatible API for centralized orchestration of applications across a fleet of sites (customer sites or F5 Distributed Cloud Regional Edges). Distributed Cloud utilizes a distributed control plane to manage scheduling and scaling of applications across multiple sites. Kubernetes Objects Virtual Kubernetes supports the following Kubernetes objects: Deployments, StatefulSets, Jobs, CronJob, DaemonSet, Service, ConfigMap, Secrets, PersistentVolumeClaim, ServiceAccount, Role, and Role Binding. For this article I will focus on the Deployments object. A deployment is commonly used for stateless applications like web-servers, front-end web-ui, etc. A deployment works well for this use case because the NGINX Unit container is serving up a stateless web application. Workload Object A workload within Distributed Cloud is used to configure and deploy components of an application in Virtual Kubernetes. Workload encapsulates all the operational characteristics of Kubernetes workload, storage, and network objects (deployments, statefulsets, jobs, persistent volume claims, configmaps, secrets, and services) configuration, as well as configuration related to where the workload is deployed and how it is advertised using L7 or L4 load balancers. Within Distributed Cloud there are four types of workloads: Simple Service, Service, Stateful Service, and Job. Namespaces In Distributed Cloud, tenant configuration objects are grouped under namespaces. Namespaces can be thought of as administrative domains. Within each Namespace, a user can create an object called vK8s and use that for application management. Each Namespace can have a maximum of one vK8s object. Distributed Cloud vK8s Configuration vK8s is configured under the Distributed Apps tile within the Distributed Cloud console. After clicking on that tile, I next created a new Virtual K8s. By clicking Add Virtual K8s from the Virtual K8s menu under Applications. This brings up the Virtual K8s configuration form. I provided a name for my vK8s site and click Save and Exit. This initiates the vK8s build. Deploy the Application Once the vK8s cluster is ready, I click on the name of my vk8s to configure the deployment of my application. For this demo, I am going to deploy my application via Workloads. I click on Workloads and then Add VK8s Workload. This brings up the Workload configuration form. I provide my Workload a name, select Service for Workload type and click Configure. Using Service allows me to have more granular controls on how my application will be advertised. Next, I click Add Item under containers to specify to Distribtued Cloud where it can pull my container image from. In the resulting form, I supply a name for my container, supply the public registry FQDN along with the container name and version and click Apply. Next I configure where to advertise the application. I want to manually configure my HTTP LB, so I chose to advertise in Cluster. Manually configuring the LB allows me to configure additional options such as Web Application Firewall (WAF). Within the Advertise in Cluster form, I specify port 8000, because that is the port my container is listening on. Click Apply and then click Save and Exit. Create an HTTP Load Balancer to Advertise the Application The next step is to create an HTTP Load Balancer to expose the application to the Internet. This can be done by clicking Manage, selecting Load Balancers, and clicking on HTTP Load Balancers. Next, click on Add HTTP Load Balancer to create a new load balancer. I gave the LB a Name, Domain, Load Balancer Type, and Checked the box to Automatically Manage DNS Records. Then I configured the Origin Pool by clicking Add Item in the Origin Pool section and then clicking Add Item under the Origin Pool. On the resulting form, I created a name for my Origin Pool and then clicked Add Item under Origin Servers to add an origin Server. In the Origin Server form, I selected K8s Service Name of Origin Server on given Sites, Service Name, and provided my vK8s workload name along with my namespace. I also selected Virtual Site and created a Virtual Site for the REs I wanted to use to access my application and clicked Continue. Back on the Origin Server form, I selected vK8s Networks on Site from the Select Network on the site drop down and then clicked Apply. On the Origin Pool form, I specified port 8000 for my origin server Port and then clicked Continue. I then clicked Apply to return to the LB configuration form. I added a WAF policy to protect my application and then clicked Save and Exit. At this point my application is now accessible on the domain name I specified in the HTTP Load Balancer config (http://aconley-wasm.amer-ent.f5demos.com/). Conclusion This is a basic demo of using NGINX Unit and Distributed Cloud Virtual Kubernetes to deploy a server-side WASM application. The principals in this demo could be expanded to build more complex applications.
