kubernetes

46 Topics

Kubernetes architecture options with F5 Distributed Cloud Services
Summary F5 Distributed Cloud Services (F5 XC) can both integrate with your existing Kubernetes (K8s) clusters and/or host a K8s workload itself. Within these distinctions, we have multiple architecture options. This article explores four major architectures in ascending order of sophistication and advantages. Architecture #1: External Load Balancer (Secure K8s Gateway) Architecture #2: CE as a pod (K8s site) Architecture #3: Managed Namespace (vK8s) Architecture #4: Managed K8s (mK8s) Kubernetes Architecture Options As K8s continues to grow, options for how we run K8s and integrate with existing K8s platforms continue to grow. F5 XC can both integrate with your existing K8s clusters and/or run a managed K8s platform itself. Multiple architectures exist within these offerings too, so I was thoroughly confused when I first heard about these possibilities. A colleague recently laid it out for me in a conversation: "Michael, listen up: XC can either integrate with your K8s platform, run inside your K8s platform, host virtual K8s (Namespace-aaS), or run a K8s platform in your environment." I replied, "That's great. Now I have a mental model for differentiating between architecture options." This article will overview these architectures and provide 101-level context: when, how, and why would you implement these options? Side note 1: F5 XC concepts and terms F5 XC is a global platform that can provide networking and app delivery services, as well as compute (K8s workloads). We call each of our global PoP's a Regional Edge (RE). RE's are highly meshed to form the backbone of the global platform. They connect your sites, they can expose your services to the Internet, and they can run workloads. This platform is extensible into your data center by running one or more XC Nodes in your network, also called a Customer Edge (CE). A CE is a compute node in your network that registers to our global control plane and is then managed by a customer as SaaS. The registration of one or more CE's creates a customer site in F5 XC. A CE can run on a hypervisor (VMWare/KVM/Etc), a Hyperscaler (AWS, Azure, GCP, etc), baremetal, or even as a k8s pod, and can be deployed in HA clusters. XC Mesh functionality provides connectivity between sites, security services, and observability. Optionally, in addition, XC App Stack functionality allows a large and arbitrary number of managed clusters to be logically grouped into a virtual site with a single K8s mgmt interface. So where Mesh services provide the networking, App Stack services provide the Kubernetes compute mgmt. Our first 2 architectures require Mesh services only, and our last two require App Stack. Side note 2: Service-to-service communication I'm often asked how to allow services between clusters to communicate with each other. This is possible and easy with XC. Each site can publish services to every other site, including K8s sites. This means that any K8s service can be reachable from other sites you choose. And this can be true in any of the architectures below, although more granular controls are possible with the more sophisticated architectures. I'll explore this common question more in a separate article. Architecture 1: External Load Balancer (Secure K8s Gateway) In a Secure Kubernetes Gateway architecture, you have integration with your existing K8s platform, using the XC node as the external load balancer for your K8s cluster. In this scenario, you create a ServiceAccount and kubeconfig file to configure XC. The XC node then performs service discovery against your K8s API server. I've covered this process in a previous article, but the advantage is that you can integrate with existing K8s platforms. This allows exposing both NodePort and ClusterIP services via the XC node. XC is not hosting any workloads in this architecture, but it is exposing your services to your local network, or remote sites, or the Internet. In the diagram above, I show a web application being accesssed from a remote site (and/or the Internet) where the origin pool is a NodePort service discovered in a K8s cluster. Architecture 2: Run a site within a K8s cluster (K8s site type) Creating a K8s site is easy - just deploy a single manifest found here. This file deploys multiple resources in your cluster, and together these resources work to provide the services of a CE, and create a customer site. I've heard this referred to as "running a CE inside of K8s" or "running your CE as a pod". However, when I say "CE node" I'm usually referring to a discreet compute node like a VM or piece of hardware; this architecture is actually a group of pods and related resources that run within K8s to create a XC customer site. With XC running inside your existing cluster, you can expose services within the cluster by DNS name because the site will resolve these from within the cluster. Your service can then be exposed anywhere by the F5 XC platform. This is similar to Architecture 1 above, but with this model, your site is simply a group of pods within K8s. An advantage here is the ability to expose services of other types (e.g. ClusterIP). A site deployed into a K8s cluster will only support Mesh functionality and does not support AppStack functionality (i.e., you cannot run a cluster within your cluster). In this architecture, XC acts as a K8s ingress controller with built-in application security. It also enables Mesh features, such as publishing of other sites' services on this site, and publishing of this site's discovered services on other sites. Architecture 3: vK8s (Namespace-as-a-Service) If the services you use include AppStack capabilities, then architectures #3 and #4 are possible for you. In these scenarios, our XC node actually runs your K8s on your workloads. We are no longer integrating XC with your existing K8s platform. XC is the platform. A simple way to run K8s workloads is to use a virtual k8s (vK8s) architecture. This could be referred to as a "managed Namespace" because by creating a vK8s object in XC you get a single namespace in a virtual cluster. Your Namespace can be fully hosted (deployed to RE's) or run on your VM's (CE's), or both. Your kubeconfig file will allow access to your Namespace via the hosted API server. Via your regular kubectl CLI (or via the web console) you can create/delete/manage K8s resources (Deployments, Services, Secrets, ServiceAccounts, etc) and view application resource metrics. This is great if you have workloads that you want to deploy to remote regions where you do not have infrastructure and would prefer to run in F5's RE's, or if you have disparate clusters across multiple sites and you'd like to manage multiple K8s clusters via a single centralized, virtual cluster. Best practice guard rails for vK8s With a vK8s architecture, you don't have your own cluster, but rather a managed Namespace. So there are some restrictions (for example, you cannot run a container as root, bind to a privileged port, or to the Host network). You cannot create CRD's, ClusterRoles, PodSecurityPolicies, or Namespaces, so K8s operators are not supported. In short, you don't have a managed cluster, but a managed Namespace on a virtual cluster. Architecture 4: mK8s (Managed K8s) In managed k8s (mk8s, also known as physical K8s or pk8s) deployment, we have an enterprise-level K8s distribution that is run at your site. This means you can use XC to deploy/manage/upgrade K8s infrastructure, but you manage the Kubernetes resources. The benefits include what is typical for 3rd-party K8s mgmt solutions, but also some key differentiators: multi-cloud, with automation for Azure, AWS, and GCP environments consumed by you as SaaS enterprise-level traffic control natively allows a large and arbitrary number of managed clusters to be logically managed with a single K8s mgmt interface You can enable kubectl access against your local cluster and disable the hosted API server, so your kubeconfig file can point to a global URL or a local endpoint on-prem. Another benefit of mK8s is that you are running a full K8s cluster at your site, not just a Namespace in a virtual cluster. The restrictions that apply to vK8s (see above) do not apply to mK8s, so you could run privileged pods if required, use Operators that make use of ClusterRoles and CRDs, and perform other tasks that require cluster-wide access. Traffic management controls with mK8s Because your workloads run in a cluster managed by XC, we can apply more sophisticated and native policies to K8s traffic than non-managed clusters in earlier architectures: Service isolation can be enforced within the cluster, so that pods in a given namespace cannot communicate with services outside of that namespace, by default. More service-to-service controls exist so that you can decide which services can reach with other services with more granularity. Egress control can be natively enforced for outbound traffic from the cluster, by namespace, labels, IP ranges, or other methods. E.g.: Svc A can reach myapi.example.com but no other Internet service. WAF policies, bot defense, L3/4 policies, etc—all of these policies that you have typically applied with network firewalls, WAF's, etc—can be applied natively within the platform. This architecture took me a long time to understand, and longer to fully appreciate. But once you have run your workloads natively on a managed K8s platform that is connected to a global backbone and capable of performing network and application delivery within the platform, the security and traffic mgmt benefits become very compelling. Conclusion: As K8s continues to expand, management solutions of your clusters make it possible to secure your K8s services, whether they are managed by XC or exist in disparate clusters. With F5 XC as a global platform consumed as a service—not a discreet installation managed by you—the available architectures here are unique and therefore can accommodate the diverse (and changing!) ways we see K8s run today. Related Articles Securely connecting Kubernetes Microservices with F5 Distributed Cloud Multi-cluster Multi-cloud Networking for K8s with F5 Distributed Cloud - Architecture Pattern Multiple Kubernetes Clusters and Path-Based Routing with F5 Distributed Cloud
MichaelOLeary
Jan 15, 2025 Place Technical Articles
9.7KViews
29likes
5Comments
Announcing F5 NGINX Gateway Fabric 1.5.0 with NGINX Code Snippets
Today we are announcing the release of F5 NGINX Gateway Fabric 1.5.0 which comes with one of the most prominent features from F5 NGINX Ingress Controller: code snippets. As always, we have made this feature available through an extension of the Gateway API, a new resource called a SnippetFilter. With it, you can define custom NGINX configuration that can be applied to any Route rule! In addition to SnippetFilters, some other highlights of this release include: Ability to retain client IP information via Proxy Pass or XForwardedFor header Configurable NGINX error log level and reduced “info” log verbosity A new Getting Started guide for installing NGF for the first time F5 NGINX Plus R33 support Bug fixes! You can see the full changelog with details here! NGINX Code Snippets via SnippetFilters With a 20 year old legacy, there is a massive range of features that F5 NGINX provides that are not yet explicitly exposed through NGINX Gateway Fabric via the Gateway API. If you are familiar with NGINX directives and configuration and just want to use a native NGINX feature, you were unable to because our control plane would overwrite any configuration you wrote. We needed a way for your directives to merge with the configuration we pushed. With SnippetFilters, we have not only enabled you to append directives to whatever context you need them in but also built them as an extension to the Gateway API. Once you define a SnippetFilter, any route rule can then use that snippet filter to apply NGINX configuration for the context you supplied. You can even change higher level contexts using these snippets, such as main, to do things like importing modules – just make sure you coordinate with everyone when making these changes! As SnippetFilters do open the opportunity for misconfiguration in a way that may impact other application teams using the same Gateway, they do have to be explicitly enabled. But if you find yourself needing a feature from NGINX that is not yet available in NGINX Gateway Fabric, or you want to include your own NGINX logic, the SnippetFilter will fill that gap between our current first-class Gateway API features and the full power of NGINX. What’s Next Next release we will be focusing on a big architectural shift in NGINX Gateway Fabric that we have been planning for a while: separating our data and control planes. Currently, both our control plane (what configures NGINX for us) and our data plane (NGINX itself) are contained within the same pod. Ideally, these should be separate for security, performance, and scalability reasons. As we are also looking at supporting multiple Gateway resources for a single NGINX Gateway Fabric installation, we thought this was a good time to make that change, prompting our next release to be 2.0! In 2.0, you’ll see the control plane start as its own pod. As you create Gateways, or as you scale existing ones, the control plane will spin up new instances of NGINX in their own pods. This translates into less overhead, better RBAC security, and the ability to provision multiple Gateway resources. For more information, check out our design on the separation! Resources For the complete changelog for NGINX Gateway Fabric 1.5.0, see the Release Notes. To try NGINX Gateway Fabric for Kubernetes with NGINX Plus, start your free 30-day trial today or contact us to discuss your use cases. If you would like to get involved, see what is coming next, or see the source code for NGINX Gateway Fabric, check out our repository on GitHub! We have weekly community meetings on Tuesdays at 9:30AM Pacific/12:30PM Eastern/5:30PM GMT. The meeting link, updates, agenda, and notes are on the NGINX Gateway Fabric Meeting Calendar. Links are also always available from our GitHub README.
mpstefan
Dec 30, 2024 Place Technical Articles
100Views
0likes
0Comments
NGINX Unit running in Distributed Cloud vK8s
F5 Distributed Cloud provides a mechanism to easily deploy applications using virtual Kubernetes (vK8s) across a global network. This results in applications running closer to the end user. This article will demonstrate creating a NGINX Unit container to run a simple server-side WebAssembly (Wasm) application and deliver it via Distributed Cloud Regional Edge (RE) sites. Distributed Cloud provides a vK8s infrastructure for deploying modern apps as well as load balancing and security services to securely and reliably deliver applications at the edge. NGINX Unit Pod A Kubernetes pod is the smallest execution unit that can be deployed in Kubernetes. A pod is a single instance of an application and may contain single or multiple containers. For this article, a single NGINX Unit container will make up the application pod and replicas of the pod will be deployed to each of the configured Distributed Cloud Points of Presence (PoPs). Building the NGINX Unit Container The first step is to build a server-side WebAssemby (Wasm) application that Unit can execute. Instructions on creating a simple Hello World Wasm application are available here. After the Hello World Wasm component is built, a NGINX Unit Container needs to be created to execute the application. To build the NGINX Unit container, download the Unit Wasm Dockerfile that is available on the NGINX Unit github repo. The Dockerfile needs to be modified to allow the container to run as a non-root user in the Distributed Cloud virtual Kubernetes (vK8s) environment. Below is the Dockerfile that I used for this demonstration. FROM debian:bullseye-slim LABEL org.opencontainers.image.title="Unit (wasm)" LABEL org.opencontainers.image.description="Official build of Unit for Docker." LABEL org.opencontainers.image.url="https://unit.nginx.org" LABEL org.opencontainers.image.source="https://github.com/nginx/unit" LABEL org.opencontainers.image.documentation="https://unit.nginx.org/installation/#docker-images" LABEL org.opencontainers.image.vendor="NGINX Docker Maintainers <docker-maint@nginx.com>" LABEL org.opencontainers.image.version="1.32.1" RUN set -ex \ && savedAptMark="$(apt-mark showmanual)" \ && apt-get update \ && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates git build-essential libssl-dev libpcre2-dev curl pkg-config \ && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \ && mkdir -p /usr/src/unit \ && mkdir -p /unit/var/lib/unit \ && mkdir -p /unit/var/run \ && mkdir -p /unit/var/log \ && mkdir -p /unit/var/tmp \ && mkdir /app \ && chmod -R 777 /unit/var/lib/unit \ && chmod -R 777 /unit/var/run \ && chmod -R 777 /unit/var/log \ && chmod -R 777 /unit/var/tmp \ && chmod 777 /app \ && cd /usr/src/unit \ && git clone --depth 1 -b 1.32.1-1 https://github.com/nginx/unit \ && cd unit \ && NCPU="$(getconf _NPROCESSORS_ONLN)" \ && DEB_HOST_MULTIARCH="$(dpkg-architecture -q DEB_HOST_MULTIARCH)" \ && CC_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_CFLAGS_MAINT_APPEND="-Wp,-D_FORTIFY_SOURCE=2 -fPIC" dpkg-buildflags --get CFLAGS)" \ && LD_OPT="$(DEB_BUILD_MAINT_OPTIONS="hardening=+all,-pie" DEB_LDFLAGS_MAINT_APPEND="-Wl,--as-needed -pie" dpkg-buildflags --get LDFLAGS)" \ && CONFIGURE_ARGS_MODULES="--prefix=/usr \ --statedir=/unit/var/lib/unit \ --debug \ --control=unix:/unit/var/run/control.unit.sock \ --runstatedir=/unit/var/run \ --pid=/unit/var/run/unit.pid \ --logdir=/unit/var/log \ --log=/unit/var/log/unit.log \ --tmpdir=/unit/var/tmp \ --openssl \ --libdir=/usr/lib/$DEB_HOST_MULTIARCH" \ && CONFIGURE_ARGS="$CONFIGURE_ARGS_MODULES \ --njs" \ && make -j $NCPU -C pkg/contrib .njs \ && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \ && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && make -j $NCPU unitd \ && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \ && make clean \ && ./configure $CONFIGURE_ARGS --cc-opt="$CC_OPT" --ld-opt="$LD_OPT" --modulesdir=/usr/lib/unit/modules \ && make -j $NCPU unitd \ && install -pm755 build/sbin/unitd /usr/sbin/unitd \ && make clean \ && apt-get install --no-install-recommends --no-install-suggests -y libclang-dev \ && export RUST_VERSION=1.76.0 \ && export RUSTUP_HOME=/usr/src/unit/rustup \ && export CARGO_HOME=/usr/src/unit/cargo \ && export PATH=/usr/src/unit/cargo/bin:$PATH \ && dpkgArch="$(dpkg --print-architecture)" \ && case "${dpkgArch##*-}" in \ amd64) rustArch="x86_64-unknown-linux-gnu"; rustupSha256="0b2f6c8f85a3d02fde2efc0ced4657869d73fccfce59defb4e8d29233116e6db" ;; \ arm64) rustArch="aarch64-unknown-linux-gnu"; rustupSha256="673e336c81c65e6b16dcdede33f4cc9ed0f08bde1dbe7a935f113605292dc800" ;; \ *) echo >&2 "unsupported architecture: ${dpkgArch}"; exit 1 ;; \ esac \ && url="https://static.rust-lang.org/rustup/archive/1.26.0/${rustArch}/rustup-init" \ && curl -L -O "$url" \ && echo "${rustupSha256} *rustup-init" | sha256sum -c - \ && chmod +x rustup-init \ && ./rustup-init -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION --default-host ${rustArch} \ && rm rustup-init \ && rustup --version \ && cargo --version \ && rustc --version \ && make -C pkg/contrib .wasmtime \ && install -pm 755 pkg/contrib/wasmtime/target/release/libwasmtime.so /usr/lib/$(dpkg-architecture -q DEB_HOST_MULTIARCH)/ \ && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/debug-modules --debug \ && ./configure wasm --include-path=`pwd`/pkg/contrib/wasmtime/crates/c-api/include --lib-path=/usr/lib/$(dpkg-architecture -q DEB_HOST_MULTIARCH)/ && ./configure wasm-wasi-component \ && make -j $NCPU wasm-install wasm-wasi-component-install \ && make clean \ && ./configure $CONFIGURE_ARGS_MODULES --cc-opt="$CC_OPT" --modulesdir=/usr/lib/unit/modules \ && ./configure wasm --include-path=`pwd`/pkg/contrib/wasmtime/crates/c-api/include --lib-path=/usr/lib/$(dpkg-architecture -q DEB_HOST_MULTIARCH)/ && ./configure wasm-wasi-component \ && make -j $NCPU wasm-install wasm-wasi-component-install \ && cd \ && rm -rf /usr/src/unit \ && for f in /usr/sbin/unitd /usr/lib/unit/modules/*.unit.so; do \ ldd $f | awk '/=>/{print $(NF-1)}' | while read n; do dpkg-query -S $n; done | sed 's/^\([^:]\+\):.*$/\1/' | sort | uniq >> /requirements.apt; \ done \ && apt-mark showmanual | xargs apt-mark auto > /dev/null \ && { [ -z "$savedAptMark" ] || apt-mark manual $savedAptMark; } \ && /bin/true \ && mkdir -p /var/lib/unit/ \ && mkdir -p /docker-entrypoint.d/ \ && apt-get update \ && apt-get --no-install-recommends --no-install-suggests -y install curl $(cat /requirements.apt) \ && apt-get purge -y --auto-remove build-essential \ && rm -rf /var/lib/apt/lists/* \ && rm -f /requirements.apt \ && ln -sf /dev/stderr /var/log/unit.log WORKDIR /app COPY --chmod=755 hello_wasi_http.wasm /app COPY ./*.json /docker-entrypoint.d/ COPY --chmod=755 docker-entrypoint.sh /usr/local/bin/ COPY welcome.* /usr/share/unit/welcome/ STOPSIGNAL SIGTERM ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] EXPOSE 8000 CMD ["unitd", "--no-daemon", "--control", "unix:/unit/var/run/control.unit.sock"] The changes to the Dockerfile are due to the non-root user not being able to write to the /var directory. Rather than modify the permissions on /var and its subdirectories, this docker file creates a /unit directory that is used for storing process information and logs. The docker-entrypoint.sh script also needed to be modified to use the /unit directory structure. The updated docker-entrypoint.sh script is shown below. #!/bin/sh set -e WAITLOOPS=5 SLEEPSEC=1 curl_put() { RET=$(/usr/bin/curl -s -w '%{http_code}' -X PUT --data-binary @$1 --unix-socket /unit/var/run/control.unit.sock http://localhost/$2) RET_BODY=$(echo $RET | /bin/sed '$ s/...$//') RET_STATUS=$(echo $RET | /usr/bin/tail -c 4) if [ "$RET_STATUS" -ne "200" ]; then echo "$0: Error: HTTP response status code is '$RET_STATUS'" echo "$RET_BODY" return 1 else echo "$0: OK: HTTP response status code is '$RET_STATUS'" echo "$RET_BODY" fi return 0 } if [ "$1" = "unitd" ] || [ "$1" = "unitd-debug" ]; then if /usr/bin/find "/unit/var/lib/unit/" -mindepth 1 -print -quit 2>/dev/null | /bin/grep -q .; then echo "$0: /unit/var/lib/unit/ is not empty, skipping initial configuration..." else echo "$0: Launching Unit daemon to perform initial configuration..." /usr/sbin/$1 --control unix:/unit/var/run/control.unit.sock for i in $(/usr/bin/seq $WAITLOOPS); do if [ ! -S /unit/var/run/control.unit.sock ]; then echo "$0: Waiting for control socket to be created..." /bin/sleep $SLEEPSEC else break fi done # even when the control socket exists, it does not mean unit has finished initialisation # this curl call will get a reply once unit is fully launched /usr/bin/curl -s -X GET --unix-socket /unit/var/run/control.unit.sock http://localhost/ if /usr/bin/find "/docker-entrypoint.d/" -mindepth 1 -print -quit 2>/dev/null | /bin/grep -q .; then echo "$0: /docker-entrypoint.d/ is not empty, applying initial configuration..." echo "$0: Looking for certificate bundles in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.pem"); do echo "$0: Uploading certificates bundle: $f" curl_put $f "certificates/$(basename $f .pem)" done echo "$0: Looking for JavaScript modules in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.js"); do echo "$0: Uploading JavaScript module: $f" curl_put $f "js_modules/$(basename $f .js)" done echo "$0: Looking for configuration snippets in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.json"); do echo "$0: Applying configuration $f"; curl_put $f "config" done echo "$0: Looking for shell scripts in /docker-entrypoint.d/..." for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name "*.sh"); do echo "$0: Launching $f"; "$f" done # warn on filetypes we don't know what to do with for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -not -name "*.sh" -not -name "*.json" -not -name "*.pem" -not -name "*.js"); do echo "$0: Ignoring $f"; done else echo "$0: /docker-entrypoint.d/ is empty, creating 'welcome' configuration..." curl_put /usr/share/unit/welcome/welcome.json "config" fi echo "$0: Stopping Unit daemon after initial configuration..." kill -TERM $(/bin/cat /unit/var/run/unit.pid) for i in $(/usr/bin/seq $WAITLOOPS); do if [ -S /unit/var/run/control.unit.sock ]; then echo "$0: Waiting for control socket to be removed..." /bin/sleep $SLEEPSEC else break fi done if [ -S /unit/var/run/control.unit.sock ]; then kill -KILL $(/bin/cat /unit/var/run/unit.pid) rm -f /unit/var/run/control.unit.sock fi echo echo "$0: Unit initial configuration complete; ready for start up..." echo fi fi exec "$@" The Dockerfile copies all files ending in .json to a directory named /docker-entrypoint.d. The docker-entrypoint.sh script checks the /docker-entrypoint.d directory for configuration files that are used to configure NGINX Unit. The following config.json file was created in the same directory as the Dockerfile and docker-entrypoint.sh script. { "listeners": { "*:8000": { "pass": "applications/wasm" } }, "applications": { "wasm": { "type": "wasm-wasi-component", "component": "/app/hello_wasi_http.wasm" } } } This config file instructs NGINX unit to listen on port 8000 and pass any client request to the Wasm application component located at /app/hello_wasi_http.wasm. The last step before building the container is to copy the Wasm application component into the same directory as the Dockerfile, docker-entrypoint.sh, and config.json files. With these four files in the same directory, the docker build command is used to create a container: docker build --no-cache --platform linux/amd64 -t hello-wasi-xc:1.0 . Here I use the --platform option to specify that the container will run on a linux/amd64 platform. I also use -t to give the image a name. The portion of the name after the ":" can be used for versioning. For example, if I added a new feature to my application, I could create a new image with -t hello-wasi-xc:1.1 to represent version 1.1 of the hello-wasi-xc application. Once the image is built, it can be uploaded to the container registry of your choice. I have access to an Azure Container Registry, so that is what I chose to use. Distributed Cloud supports pulling images from both public and private container registries. To push the image to my registry, I had to first tag the image with the registry location ({{registry_name}} should be replaced with the name of your registry): docker tag hello-wasi-xc:1.0 {{registry_name}}/examples/hello-wasi-xc:1.0 Next, I logged into my Azure registry: az login az acr login --name {{registry_name}} I then pushed the image to the registry: docker push {{registry_name}}/examples/hello-wasi-xc:1.0 F5 Distributed Cloud Virtual Kubernetes F5 Distributed Cloud Services support a Kubernetes compatible API for centralized orchestration of applications across a fleet of sites (customer sites or F5 Distributed Cloud Regional Edges). Distributed Cloud utilizes a distributed control plane to manage scheduling and scaling of applications across multiple sites. Kubernetes Objects Virtual Kubernetes supports the following Kubernetes objects: Deployments, StatefulSets, Jobs, CronJob, DaemonSet, Service, ConfigMap, Secrets, PersistentVolumeClaim, ServiceAccount, Role, and Role Binding. For this article I will focus on the Deployments object. A deployment is commonly used for stateless applications like web-servers, front-end web-ui, etc. A deployment works well for this use case because the NGINX Unit container is serving up a stateless web application. Workload Object A workload within Distributed Cloud is used to configure and deploy components of an application in Virtual Kubernetes. Workload encapsulates all the operational characteristics of Kubernetes workload, storage, and network objects (deployments, statefulsets, jobs, persistent volume claims, configmaps, secrets, and services) configuration, as well as configuration related to where the workload is deployed and how it is advertised using L7 or L4 load balancers. Within Distributed Cloud there are four types of workloads: Simple Service, Service, Stateful Service, and Job. Namespaces In Distributed Cloud, tenant configuration objects are grouped under namespaces. Namespaces can be thought of as administrative domains. Within each Namespace, a user can create an object called vK8s and use that for application management. Each Namespace can have a maximum of one vK8s object. Distributed Cloud vK8s Configuration vK8s is configured under the Distributed Apps tile within the Distributed Cloud console. After clicking on that tile, I next created a new Virtual K8s. By clicking Add Virtual K8s from the Virtual K8s menu under Applications. This brings up the Virtual K8s configuration form. I provided a name for my vK8s site and click Save and Exit. This initiates the vK8s build. Deploy the Application Once the vK8s cluster is ready, I click on the name of my vk8s to configure the deployment of my application. For this demo, I am going to deploy my application via Workloads. I click on Workloads and then Add VK8s Workload. This brings up the Workload configuration form. I provide my Workload a name, select Service for Workload type and click Configure. Using Service allows me to have more granular controls on how my application will be advertised. Next, I click Add Item under containers to specify to Distribtued Cloud where it can pull my container image from. In the resulting form, I supply a name for my container, supply the public registry FQDN along with the container name and version and click Apply. Next I configure where to advertise the application. I want to manually configure my HTTP LB, so I chose to advertise in Cluster. Manually configuring the LB allows me to configure additional options such as Web Application Firewall (WAF). Within the Advertise in Cluster form, I specify port 8000, because that is the port my container is listening on. Click Apply and then click Save and Exit. Create an HTTP Load Balancer to Advertise the Application The next step is to create an HTTP Load Balancer to expose the application to the Internet. This can be done by clicking Manage, selecting Load Balancers, and clicking on HTTP Load Balancers. Next, click on Add HTTP Load Balancer to create a new load balancer. I gave the LB a Name, Domain, Load Balancer Type, and Checked the box to Automatically Manage DNS Records. Then I configured the Origin Pool by clicking Add Item in the Origin Pool section and then clicking Add Item under the Origin Pool. On the resulting form, I created a name for my Origin Pool and then clicked Add Item under Origin Servers to add an origin Server. In the Origin Server form, I selected K8s Service Name of Origin Server on given Sites, Service Name, and provided my vK8s workload name along with my namespace. I also selected Virtual Site and created a Virtual Site for the REs I wanted to use to access my application and clicked Continue. Back on the Origin Server form, I selected vK8s Networks on Site from the Select Network on the site drop down and then clicked Apply. On the Origin Pool form, I specified port 8000 for my origin server Port and then clicked Continue. I then clicked Apply to return to the LB configuration form. I added a WAF policy to protect my application and then clicked Save and Exit. At this point my application is now accessible on the domain name I specified in the HTTP Load Balancer config (http://aconley-wasm.amer-ent.f5demos.com/). Conclusion This is a basic demo of using NGINX Unit and Distributed Cloud Virtual Kubernetes to deploy a server-side WASM application. The principals in this demo could be expanded to build more complex applications.
aconley
Nov 07, 2024 Place Technical Articles
161Views
0likes
0Comments
BIG-IP Next SPK: a Kubernetes native ingress and egress gateway for Telco workloads
BIG-IP Next SPK: a Kubernetes native ingress and egress gateway for Telco workloads that is CNF agnostic.
Ulises_Alonso
Oct 27, 2024 Place Technical Articles
2.8KViews
5likes
1Comment
How to Prepare Your Network Infrastructure to Add HPC Clusters for AI to Your Data Center
HPC AI clusters are getting deployed as highly-engineered 'lego blocks' which are opaque to established data center operations and standards. By taking advantage of established Kubernetes based networking solutions that provide high-speed intelligent networking, you can save yourself from expensive cost overruns, data center re-auditing, and delays. By using Kubernetes based solutions which take advantage of the high-speed networking solutions already required by HP AI deployments, you further optimize your investment in AI.
John_Gruber
Sep 25, 2024 Place Technical Articles
249Views
4likes
0Comments
Announcing F5 NGINX Gateway Fabric 1.4.0 with IPv6 and TLS Passthrough
We announced the next release of F5 NGINX Gateway Fabric version 1.4.0 which includes a lot of smaller but very necessary features. This allows us to dedicate more time to advancing our non-functional testing framework and ensuring we maintain top performance across releases. Nevertheless, we have some great highlights of this release: IPv6 support TLS passthrough (via TLSRoute) Server zone metrics Ability to add custom pod annotations Plenty of bug fixes! During this release cycle, we discovered a bug around our custom policies that occurred when you had the same path for more than one Route: The policy would not be applied to either Route. For this release, we’ve decided to enforce a restriction so that policies cannot be applied when two or more routes share the same path. However, we are pursuing a long-term solution to lift this restriction on this edge case, as we understand that use cases that route based on header, query parameter, or other request attributes on the same path do exist. IPv6 Support While most Kubernetes clusters are still utilizing IPv4, we recognized that anyone employing a IPv6 cluster would have no ability to deploy NGINX Gateway Fabric. Thus, we implemented a simple feature to dual IPv4/IPv6 networking for NGINX Gateway Fabric. This option is enabled by default, so you can simply install as normal on an IPv6 cluster. TLS Passthrough New with 1.4 is TLSRoute support. This Route type enables the TLS Passthrough use case and is similar to setting up an HTTPRoute. This allows you to pass encrypted traffic through NGINX Gateway Fabric where it is terminated by your backend application, ensuring end-to-end encryption. As most information passes through NGINX Gateway Fabric with this route, setup is easy. You can enable TLS passthrough for any application using our guide available here. Non-Functional Testing This release marks the completion of automating our non-functional testing that we execute before each release. If you are unfamiliar with these tests, our team runs NGINX Gateway Fabric through a series of scenarios, non-functional tests, to test if our performance is regressing or improving from previous releases. As an infrastructure product that you rely on, it is our top priority to ensure that stability and performance are not compromised as new features are released. The results of all non-functional testing are available in the GitHub repository for anyone to see and should give you an idea of how well NGINX Gateway Fabric performs in general and across releases. What’s Next NGINX Gateway Fabric 1.5.0 will bring NGINX code snippets to the Gateway API with a first-class Upstream Settings policy to configure keepalive connections and NGINX zone size. If you are familiar with NGINX or find that you need to use a feature that NGINX provides that is not yet available via a Gateway API extension, you can put a NGINX code snippet within a SnippetFilter to apply NGINX configuration to a Route rule. You will even be able to use the feature to load other modules NGINX provides and leverage the vast wealth of NGINX functionality. We will still be providing many NGINX features via first-class policies and filters, such as the Upstream Settings policy, as they allow us to handle much of the complexity of translating to Gateway API for you. These custom policies and filters allow us to handle a lot of the complexity of applying NGINX config across the Gateway API framework for you. The Upstream Settings policy can set upstream management directives that are unable to be applied via snippets effectively. We will continue to deliver these custom policies and filters across all of our releases, in addition to new Gateway API resources and NGINX Gateway Fabric specific features. You can see a preview of the full snippet design here, though not all features may be implemented in one release cycle. For more information on our strategy towards first-class NGINX customization via Gateway API extensions, see our full enhancement proposal here. Resources For the complete changelog for NGINX Gateway Fabric 1.4.0, see the Release Notes. To try NGINX Gateway Fabric for Kubernetes with NGINX Plus, start your free 30-day trial today or contact us to discuss your use cases. If you would like to get involved, see what is coming next, or see the source code for NGINX Gateway Fabric, check out our repository on GitHub! We have weekly community meetings on Tuesdays at 9:30AM Pacific/12:30PM Eastern/5:30PM GMT. The meeting link, updates, agenda, and notes are on the NGINX Gateway Fabric Meeting Calendar. Links are also always available from our GitHub readme.
mpstefan
Sep 20, 2024 Place Technical Articles
131Views
1like
0Comments
Announcing F5 NGINX Gateway Fabric 1.3.0 with Tracing, GRPCRoute, and Client Settings
The release of NGINX Gateway Fabric version 1.3.0, introduces plenty of highly requested features and improvements. GRPCRoutes are now supported to manage gRPC traffic, similar to the handling of HTTPRoute. The update includes new custom policies like ClientSettingsPolicy for client request configurations and ObservabilityPolicy for enabling application tracing with OpenTelemetry support. The GRPCRoute allows for efficient routing, header modifications, traffic weighting, and error conversion from HTTP to gRPC. We will explain how to set up NGINX Gateway Fabric to manage gRPC traffic using a Gateway and a GRPCRoute, providing a detailed example of the setup. It also outlines how to enable tracing through the NginxProxy resource and ObservabilityPolicy, emphasizing a selective approach to tracing to avoid data overload. Additionally, the ClientSettingsPolicy allows for the customization of NGINX directives at the Gateway or Route level, giving users control over certain NGINX behaviors with the possibility of overriding Gateway defaults at the Route level. Looking ahead, the NGINX Gateway Fabric team plans to work on TLS Passthrough, IPv6, and improvements to the testing suite, while preparing for larger updates like NGINX directive customization and separation of data and control planes. Check the end of the article to see how to get involved in the development process through GitHub and participate in bi-weekly community meetings. Further resources and links are also provided within.
mpstefan
Aug 08, 2024 Place Technical Articles
255Views
0likes
0Comments
Securing and Scaling Hybrid Apps with F5/NGINX (Part 3)
In part 2 of our series, I demonstrated how to configure ZT (Zero Trust) use cases centering around authentication with NGINX Plus in hybrid environments. We deployed NGINX Plus as the external LB to route and authenticate users connecting to my Kubernetes applications. In this article, we explore other areas of the ZT spectrum configurable on the External LB Service, including: Authorization and Access Encryption mTLS Monitoring/Auditing ZT Use case #1: Authorization Many people think that authentication and authorization can be used interchangeably. However, they both mean different things. Authentication involves the process of verifying user identities based on the credentials presented. Even though authenticated users are verified by the system, they do not necessarily have the authority to access protected applications. That is where authorization comes into play. Authorization involves the process of verifying the authority of an identity before granting access to application. Authorization in the context of OIDC authentication involves retrieving claims from user ID tokens and setting conditions to validate whether the user is authorized to enter the system. An authenticated user is granted an ID token from the IdP with specific user information through JWT claims. The configuration of these claims is typically set from the IdP. Revisiting the OIDC auth use case configured in the previous section, we can retrieve the ID tokens of authenticated users from the NGINX key-value store. $ curl -i http://localhost:8010/api/9/http/keyvals/oidc_acess_tokens Then we can view the decoded value of the ID token using jwt.io. Below is an example of decoded payload data from the ID token. { "exp": 1716219261, "iat": 1716219201, "admin": true, "name": "Micash", "zone_info": "America/Los_Angeles" "jti": "9f8ff4bd-4857-4e12-9634-e5876f786f98", "iss": "http://idp.f5lab.com:8080/auth/realms/master", "aud": "account", "typ": "Bearer", "azp": "appworld2024", "nonce": "gMNK3tu06j6tp5-jGa3aRhkj4F0P-Z3e04UfcFeqbes" } NGINX Plus has access to these claims as embedded variables. They are accessed by prefixing $jwt_claim_ to the desired field (for example, $jwt_claim_admin for the admin claim). We can easily set conditions on these claims and block unauthorized users before they even reach the back-end applications. Going back to our frontend.conf file in the previous part of our series. We can set $jwt_flag variable to 0 or 1 based on the value of the admin JWT claim. We then use the jwt_claim_require directive to validate the ID token. ID tokens with admin claims set to false will be rejected. map $jwt_claim_admin $jwt_status { "true" 1; default 0; } server { include conf.d/openid_connect.server_conf; # Authorization code flow and Relying Party processing error_log /var/log/nginx/error.log debug; # Reduce severity level as required listen [::]:443 ssl ipv6only=on; listen 443 ssl; server_name example.work.gd; ssl_certificate /etc/ssl/nginx/default.crt; # self-signed for example only ssl_certificate_key /etc/ssl/nginx/default.key; location / { # This site is protected with OpenID Connect auth_jwt "" token=$session_jwt; error_page 401 = @do_oidc_flow; auth_jwt_key_request /_jwks_uri; # Enable when using URL auth_jwt_require $jwt_status; proxy_pass https://cluster1-https; # The backend site/app } } Note: Authorization with NGINX Plus is not restricted to only JWT tokens. You can technically set conditions on a variety of attributes, such as: Session cookies HTTP headers Source/Destination IP addresses ZT use case #2: Mutual TLS Authentication (mTLS) When it comes to ZT, mTLS is one of the mainstream use cases falling under the Zero Trust umbrella. For example, enterprises are using Service Mesh technologies to stay compliant with ZT standards. This is because Service Mesh technologies aim to secure service to service communication using mTLS. In many ways, mTLS is similar to the OIDC use case we implemented in the previous section. Only here, we are leveraging digital certificates to encrypt and authenticate traffic. This underlying framework is defined by PKI (Public Key Infrastructure). To explain this framework in simple terms we can refer to a simple example; the driver's license you carry in your wallet. Your driver’s license can be used to validate your identity, the same way digital certificates can be used to validate the identity of applications. Similarly, only the state can issue valid driver's licenses, the same way only Certificate Authorities (CAs) can issue valid certificates to applications. It is also important that only the state can issue valid certificates. Therefore, every CA must have a private secure key to sign and issue valid certificates. Configuring mTLS with NGINX can be broken down in two parts: Ingress mTLS; Securing SSL client traffic and validating client certificates against a trusted CA. Egress mTLS; securing SSL upstream traffic and offloading authentication of TLS material to a trusted HTTPS back-end server. Ingress mTLS You can configure ingress mTLS on the NLK deployment by simply referencing the trusted certificate authority adding the ssl_client_certificate directive in the server context. This will configure NGINX to validate client certificates with the referenced CA. Note: If you do not have a CA, you can create one using OpenSSL or Cloudflare PKI and TLS toolkits server { listen 443 ssl; status_zone https://cafe.example.com; server_name cafe.example.com; ssl_certificate /etc/ssl/nginx/default.crt; ssl_certificate_key /etc/ssl/nginx/default.key; ssl_client_certificate /etc/ssl/ca.crt; } Egress mTLS Egress mTLS is a slight alternative to ingress mTLS where NGINX verifies certificates of upstream applications rather than certificates originating from clients. This feature can be enabled by adding the proxy_ssl_trusted_certificate directive to the server context. You can reference the same trusted CA we used for verification when configuring ingress mTLS or reference a different CA. In addition to verifying server certificates, NGINX as a reverse-proxy can pass over certs/keys and offload verification to HTTPS upstream applications. This can be done by adding the proxy_ssl_certificate and proxy_ssl_certificate_key directives in the server context. server { listen 443 ssl; status_zone https://cafe.example.com; server_name cafe.example.com; ssl_certificate /etc/ssl/nginx/default.crt; ssl_certificate_key /etc/ssl/nginx/default.key; #Ingress mTLS ssl_client_certificate /etc/ssl/ca.crt; #Egress mTLS proxy_ssl_certificate /etc/nginx/secrets/default-egress.crt; proxy_ssl_certificate_key /etc/nginx/secrets/default-egress.key; proxy_ssl_trusted_certificate /etc/nginx/secrets/default-egress-ca.crt; } ZT use case #3: Secure Assertion Markup Language (SAML) SAML (Security Assertion Markup Language) is an alternative SSO solution to OIDC. Many organizations may choose between SAML and OIDC depending on requirements and IdPs they currently run in production. SAML requires a SP (Service Provider) to exchange XML messages via HTTP POST binding to a SAML IdP. Once exchanges between the SP and IdP are successful, the user will have session access to the protected backed applications with one set of user credentials. In this section, we will configure NGINX Plus as the SP and enable SAML with the IdP. This will be like how we configured NGINX Plus as the relying party in an OIDC authorization code flow (See ZT Use case #1). Setting up the IdP The one prerequisite is setting up your IdP. In our example, we will set up the Microsoft Entra ID on Azure. You can use the SAML IdP of your choosing. Once the SAML application is created in your IdP, you can access the SSO fields necessary to link your SP (NGINX Plus) to your IdP (Microsoft Entra ID). You will need to edit the basic SAML configuration by clicking on the pencil icon next to Edit in Basic SAML Configuration, as seen in the figure above. Add the following values and click Save: Identifier (Entity ID) -- https://fourth.run.place Reply URL (Assertion Consumer Service URL) -- https://fourth.run.place/saml/acs Sign on URL: https://fourth.run.place Logout URL (Optional): https://fourth.run.place/saml/sls Finally download the Certificate (Raw) from Microsoft Entra ID and save it to your NGINX Plus instance. This certificate is used to verify signed SAML assertions received from the IdP. Once the certificate is saved on the NGINX Plus instance, extract the public key from the downloaded certificate and convert it to SPKI format. We will use this certificate later when we configure NGINX Plus in the next section. $ openssl x509 -in demo-nginx.der -outform DER -out demo-nginx.der $ openssl x509 -inform DER -in demo-nginx.der -pubkey -noout > demo-nginx.spki Configuring NGINX Plus as the SAML Service Provider After the IdP is setup, we can configure NGINX Plus as the SP to exchange and validate XML messages with the IdP. Once logged into the NGINX Plus instance, simply clone the nginx SAML GitHub repo. $ git clone https://github.com/nginxinc/nginx-saml.git && cd nginx-saml Copy the config files into the /etc/nginx/conf.d directory. $ cp frontend.conf saml_sp.js saml_sp.server_conf saml_sp_configuration.conf /etc/nginx/conf.d/ Notice that by default, frontend.conf listens on port 8010 with clear text http. You can merge kube_lb.conf into frontend.conf to enable TLS termination and update the upstream context with application endpoints you wish to protect with SAML. Finally we will need to edit the saml_sp_configuration.conf file and update variables in the map context based on the parameters of your SP and IdP: $saml_sp_entity_id; https://fourth.run.place $saml_sp_acs_url; https://fourth.run.place/saml/acs $saml_sp_sign_authn; false $saml_sp_want_signed_response; false $saml_sp_want_signed_assertion; true $saml_sp_want_encrypted_assertion; false $saml_idp_entity_id; Unique identifier that identifies the IdP to the SP. This field is retrieved from your IdP $saml_idp_sso_url; This is the login URL and is also retrieved from the IdP $saml_idp_verification_certificate; Variable referencing the certificate downloaded from the previous section when setting up the IdP. This certificate will verify signed assertions received from the IdP. Use the full directory (/etc/nginx/conf.d/demo-nginx.spki) $saml_sp_slo_url; https://fourth.run.place/saml/sls $saml_idp_slo_url; This is the logout URL retrieved from the IdP $saml_sp_want_signed_slo; true The remaining variables defined in saml_sp_configuration.conf can be left unchanged, unless there is a specific requirement for enabling them. Once the variables are set appropriately, we can reload NGINX Plus. $ nginx -s reload Testing Now we will verify the SAML flow. open your browser and enter https://fourth.run.place in the address bar. This should redirect me to the IDP login page. Once you login with your credentials, I should be granted access to my protected application ZT use case #4: Monitoring/Auditing NGINX logs/metrics can be exported to a variety of 3rd party providers including: Splunk, Prometheus/Grafana, cloud providers (AWS CloudWatch and Azure Monitor Logs), Datadog, ELK stack, and more. You can monitor NGINX metrics and logs natively with NGINX Instance Manager or NGINX SaaS. The NGINX Plus API provides me a lot of flexibility by exporting metrics to any third-party tool that accepts JSON. For example, you can export NGINX Plus API metrics to our native real-time dashboard from part 1. native real-time dashboard from part 1 Whichever tool I chose, monitoring/auditing my data generated from my IT systems is key to understanding and optimizing my applications. Conclusion Cloud providers offer a convenient way to expose Kubernetes Services to the internet. Simply create Kubernetes Service of type: LoadBalancer and external users connect to your services via public entry point. However, cloud load balancers do nothing more than basic TCP/HTTP load balancing. You can configure NGINX Plus with many Zero Trust capabilities as you scale out your environment to multiple clusters in different regions, which is what we will cover in the next part of our series.
Rawdata
Jun 18, 2024 Place Technical Articles
303Views
2likes
0Comments
Use topology labels to reduce cross-AZ ingress traffic with F5 CIS and EKS
This solution outlines how to use topology labels and multiple CIS instances to reduce cross-AZ traffic on AWS EKS cluster ingress.
MichaelOLeary
May 23, 2024 Place Technical Articles
524Views
0likes
0Comments
Securing and Scaling Hybrid Application with F5 NGINX (Part 1)
If you are using Kubernetes in production, then you are likely using an ingress controller. The ingress controller is the core engine managing traffic entering and exiting the Kubernetes cluster. Because the ingress controller is a deployment running inside the cluster, how do you route traffic to the ingress controller? How do you route external traffic to internal Kubernetes Services? Cloud providers offer a simple convenient way to expose Kubernetes Services using an external load balancer. Simply deploy a Managed Kubernetes Service (EKS, GKE, AKS) and create a Kubernetes Service of type LoadBalancer. The cloud providers will host and deploy a load balancer providing a public IP address. External users can connect to Kubernetes Services using this public entry point. However, this integration only applies to Managed Kubernetes Services hosted by cloud providers. If you are deploying Kubernetes in private cloud/on-prem environments, you will need to deploy your own load balancer and integrate it with the Kubernetes cluster. Furthermore, Kubernetes Load Balancing integrations in the cloud are limited to TCP Load Balancing and generally lack visibility into metrics, logs, and traces. We propose: A solution that applies regardless of the underlying infrastructure running your workloads Guidance around sizing to avoid bottlenecks from high traffic volumes Application delivery use cases that go beyond basic TCP/HTTP load balancing In the solution depicted below, I deploy NGINX Plus as the external LB service for Kubernetes and route traffic to the NGINX Ingress Controller. The NGINX Ingress Controller will then route the traffic to the application backends. The NLK (NGINX Load Balancer for Kubernetes) deployment is a new controller by NGINX that monitors specified Kubernetes Services and sends API calls to manage upstream endpoints of the NGINX External Load Balancer In this article, I will deploy the components both inside the Kubernetes cluster and NGINX Plus as the external load balancer. Note: I like to deploy both the NLK and Kubernetes cluster in the same subnet to avoid network issues. This is not a hard requirement. Prerequisites The blog assumes you have experience operating in Kubernetes environments. In addition, you have the following: Access to a Kubernetes environment; Bare Metal, Rancher Kubernetes Engine (RKE), VMWare Tanzu Kubernetes (VTK), Amazon Elastic Kubernetes (EKS), Google Kubernetes Engine (GKE), Microsoft Azure Kubernetes Service (AKS), and RedHat OpenShift NGINX Ingress Controller – Deploy NGINX Ingress Controller in the Kubernetes cluster. Installation instructions can be found in the documentation. NGINX Plus – Deploy NGINX Plus on VM or bare metal with SSH access. This will be the external LB service for the Kubernetes cluster. Installation instructions can be found in the documentation. You must have a valid license for NGINX Plus. You can get started today by requesting a 30-day free trial. Setting up the Kubernetes environment I start with deploying the back-end applications. You can deploy your own applications, or you can deploy our basic café application as an example. $ kubectl apply –f cafe.yaml Now I will configure routes and TLS settings for the ingress controller $ kubectl apply –f cafe-secret.yaml $ kubectl apply –f cafe-virtualserver.yaml To ensure the ingress rules are successfully applied, you can examine the output of kubectl get vs. The VirtualServer definition should be in the Valid state. NAMESPACE NAME STATE HOST IP PORTS default cafe-vs Valid cafe.example.com Setting up NGINX Plus as the external LB A fresh install of NGINX Plus will provide the default.conf file in the /etc/nginx/conf.d directory. We will add two additional files into this directory. Simply copy the bulleted files into your /etc/nginx/conf.d directory dashboard.conf; This will enable the real-time monitoring dashboard for NGINX Plus kube_lb.conf; The nginx configuration as the external load balancer for Kubernetes. You can change the configuration file to fit your requirements. In part 1 of this series, we enabled basic routing and TLS for one cluster. You will also need to generate TLS cert/keys and place them in the /etc/ssl/nginx folder of the NGINX Plus instance. For the sake of this example, we will generate a self-signed certificate with openssl. $ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout default.key -out default.crt -subj "/CN=NLK" Note: Using self-signed certificates is for testing purposes only. In a live production environment, we recommend using a secure vault that will hold your secrets and trusted CAs (Certificate Authorities). Now I can validate the configuration and reload nginx for the changes to take effect. $ nginx –t $ nginx –s reload I can now connect to the NGINX Plus dashboard by opening a browser and entering http://<external-ip-nginx>:9000/dashboard.html#upstreams The HTTP upstream table should be empty as we have not deployed the NLK Controller yet. We will do that in the next section. Installing the NLK Controller You can install the NLK Controller as a Kubernetes deployment that will configure upstream endpoints for the external load balancer using the NGINX Plus API. First, we will create the NLK namespace $ kubectl create ns nlk And apply the RBAC settings for the NKL deployment $ kubectl apply -f serviceaccount.yaml $ kubectl apply -f clusterrole.yaml $ kubectl apply -f clusterrolebinding.yaml $ kubectl apply -f secret.yaml The next step is to create a ConfigMap defining the API endpoint of the NGINX Plus external load balancer. The API endpoint is used by the NLK Controller to configure the NGINX Plus upstream endpoints. We simply modify the nginx-hosts field in the manifest from our GitHub repository to the IP address of the NGINX external load balancer. nginx-hosts: http://<nginx-plus-external-ip>:9000/api Apply the updated ConfigMap and deploy the NLK controller $ kubectl apply –f nkl-configmap.yaml $ kubectl apply –f nkl-deployment I can verify the NLK controller deployment is running and the ConfigMap data is applied. $ kubectl get pods –o wide –n nlk $ kubectl describe cm nginx-config –n nlk You should see the NLK deployment in status Running and the URL should be defined under nginx-hosts. The URL is the NGINX Plus API endpoint of the external load Balancer. Now that the NKL Controller is successfully deployed, the external load balancer is ready to route traffic to the cluster. The final step is deploying a Kubernetes Service type NodePort to expose the Kubernetes cluster to NGINX Plus. $ kubectl apply –f nodeport.yaml There are a couple things to note about the NodePort Service manifest. Fields on line 7 and 14 are required for the NLK deployment to configure the external load balancer appropriately: The nginxinc.io/nkl-cluster annotation The port name matching the upstream block definition in the NGINX Plus configuration (See line 42 in kube_lb.conf) and preceding nkl- apiVersion: v1 kind: Service metadata: name: nginx-ingress namespace: nginx-ingress annotations: nginxinc.io/nlk-cluster1-https: "http" # Must be added spec: type: NodePort ports: - port: 443 targetPort: 443 protocol: TCP name: nlk-cluster1-https selector: app: nginx-ingress Once the service is applied, you can note down the assigned nodeport selecting the NGINX Ingress Controller deployment. In this example, that node port is 32222. $ kubectl get svc –o wide –n nginx-ingress NAME TYPE CLUSTER-IP PORT(S) SELECTOR nginx-ingress NodePort x.x.x.x 443:32222/TCP app=nginx-ingress If I reconnect to my NGINX Pus dashboard, the upstream tab should be populated with the worker node IPs of the Kubernetes cluster and matching the node port of the nginx-ingress Service (32222). You can list the node IPs of your cluster to make sure they match the IPs in the dashboard upstream tab. $ kubectl get nodes -o wide | awk '{print $6}' INTERNAL-IP 10.224.0.6 10.224.0.5 10.224.0.4 Now I can connect to the Kubernetes application from our local machine. The hostname we used in our example (cafe.example.com) should resolve to the IP address of the NGINX Plus load balancer. Wrapping it up Most enterprises deploying Kubernetes in production will install an ingress controller. It is the DeFacto standard for application delivery in container orchestrators like Kubernetes. DevOps/NetOps engineers are now looking for guidance on how to scale out their Kubernetes footprint in the most efficient way possible. Because enterprises are embracing the hybrid approach, they will need to implement their own integrations outside of cloud providers. The solution we propose: is applicable to hybrid environments (particularly on-prem) Sizing information to avoid bottlenecks from large traffic volumes Enterprise Load Balancing capabilities that stretch beyond a TCP LoadBalancing Service In the next part of our series, I will dive into the third bullet point into much more detail and cover Zero Trust use cases with NGINX Plus, providing extra later of security in your hybrid model.
Rawdata
Apr 16, 2024 Place Technical Articles
723Views
0likes
0Comments

"}},"componentScriptGroups({\"componentId\":\"custom.widget.Beta_MetaNav\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"component({\"componentId\":\"custom.widget.Beta_Footer\"})":{"__typename":"Component","render({\"context\":{\"component\":{\"entities\":[],\"props\":{}},\"page\":{\"entities\":[],\"name\":\"TagPage\",\"props\":{},\"url\":\"https://community.f5.com/tag/kubernetes\"}}})":{"__typename":"ComponentRenderResult","html":"

Trademarks Policies Privacy California Privacy Do Not Sell My Personal Information

"}},"componentScriptGroups({\"componentId\":\"custom.widget.Beta_Footer\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"component({\"componentId\":\"custom.widget.Tag_Manager_Helper\"})":{"__typename":"Component","render({\"context\":{\"component\":{\"entities\":[],\"props\":{}},\"page\":{\"entities\":[],\"name\":\"TagPage\",\"props\":{},\"url\":\"https://community.f5.com/tag/kubernetes\"}}})":{"__typename":"ComponentRenderResult","html":" "}},"componentScriptGroups({\"componentId\":\"custom.widget.Tag_Manager_Helper\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"component({\"componentId\":\"custom.widget.Consent_Blackbar\"})":{"__typename":"Component","render({\"context\":{\"component\":{\"entities\":[],\"props\":{}},\"page\":{\"entities\":[],\"name\":\"TagPage\",\"props\":{},\"url\":\"https://community.f5.com/tag/kubernetes\"}}})":{"__typename":"ComponentRenderResult","html":"

"}},"componentScriptGroups({\"componentId\":\"custom.widget.Consent_Blackbar\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/community/NavbarDropdownToggle\"]})":[{"__ref":"CachedAsset:text:en_US-components/community/NavbarDropdownToggle-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/common/OverflowNav\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/common/OverflowNav-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageView/MessageViewInline\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageView/MessageViewInline-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/common/Pager/PagerLoadMore\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/common/Pager/PagerLoadMore-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/customComponent/CustomComponent\"]})":[{"__ref":"CachedAsset:text:en_US-components/customComponent/CustomComponent-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/users/UserLink\"]})":[{"__ref":"CachedAsset:text:en_US-components/users/UserLink-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageSubject\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageSubject-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageBody\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageBody-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageTime\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageTime-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/nodes/NodeIcon\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/nodes/NodeIcon-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageUnreadCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageUnreadCount-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageViewCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageViewCount-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/kudos/KudosCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/kudos/KudosCount-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageRepliesCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageRepliesCount-1743097587452"}],"cachedText({\"lastModified\":\"1743097587452\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/users/UserAvatar\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/users/UserAvatar-1743097587452"}]},"CachedAsset:pages-1742463026728":{"__typename":"CachedAsset","id":"pages-1742463026728","value":[{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetInvolved.MvpProgram","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved/mvp-program","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"BlogViewAllPostsPage","type":"BLOG","urlPath":"/category/:categoryId/blog/:boardId/all-posts/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"CasePortalPage","type":"CASE_PORTAL","urlPath":"/caseportal","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"CreateGroupHubPage","type":"GROUP_HUB","urlPath":"/groups/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"CaseViewPage","type":"CASE_DETAILS","urlPath":"/case/:caseId/:caseNumber","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"InboxPage","type":"COMMUNITY","urlPath":"/inbox","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetInvolved.AdvocacyProgram","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved/advocacy-program","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetHelp.NonCustomer","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/non-customer","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HelpFAQPage","type":"COMMUNITY","urlPath":"/help","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetHelp.F5Customer","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/f5-customer","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"IdeaMessagePage","type":"IDEA_POST","urlPath":"/idea/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"IdeaViewAllIdeasPage","type":"IDEA","urlPath":"/category/:categoryId/ideas/:boardId/all-ideas/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"LoginPage","type":"USER","urlPath":"/signin","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"BlogPostPage","type":"BLOG","urlPath":"/category/:categoryId/blogs/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetInvolved","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.Learn","type":"COMMUNITY","urlPath":"/c/how-do-i/learn","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1739501996000,"localOverride":null,"page":{"id":"Test","type":"CUSTOM","urlPath":"/custom-test-2","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ThemeEditorPage","type":"COMMUNITY","urlPath":"/designer/themes","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"TkbViewAllArticlesPage","type":"TKB","urlPath":"/category/:categoryId/kb/:boardId/all-articles/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"OccasionEditPage","type":"EVENT","urlPath":"/event/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"OAuthAuthorizationAllowPage","type":"USER","urlPath":"/auth/authorize/allow","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"PageEditorPage","type":"COMMUNITY","urlPath":"/designer/pages","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"PostPage","type":"COMMUNITY","urlPath":"/category/:categoryId/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ForumBoardPage","type":"FORUM","urlPath":"/category/:categoryId/discussions/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"TkbBoardPage","type":"TKB","urlPath":"/category/:categoryId/kb/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"EventPostPage","type":"EVENT","urlPath":"/category/:categoryId/events/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"UserBadgesPage","type":"COMMUNITY","urlPath":"/users/:login/:userId/badges","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"GroupHubMembershipAction","type":"GROUP_HUB","urlPath":"/membership/join/:nodeId/:membershipType","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"MaintenancePage","type":"COMMUNITY","urlPath":"/maintenance","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"IdeaReplyPage","type":"IDEA_REPLY","urlPath":"/idea/:boardId/:messageSubject/:messageId/comments/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"UserSettingsPage","type":"USER","urlPath":"/mysettings/:userSettingsTab","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"GroupHubsPage","type":"GROUP_HUB","urlPath":"/groups","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ForumPostPage","type":"FORUM","urlPath":"/category/:categoryId/discussions/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"OccasionRsvpActionPage","type":"OCCASION","urlPath":"/event/:boardId/:messageSubject/:messageId/rsvp/:responseType","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"VerifyUserEmailPage","type":"USER","urlPath":"/verifyemail/:userId/:verifyEmailToken","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"AllOccasionsPage","type":"OCCASION","urlPath":"/category/:categoryId/events/:boardId/all-events/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"EventBoardPage","type":"EVENT","urlPath":"/category/:categoryId/events/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"TkbReplyPage","type":"TKB_REPLY","urlPath":"/kb/:boardId/:messageSubject/:messageId/comments/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"IdeaBoardPage","type":"IDEA","urlPath":"/category/:categoryId/ideas/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"CommunityGuideLinesPage","type":"COMMUNITY","urlPath":"/communityguidelines","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"CaseCreatePage","type":"SALESFORCE_CASE_CREATION","urlPath":"/caseportal/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"TkbEditPage","type":"TKB","urlPath":"/kb/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ForgotPasswordPage","type":"USER","urlPath":"/forgotpassword","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"IdeaEditPage","type":"IDEA","urlPath":"/idea/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"TagPage","type":"COMMUNITY","urlPath":"/tag/:tagName","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"BlogBoardPage","type":"BLOG","urlPath":"/category/:categoryId/blog/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"OccasionMessagePage","type":"OCCASION_TOPIC","urlPath":"/event/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ManageContentPage","type":"COMMUNITY","urlPath":"/managecontent","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ClosedMembershipNodeNonMembersPage","type":"GROUP_HUB","urlPath":"/closedgroup/:groupHubId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetHelp.Community","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/community","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"CommunityPage","type":"COMMUNITY","urlPath":"/","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetInvolved.ContributeCode","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved/contribute-code","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ForumMessagePage","type":"FORUM_TOPIC","urlPath":"/discussions/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"IdeaPostPage","type":"IDEA","urlPath":"/category/:categoryId/ideas/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"BlogMessagePage","type":"BLOG_ARTICLE","urlPath":"/blog/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"RegistrationPage","type":"USER","urlPath":"/register","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"EditGroupHubPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ForumEditPage","type":"FORUM","urlPath":"/discussions/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ResetPasswordPage","type":"USER","urlPath":"/resetpassword/:userId/:resetPasswordToken","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"TkbMessagePage","type":"TKB_ARTICLE","urlPath":"/kb/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.Learn.AboutIrules","type":"COMMUNITY","urlPath":"/c/how-do-i/learn/about-irules","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"BlogEditPage","type":"BLOG","urlPath":"/blog/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetHelp.F5Support","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/f5-support","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ManageUsersPage","type":"USER","urlPath":"/users/manage/:tab?/:manageUsersTab?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ForumReplyPage","type":"FORUM_REPLY","urlPath":"/discussions/:boardId/:messageSubject/:messageId/replies/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"PrivacyPolicyPage","type":"COMMUNITY","urlPath":"/privacypolicy","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"NotificationPage","type":"COMMUNITY","urlPath":"/notifications","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"UserPage","type":"USER","urlPath":"/users/:login/:userId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HealthCheckPage","type":"COMMUNITY","urlPath":"/health","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"OccasionReplyPage","type":"OCCASION_REPLY","urlPath":"/event/:boardId/:messageSubject/:messageId/comments/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ManageMembersPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId/manage/:tab?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"SearchResultsPage","type":"COMMUNITY","urlPath":"/search","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"BlogReplyPage","type":"BLOG_REPLY","urlPath":"/blog/:boardId/:messageSubject/:messageId/replies/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"GroupHubPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"TermsOfServicePage","type":"COMMUNITY","urlPath":"/termsofservice","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetHelp","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI.GetHelp.SecurityIncident","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/security-incident","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"CategoryPage","type":"CATEGORY","urlPath":"/category/:categoryId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"ForumViewAllTopicsPage","type":"FORUM","urlPath":"/category/:categoryId/discussions/:boardId/all-topics/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"TkbPostPage","type":"TKB","urlPath":"/category/:categoryId/kbs/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"GroupHubPostPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742463026728,"localOverride":null,"page":{"id":"HowDoI","type":"COMMUNITY","urlPath":"/c/how-do-i","__typename":"PageDescriptor"},"__typename":"PageResource"}],"localOverride":false},"CachedAsset:text:en_US-components/context/AppContext/AppContextProvider-0":{"__typename":"CachedAsset","id":"text:en_US-components/context/AppContext/AppContextProvider-0","value":{"noCommunity":"Cannot find community","noUser":"Cannot find current user","noNode":"Cannot find node with id {nodeId}","noMessage":"Cannot find message with id {messageId}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/Loading/LoadingDot-0":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/Loading/LoadingDot-0","value":{"title":"Loading..."},"localOverride":false},"User:user:-1":{"__typename":"User","id":"user:-1","uid":-1,"login":"Former Member","email":"","avatar":null,"rank":null,"kudosWeight":1,"registrationData":{"__typename":"RegistrationData","status":"ANONYMOUS","registrationTime":null,"confirmEmailStatus":false,"registrationAccessLevel":"VIEW","ssoRegistrationFields":[]},"ssoId":null,"profileSettings":{"__typename":"ProfileSettings","dateDisplayStyle":{"__typename":"InheritableStringSettingWithPossibleValues","key":"layout.friendly_dates_enabled","value":"false","localValue":"true","possibleValues":["true","false"]},"dateDisplayFormat":{"__typename":"InheritableStringSetting","key":"layout.format_pattern_date","value":"dd-MMM-yyyy","localValue":"MM-dd-yyyy"},"language":{"__typename":"InheritableStringSettingWithPossibleValues","key":"profile.language","value":"en-US","localValue":null,"possibleValues":["en-US"]}},"deleted":false},"Theme:customTheme1":{"__typename":"Theme","id":"customTheme1"},"CachedAsset:theme:customTheme1-1742463026290":{"__typename":"CachedAsset","id":"theme:customTheme1-1742463026290","value":{"id":"customTheme1","animation":{"fast":"150ms","normal":"250ms","slow":"500ms","slowest":"750ms","function":"cubic-bezier(0.07, 0.91, 0.51, 1)","__typename":"AnimationThemeSettings"},"avatar":{"borderRadius":"50%","collections":["custom"],"__typename":"AvatarThemeSettings"},"basics":{"browserIcon":{"imageAssetName":"JimmyPackets-512-1702592938213.png","imageLastModified":"1702592945815","__typename":"ThemeAsset"},"customerLogo":{"imageAssetName":"f5_logo_fix-1704824537976.svg","imageLastModified":"1704824540697","__typename":"ThemeAsset"},"maximumWidthOfPageContent":"1600px","oneColumnNarrowWidth":"800px","gridGutterWidthMd":"30px","gridGutterWidthXs":"10px","pageWidthStyle":"WIDTH_OF_PAGE_CONTENT","__typename":"BasicsThemeSettings"},"buttons":{"borderRadiusSm":"5px","borderRadius":"5px","borderRadiusLg":"5px","paddingY":"5px","paddingYLg":"7px","paddingYHero":"var(--lia-bs-btn-padding-y-lg)","paddingX":"12px","paddingXLg":"14px","paddingXHero":"42px","fontStyle":"NORMAL","fontWeight":"400","textTransform":"NONE","disabledOpacity":0.5,"primaryTextColor":"var(--lia-bs-white)","primaryTextHoverColor":"var(--lia-bs-white)","primaryTextActiveColor":"var(--lia-bs-white)","primaryBgColor":"var(--lia-bs-primary)","primaryBgHoverColor":"hsl(var(--lia-bs-primary-h), var(--lia-bs-primary-s), calc(var(--lia-bs-primary-l) * 0.85))","primaryBgActiveColor":"hsl(var(--lia-bs-primary-h), var(--lia-bs-primary-s), calc(var(--lia-bs-primary-l) * 0.7))","primaryBorder":"1px solid transparent","primaryBorderHover":"1px solid transparent","primaryBorderActive":"1px solid transparent","primaryBorderFocus":"1px solid var(--lia-bs-white)","primaryBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","secondaryTextColor":"var(--lia-bs-gray-900)","secondaryTextHoverColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.95))","secondaryTextActiveColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.9))","secondaryBgColor":"var(--lia-bs-gray-400)","secondaryBgHoverColor":"hsl(var(--lia-bs-gray-400-h), var(--lia-bs-gray-400-s), calc(var(--lia-bs-gray-400-l) * 0.96))","secondaryBgActiveColor":"hsl(var(--lia-bs-gray-400-h), var(--lia-bs-gray-400-s), calc(var(--lia-bs-gray-400-l) * 0.92))","secondaryBorder":"1px solid transparent","secondaryBorderHover":"1px solid transparent","secondaryBorderActive":"1px solid transparent","secondaryBorderFocus":"1px solid transparent","secondaryBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","tertiaryTextColor":"var(--lia-bs-gray-900)","tertiaryTextHoverColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.95))","tertiaryTextActiveColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.9))","tertiaryBgColor":"transparent","tertiaryBgHoverColor":"transparent","tertiaryBgActiveColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.04)","tertiaryBorder":"1px solid transparent","tertiaryBorderHover":"1px solid hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","tertiaryBorderActive":"1px solid transparent","tertiaryBorderFocus":"1px solid transparent","tertiaryBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","destructiveTextColor":"var(--lia-bs-danger)","destructiveTextHoverColor":"hsl(var(--lia-bs-danger-h), var(--lia-bs-danger-s), calc(var(--lia-bs-danger-l) * 0.95))","destructiveTextActiveColor":"hsl(var(--lia-bs-danger-h), var(--lia-bs-danger-s), calc(var(--lia-bs-danger-l) * 0.9))","destructiveBgColor":"var(--lia-bs-gray-300)","destructiveBgHoverColor":"hsl(var(--lia-bs-gray-300-h), var(--lia-bs-gray-300-s), calc(var(--lia-bs-gray-300-l) * 0.96))","destructiveBgActiveColor":"hsl(var(--lia-bs-gray-300-h), var(--lia-bs-gray-300-s), calc(var(--lia-bs-gray-300-l) * 0.92))","destructiveBorder":"1px solid transparent","destructiveBorderHover":"1px solid transparent","destructiveBorderActive":"1px solid transparent","destructiveBorderFocus":"1px solid transparent","destructiveBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","__typename":"ButtonsThemeSettings"},"border":{"color":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","mainContent":"NONE","sideContent":"NONE","radiusSm":"3px","radius":"5px","radiusLg":"9px","radius50":"100vw","__typename":"BorderThemeSettings"},"boxShadow":{"xs":"0 0 0 1px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.08), 0 3px 0 -1px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.08)","sm":"0 2px 4px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.06)","md":"0 5px 15px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.15)","lg":"0 10px 30px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.15)","__typename":"BoxShadowThemeSettings"},"cards":{"bgColor":"var(--lia-panel-bg-color)","borderRadius":"var(--lia-panel-border-radius)","boxShadow":"var(--lia-box-shadow-xs)","__typename":"CardsThemeSettings"},"chip":{"maxWidth":"300px","height":"30px","__typename":"ChipThemeSettings"},"coreTypes":{"defaultMessageLinkColor":"var(--lia-bs-primary)","defaultMessageLinkDecoration":"none","defaultMessageLinkFontStyle":"NORMAL","defaultMessageLinkFontWeight":"400","defaultMessageFontStyle":"NORMAL","defaultMessageFontWeight":"400","forumColor":"#0C5C8D","forumFontFamily":"var(--lia-bs-font-family-base)","forumFontWeight":"var(--lia-default-message-font-weight)","forumLineHeight":"var(--lia-bs-line-height-base)","forumFontStyle":"var(--lia-default-message-font-style)","forumMessageLinkColor":"var(--lia-default-message-link-color)","forumMessageLinkDecoration":"var(--lia-default-message-link-decoration)","forumMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","forumMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","forumSolvedColor":"#62C026","blogColor":"#730015","blogFontFamily":"var(--lia-bs-font-family-base)","blogFontWeight":"var(--lia-default-message-font-weight)","blogLineHeight":"1.75","blogFontStyle":"var(--lia-default-message-font-style)","blogMessageLinkColor":"var(--lia-default-message-link-color)","blogMessageLinkDecoration":"var(--lia-default-message-link-decoration)","blogMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","blogMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","tkbColor":"#C20025","tkbFontFamily":"var(--lia-bs-font-family-base)","tkbFontWeight":"var(--lia-default-message-font-weight)","tkbLineHeight":"1.75","tkbFontStyle":"var(--lia-default-message-font-style)","tkbMessageLinkColor":"var(--lia-default-message-link-color)","tkbMessageLinkDecoration":"var(--lia-default-message-link-decoration)","tkbMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","tkbMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","qandaColor":"#4099E2","qandaFontFamily":"var(--lia-bs-font-family-base)","qandaFontWeight":"var(--lia-default-message-font-weight)","qandaLineHeight":"var(--lia-bs-line-height-base)","qandaFontStyle":"var(--lia-default-message-link-font-style)","qandaMessageLinkColor":"var(--lia-default-message-link-color)","qandaMessageLinkDecoration":"var(--lia-default-message-link-decoration)","qandaMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","qandaMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","qandaSolvedColor":"#3FA023","ideaColor":"#F3704B","ideaFontFamily":"var(--lia-bs-font-family-base)","ideaFontWeight":"var(--lia-default-message-font-weight)","ideaLineHeight":"var(--lia-bs-line-height-base)","ideaFontStyle":"var(--lia-default-message-font-style)","ideaMessageLinkColor":"var(--lia-default-message-link-color)","ideaMessageLinkDecoration":"var(--lia-default-message-link-decoration)","ideaMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","ideaMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","contestColor":"#FCC845","contestFontFamily":"var(--lia-bs-font-family-base)","contestFontWeight":"var(--lia-default-message-font-weight)","contestLineHeight":"var(--lia-bs-line-height-base)","contestFontStyle":"var(--lia-default-message-link-font-style)","contestMessageLinkColor":"var(--lia-default-message-link-color)","contestMessageLinkDecoration":"var(--lia-default-message-link-decoration)","contestMessageLinkFontStyle":"ITALIC","contestMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","occasionColor":"#EE4B5B","occasionFontFamily":"var(--lia-bs-font-family-base)","occasionFontWeight":"var(--lia-default-message-font-weight)","occasionLineHeight":"var(--lia-bs-line-height-base)","occasionFontStyle":"var(--lia-default-message-font-style)","occasionMessageLinkColor":"var(--lia-default-message-link-color)","occasionMessageLinkDecoration":"var(--lia-default-message-link-decoration)","occasionMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","occasionMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","grouphubColor":"#491B62","categoryColor":"#949494","communityColor":"#FFFFFF","productColor":"#949494","__typename":"CoreTypesThemeSettings"},"colors":{"black":"#000000","white":"#FFFFFF","gray100":"#F7F7F7","gray200":"#F7F7F7","gray300":"#E8E8E8","gray400":"#D9D9D9","gray500":"#CCCCCC","gray600":"#949494","gray700":"#707070","gray800":"#545454","gray900":"#333333","dark":"#545454","light":"#F7F7F7","primary":"#0C5C8D","secondary":"#333333","bodyText":"#222222","bodyBg":"#F5F5F5","info":"#1D9CD3","success":"#62C026","warning":"#FFD651","danger":"#C20025","alertSystem":"#FF6600","textMuted":"#707070","highlight":"#FFFCAD","outline":"var(--lia-bs-primary)","custom":["#C20025","#081B85","#009639","#B3C6D7","#7CC0EB","#F29A36"],"__typename":"ColorsThemeSettings"},"divider":{"size":"3px","marginLeft":"4px","marginRight":"4px","borderRadius":"50%","bgColor":"var(--lia-bs-gray-600)","bgColorActive":"var(--lia-bs-gray-600)","__typename":"DividerThemeSettings"},"dropdown":{"fontSize":"var(--lia-bs-font-size-sm)","borderColor":"var(--lia-bs-border-color)","borderRadius":"var(--lia-bs-border-radius-sm)","dividerBg":"var(--lia-bs-gray-300)","itemPaddingY":"5px","itemPaddingX":"20px","headerColor":"var(--lia-bs-gray-700)","__typename":"DropdownThemeSettings"},"email":{"link":{"color":"#0069D4","hoverColor":"#0061c2","decoration":"none","hoverDecoration":"underline","__typename":"EmailLinkSettings"},"border":{"color":"#e4e4e4","__typename":"EmailBorderSettings"},"buttons":{"borderRadiusLg":"5px","paddingXLg":"16px","paddingYLg":"7px","fontWeight":"700","primaryTextColor":"#ffffff","primaryTextHoverColor":"#ffffff","primaryBgColor":"#0069D4","primaryBgHoverColor":"#005cb8","primaryBorder":"1px solid transparent","primaryBorderHover":"1px solid transparent","__typename":"EmailButtonsSettings"},"panel":{"borderRadius":"5px","borderColor":"#e4e4e4","__typename":"EmailPanelSettings"},"__typename":"EmailThemeSettings"},"emoji":{"skinToneDefault":"#ffcd43","skinToneLight":"#fae3c5","skinToneMediumLight":"#e2cfa5","skinToneMedium":"#daa478","skinToneMediumDark":"#a78058","skinToneDark":"#5e4d43","__typename":"EmojiThemeSettings"},"heading":{"color":"var(--lia-bs-body-color)","fontFamily":"Inter","fontStyle":"NORMAL","fontWeight":"600","h1FontSize":"30px","h2FontSize":"25px","h3FontSize":"20px","h4FontSize":"18px","h5FontSize":"16px","h6FontSize":"16px","lineHeight":"1.2","subHeaderFontSize":"11px","subHeaderFontWeight":"500","h1LetterSpacing":"normal","h2LetterSpacing":"normal","h3LetterSpacing":"normal","h4LetterSpacing":"normal","h5LetterSpacing":"normal","h6LetterSpacing":"normal","subHeaderLetterSpacing":"2px","h1FontWeight":"var(--lia-bs-headings-font-weight)","h2FontWeight":"var(--lia-bs-headings-font-weight)","h3FontWeight":"var(--lia-bs-headings-font-weight)","h4FontWeight":"var(--lia-bs-headings-font-weight)","h5FontWeight":"var(--lia-bs-headings-font-weight)","h6FontWeight":"var(--lia-bs-headings-font-weight)","__typename":"HeadingThemeSettings"},"icons":{"size10":"10px","size12":"12px","size14":"14px","size16":"16px","size20":"20px","size24":"24px","size30":"30px","size40":"40px","size50":"50px","size60":"60px","size80":"80px","size120":"120px","size160":"160px","__typename":"IconsThemeSettings"},"imagePreview":{"bgColor":"var(--lia-bs-gray-900)","titleColor":"var(--lia-bs-white)","controlColor":"var(--lia-bs-white)","controlBgColor":"var(--lia-bs-gray-800)","__typename":"ImagePreviewThemeSettings"},"input":{"borderColor":"var(--lia-bs-gray-600)","disabledColor":"var(--lia-bs-gray-600)","focusBorderColor":"var(--lia-bs-primary)","labelMarginBottom":"10px","btnFontSize":"var(--lia-bs-font-size-sm)","focusBoxShadow":"0 0 0 3px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","checkLabelMarginBottom":"2px","checkboxBorderRadius":"3px","borderRadiusSm":"var(--lia-bs-border-radius-sm)","borderRadius":"var(--lia-bs-border-radius)","borderRadiusLg":"var(--lia-bs-border-radius-lg)","formTextMarginTop":"4px","textAreaBorderRadius":"var(--lia-bs-border-radius)","activeFillColor":"var(--lia-bs-primary)","__typename":"InputThemeSettings"},"loading":{"dotDarkColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.2)","dotLightColor":"hsla(var(--lia-bs-white-h), var(--lia-bs-white-s), var(--lia-bs-white-l), 0.5)","barDarkColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.06)","barLightColor":"hsla(var(--lia-bs-white-h), var(--lia-bs-white-s), var(--lia-bs-white-l), 0.4)","__typename":"LoadingThemeSettings"},"link":{"color":"var(--lia-bs-primary)","hoverColor":"hsl(var(--lia-bs-primary-h), var(--lia-bs-primary-s), calc(var(--lia-bs-primary-l) - 10%))","decoration":"none","hoverDecoration":"underline","__typename":"LinkThemeSettings"},"listGroup":{"itemPaddingY":"15px","itemPaddingX":"15px","borderColor":"var(--lia-bs-gray-300)","__typename":"ListGroupThemeSettings"},"modal":{"contentTextColor":"var(--lia-bs-body-color)","contentBg":"var(--lia-bs-white)","backgroundBg":"var(--lia-bs-black)","smSize":"440px","mdSize":"760px","lgSize":"1080px","backdropOpacity":0.3,"contentBoxShadowXs":"var(--lia-bs-box-shadow-sm)","contentBoxShadow":"var(--lia-bs-box-shadow)","headerFontWeight":"700","__typename":"ModalThemeSettings"},"navbar":{"position":"FIXED","background":{"attachment":null,"clip":null,"color":"var(--lia-bs-white)","imageAssetName":null,"imageLastModified":"0","origin":null,"position":"CENTER_CENTER","repeat":"NO_REPEAT","size":"COVER","__typename":"BackgroundProps"},"backgroundOpacity":0.8,"paddingTop":"15px","paddingBottom":"15px","borderBottom":"1px solid var(--lia-bs-border-color)","boxShadow":"var(--lia-bs-box-shadow-sm)","brandMarginRight":"30px","brandMarginRightSm":"10px","brandLogoHeight":"30px","linkGap":"10px","linkJustifyContent":"flex-start","linkPaddingY":"5px","linkPaddingX":"10px","linkDropdownPaddingY":"9px","linkDropdownPaddingX":"var(--lia-nav-link-px)","linkColor":"var(--lia-bs-body-color)","linkHoverColor":"var(--lia-bs-primary)","linkFontSize":"var(--lia-bs-font-size-sm)","linkFontStyle":"NORMAL","linkFontWeight":"400","linkTextTransform":"NONE","linkLetterSpacing":"normal","linkBorderRadius":"var(--lia-bs-border-radius-sm)","linkBgColor":"transparent","linkBgHoverColor":"transparent","linkBorder":"none","linkBorderHover":"none","linkBoxShadow":"none","linkBoxShadowHover":"none","linkTextBorderBottom":"none","linkTextBorderBottomHover":"none","dropdownPaddingTop":"10px","dropdownPaddingBottom":"15px","dropdownPaddingX":"10px","dropdownMenuOffset":"2px","dropdownDividerMarginTop":"10px","dropdownDividerMarginBottom":"10px","dropdownBorderColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","controllerBgHoverColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.1)","controllerIconColor":"var(--lia-bs-body-color)","controllerIconHoverColor":"var(--lia-bs-body-color)","controllerTextColor":"var(--lia-nav-controller-icon-color)","controllerTextHoverColor":"var(--lia-nav-controller-icon-hover-color)","controllerHighlightColor":"hsla(30, 100%, 50%)","controllerHighlightTextColor":"var(--lia-yiq-light)","controllerBorderRadius":"var(--lia-border-radius-50)","hamburgerColor":"var(--lia-nav-controller-icon-color)","hamburgerHoverColor":"var(--lia-nav-controller-icon-color)","hamburgerBgColor":"transparent","hamburgerBgHoverColor":"transparent","hamburgerBorder":"none","hamburgerBorderHover":"none","collapseMenuMarginLeft":"20px","collapseMenuDividerBg":"var(--lia-nav-link-color)","collapseMenuDividerOpacity":0.16,"__typename":"NavbarThemeSettings"},"pager":{"textColor":"var(--lia-bs-link-color)","textFontWeight":"var(--lia-font-weight-md)","textFontSize":"var(--lia-bs-font-size-sm)","__typename":"PagerThemeSettings"},"panel":{"bgColor":"var(--lia-bs-white)","borderRadius":"var(--lia-bs-border-radius)","borderColor":"var(--lia-bs-border-color)","boxShadow":"none","__typename":"PanelThemeSettings"},"popover":{"arrowHeight":"8px","arrowWidth":"16px","maxWidth":"300px","minWidth":"100px","headerBg":"var(--lia-bs-white)","borderColor":"var(--lia-bs-border-color)","borderRadius":"var(--lia-bs-border-radius)","boxShadow":"0 0.5rem 1rem hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.15)","__typename":"PopoverThemeSettings"},"prism":{"color":"#000000","bgColor":"#f5f2f0","fontFamily":"var(--font-family-monospace)","fontSize":"var(--lia-bs-font-size-base)","fontWeightBold":"var(--lia-bs-font-weight-bold)","fontStyleItalic":"italic","tabSize":2,"highlightColor":"#b3d4fc","commentColor":"#62707e","punctuationColor":"#6f6f6f","namespaceOpacity":"0.7","propColor":"#990055","selectorColor":"#517a00","operatorColor":"#906736","operatorBgColor":"hsla(0, 0%, 100%, 0.5)","keywordColor":"#0076a9","functionColor":"#d3284b","variableColor":"#c14700","__typename":"PrismThemeSettings"},"rte":{"bgColor":"var(--lia-bs-white)","borderRadius":"var(--lia-panel-border-radius)","boxShadow":" var(--lia-panel-box-shadow)","customColor1":"#bfedd2","customColor2":"#fbeeb8","customColor3":"#f8cac6","customColor4":"#eccafa","customColor5":"#c2e0f4","customColor6":"#2dc26b","customColor7":"#f1c40f","customColor8":"#e03e2d","customColor9":"#b96ad9","customColor10":"#3598db","customColor11":"#169179","customColor12":"#e67e23","customColor13":"#ba372a","customColor14":"#843fa1","customColor15":"#236fa1","customColor16":"#ecf0f1","customColor17":"#ced4d9","customColor18":"#95a5a6","customColor19":"#7e8c8d","customColor20":"#34495e","customColor21":"#000000","customColor22":"#ffffff","defaultMessageHeaderMarginTop":"14px","defaultMessageHeaderMarginBottom":"10px","defaultMessageItemMarginTop":"0","defaultMessageItemMarginBottom":"10px","diffAddedColor":"hsla(170, 53%, 51%, 0.4)","diffChangedColor":"hsla(43, 97%, 63%, 0.4)","diffNoneColor":"hsla(0, 0%, 80%, 0.4)","diffRemovedColor":"hsla(9, 74%, 47%, 0.4)","specialMessageHeaderMarginTop":"14px","specialMessageHeaderMarginBottom":"10px","specialMessageItemMarginTop":"0","specialMessageItemMarginBottom":"10px","__typename":"RteThemeSettings"},"tags":{"bgColor":"var(--lia-bs-gray-200)","bgHoverColor":"var(--lia-bs-gray-400)","borderRadius":"var(--lia-bs-border-radius-sm)","color":"var(--lia-bs-body-color)","hoverColor":"var(--lia-bs-body-color)","fontWeight":"var(--lia-font-weight-md)","fontSize":"var(--lia-font-size-xxs)","textTransform":"UPPERCASE","letterSpacing":"0.5px","__typename":"TagsThemeSettings"},"toasts":{"borderRadius":"var(--lia-bs-border-radius)","paddingX":"12px","__typename":"ToastsThemeSettings"},"typography":{"fontFamilyBase":"Atkinson Hyperlegible","fontStyleBase":"NORMAL","fontWeightBase":"400","fontWeightLight":"300","fontWeightNormal":"400","fontWeightMd":"500","fontWeightBold":"700","letterSpacingSm":"normal","letterSpacingXs":"normal","lineHeightBase":"1.3","fontSizeBase":"15px","fontSizeXxs":"11px","fontSizeXs":"12px","fontSizeSm":"13px","fontSizeLg":"20px","fontSizeXl":"24px","smallFontSize":"14px","customFonts":[],"__typename":"TypographyThemeSettings"},"unstyledListItem":{"marginBottomSm":"5px","marginBottomMd":"10px","marginBottomLg":"15px","marginBottomXl":"20px","marginBottomXxl":"25px","__typename":"UnstyledListItemThemeSettings"},"yiq":{"light":"#ffffff","dark":"#000000","__typename":"YiqThemeSettings"},"colorLightness":{"primaryDark":0.36,"primaryLight":0.74,"primaryLighter":0.89,"primaryLightest":0.95,"infoDark":0.39,"infoLight":0.72,"infoLighter":0.85,"infoLightest":0.93,"successDark":0.24,"successLight":0.62,"successLighter":0.8,"successLightest":0.91,"warningDark":0.39,"warningLight":0.68,"warningLighter":0.84,"warningLightest":0.93,"dangerDark":0.41,"dangerLight":0.72,"dangerLighter":0.89,"dangerLightest":0.95,"__typename":"ColorLightnessThemeSettings"},"localOverride":false,"__typename":"Theme"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/Loading/LoadingDot-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/Loading/LoadingDot-1743097587452","value":{"title":"Loading..."},"localOverride":false},"CachedAsset:text:en_US-components/common/EmailVerification-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/common/EmailVerification-1743097587452","value":{"email.verification.title":"Email Verification Required","email.verification.message.update.email":"To participate in the community, you must first verify your email address. The verification email was sent to {email}. To change your email, visit My Settings.","email.verification.message.resend.email":"To participate in the community, you must first verify your email address. The verification email was sent to {email}. Resend email."},"localOverride":false},"CachedAsset:text:en_US-pages/tags/TagPage-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-pages/tags/TagPage-1743097587452","value":{"tagPageTitle":"Tag:\"{tagName}\" | {communityTitle}","tagPageForNodeTitle":"Tag:\"{tagName}\" in \"{title}\" | {communityTitle}","name":"Tags Page","tag":"Tag: {tagName}"},"localOverride":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bi0zNC0xM2k0MzE3N0Q2NjFBRDg5NDAy\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bi0zNC0xM2k0MzE3N0Q2NjFBRDg5NDAy","mimeType":"image/png"},"Category:category:Articles":{"__typename":"Category","id":"category:Articles","entityType":"CATEGORY","displayId":"Articles","nodeType":"category","depth":1,"title":"Articles","shortTitle":"Articles","parent":{"__ref":"Category:category:top"},"categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:top":{"__typename":"Category","id":"category:top","displayId":"top","nodeType":"category","depth":0,"title":"Top"},"Tkb:board:TechnicalArticles":{"__typename":"Tkb","id":"board:TechnicalArticles","entityType":"TKB","displayId":"TechnicalArticles","nodeType":"board","depth":2,"conversationStyle":"TKB","title":"Technical Articles","description":"F5 SMEs share good practice.","avatar":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bi0zNC0xM2k0MzE3N0Q2NjFBRDg5NDAy\"}"},"profileSettings":{"__typename":"ProfileSettings","language":null},"parent":{"__ref":"Category:category:Articles"},"ancestors":{"__typename":"CoreNodeConnection","edges":[{"__typename":"CoreNodeEdge","node":{"__ref":"Community:community:zihoc95639"}},{"__typename":"CoreNodeEdge","node":{"__ref":"Category:category:Articles"}}]},"userContext":{"__typename":"NodeUserContext","canAddAttachments":false,"canUpdateNode":false,"canPostMessages":false,"isSubscribed":false},"boardPolicies":{"__typename":"BoardPolicies","canPublishArticleOnCreate":{"__typename":"PolicyResult","failureReason":{"__typename":"FailureReason","message":"error.lithium.policies.forums.policy_can_publish_on_create_workflow_action.accessDenied","key":"error.lithium.policies.forums.policy_can_publish_on_create_workflow_action.accessDenied","args":[]}},"canReadNode":{"__typename":"PolicyResult","failureReason":null}},"tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"shortTitle":"Technical Articles","tagPolicies":{"__typename":"TagPolicies","canSubscribeTagOnNode":{"__typename":"PolicyResult","failureReason":{"__typename":"FailureReason","message":"error.lithium.policies.labels.action.corenode.subscribe_labels.allow.accessDenied","key":"error.lithium.policies.labels.action.corenode.subscribe_labels.allow.accessDenied","args":[]}},"canManageTagDashboard":{"__typename":"PolicyResult","failureReason":{"__typename":"FailureReason","message":"error.lithium.policies.labels.action.corenode.admin_labels.allow.accessDenied","key":"error.lithium.policies.labels.action.corenode.admin_labels.allow.accessDenied","args":[]}}}},"CachedAsset:quilt:f5.prod:pages/tags/TagPage:board:TechnicalArticles-1743097590090":{"__typename":"CachedAsset","id":"quilt:f5.prod:pages/tags/TagPage:board:TechnicalArticles-1743097590090","value":{"id":"TagPage","container":{"id":"Common","headerProps":{"removeComponents":["community.widget.bannerWidget"],"__typename":"QuiltContainerSectionProps"},"items":[{"id":"tag-header-widget","layout":"ONE_COLUMN","bgColor":"var(--lia-bs-white)","showBorder":"BOTTOM","sectionEditLevel":"LOCKED","columnMap":{"main":[{"id":"tags.widget.TagsHeaderWidget","__typename":"QuiltComponent"}],"__typename":"OneSectionColumns"},"__typename":"OneColumnQuiltSection"},{"id":"messages-list-for-tag-widget","layout":"ONE_COLUMN","columnMap":{"main":[{"id":"messages.widget.messageListForNodeByRecentActivityWidget","props":{"viewVariant":{"type":"inline","props":{"useUnreadCount":true,"useViewCount":true,"useAuthorLogin":true,"clampBodyLines":3,"useAvatar":true,"useBoardIcon":false,"useKudosCount":true,"usePreviewMedia":true,"useTags":false,"useNode":true,"useNodeLink":true,"useTextBody":true,"truncateBodyLength":-1,"useBody":true,"useRepliesCount":true,"useSolvedBadge":true,"timeStampType":"conversation.lastPostingActivityTime","useMessageTimeLink":true,"clampSubjectLines":2}},"panelType":"divider","useTitle":false,"hideIfEmpty":false,"pagerVariant":{"type":"loadMore"},"style":"list","showTabs":true,"tabItemMap":{"default":{"mostRecent":true,"mostRecentUserContent":false,"newest":false},"additional":{"mostKudoed":true,"mostViewed":true,"mostReplies":false,"noReplies":false,"noSolutions":false,"solutions":false}}},"__typename":"QuiltComponent"}],"__typename":"OneSectionColumns"},"__typename":"OneColumnQuiltSection"}],"__typename":"QuiltContainer"},"__typename":"Quilt"},"localOverride":false},"CachedAsset:quiltWrapper:f5.prod:Common:1742462921290":{"__typename":"CachedAsset","id":"quiltWrapper:f5.prod:Common:1742462921290","value":{"id":"Common","header":{"backgroundImageProps":{"assetName":"header.jpg","backgroundSize":"COVER","backgroundRepeat":"NO_REPEAT","backgroundPosition":"LEFT_CENTER","lastModified":"1702932449000","__typename":"BackgroundImageProps"},"backgroundColor":"transparent","items":[{"id":"custom.widget.Beta_MetaNav","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"community.widget.navbarWidget","props":{"showUserName":false,"showRegisterLink":true,"style":{"boxShadow":"var(--lia-bs-box-shadow-sm)","linkFontWeight":"700","controllerHighlightColor":"hsla(30, 100%, 50%)","dropdownDividerMarginBottom":"10px","hamburgerBorderHover":"none","linkFontSize":"15px","linkBoxShadowHover":"none","backgroundOpacity":0.4,"controllerBorderRadius":"var(--lia-border-radius-50)","hamburgerBgColor":"transparent","linkTextBorderBottom":"none","hamburgerColor":"var(--lia-nav-controller-icon-color)","brandLogoHeight":"48px","linkLetterSpacing":"normal","linkBgHoverColor":"transparent","collapseMenuDividerOpacity":0.16,"paddingBottom":"10px","dropdownPaddingBottom":"15px","dropdownMenuOffset":"2px","hamburgerBgHoverColor":"transparent","borderBottom":"0","hamburgerBorder":"none","dropdownPaddingX":"10px","brandMarginRightSm":"10px","linkBoxShadow":"none","linkJustifyContent":"center","linkColor":"var(--lia-bs-primary)","collapseMenuDividerBg":"var(--lia-nav-link-color)","dropdownPaddingTop":"10px","controllerHighlightTextColor":"var(--lia-yiq-dark)","background":{"imageAssetName":"","color":"var(--lia-bs-white)","size":"COVER","repeat":"NO_REPEAT","position":"CENTER_CENTER","imageLastModified":""},"linkBorderRadius":"var(--lia-bs-border-radius-sm)","linkHoverColor":"var(--lia-bs-primary)","position":"FIXED","linkBorder":"none","linkTextBorderBottomHover":"2px solid #0C5C8D","brandMarginRight":"30px","hamburgerHoverColor":"var(--lia-nav-controller-icon-color)","linkBorderHover":"none","collapseMenuMarginLeft":"20px","linkFontStyle":"NORMAL","linkPaddingX":"10px","paddingTop":"10px","linkPaddingY":"5px","linkTextTransform":"NONE","dropdownBorderColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","controllerBgHoverColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.1)","linkDropdownPaddingX":"var(--lia-nav-link-px)","linkBgColor":"transparent","linkDropdownPaddingY":"9px","controllerIconColor":"#0C5C8D","dropdownDividerMarginTop":"10px","linkGap":"10px","controllerIconHoverColor":"#0C5C8D"},"links":{"sideLinks":[],"mainLinks":[{"children":[{"linkType":"INTERNAL","id":"migrated-link-1","params":{"boardId":"TechnicalForum","categoryId":"Forums"},"routeName":"ForumBoardPage"},{"linkType":"INTERNAL","id":"migrated-link-2","params":{"boardId":"WaterCooler","categoryId":"Forums"},"routeName":"ForumBoardPage"}],"linkType":"INTERNAL","id":"migrated-link-0","params":{"categoryId":"Forums"},"routeName":"CategoryPage"},{"children":[{"linkType":"INTERNAL","id":"migrated-link-4","params":{"boardId":"codeshare","categoryId":"CrowdSRC"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"migrated-link-5","params":{"boardId":"communityarticles","categoryId":"CrowdSRC"},"routeName":"TkbBoardPage"}],"linkType":"INTERNAL","id":"migrated-link-3","params":{"categoryId":"CrowdSRC"},"routeName":"CategoryPage"},{"children":[{"linkType":"INTERNAL","id":"migrated-link-7","params":{"boardId":"TechnicalArticles","categoryId":"Articles"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"article-series","params":{"boardId":"article-series","categoryId":"Articles"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"security-insights","params":{"boardId":"security-insights","categoryId":"Articles"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"migrated-link-8","params":{"boardId":"DevCentralNews","categoryId":"Articles"},"routeName":"TkbBoardPage"}],"linkType":"INTERNAL","id":"migrated-link-6","params":{"categoryId":"Articles"},"routeName":"CategoryPage"},{"children":[{"linkType":"INTERNAL","id":"migrated-link-10","params":{"categoryId":"CommunityGroups"},"routeName":"CategoryPage"},{"linkType":"INTERNAL","id":"migrated-link-11","params":{"categoryId":"F5-Groups"},"routeName":"CategoryPage"}],"linkType":"INTERNAL","id":"migrated-link-9","params":{"categoryId":"GroupsCategory"},"routeName":"CategoryPage"},{"children":[],"linkType":"INTERNAL","id":"migrated-link-12","params":{"boardId":"Events","categoryId":"top"},"routeName":"EventBoardPage"},{"children":[],"linkType":"INTERNAL","id":"migrated-link-13","params":{"boardId":"Suggestions","categoryId":"top"},"routeName":"IdeaBoardPage"},{"children":[],"linkType":"EXTERNAL","id":"Common-external-link","url":"https://community.f5.com/c/how-do-i","target":"SELF"}]},"className":"QuiltComponent_lia-component-edit-mode__lQ9Z6","showSearchIcon":false},"__typename":"QuiltComponent"},{"id":"community.widget.bannerWidget","props":{"backgroundColor":"transparent","visualEffects":{"showBottomBorder":false},"backgroundImageProps":{"backgroundSize":"COVER","backgroundPosition":"CENTER_CENTER","backgroundRepeat":"NO_REPEAT"},"fontColor":"#222222"},"__typename":"QuiltComponent"},{"id":"community.widget.breadcrumbWidget","props":{"backgroundColor":"var(--lia-bs-primary)","linkHighlightColor":"#FFFFFF","visualEffects":{"showBottomBorder":false},"backgroundOpacity":60,"linkTextColor":"#FFFFFF"},"__typename":"QuiltComponent"}],"__typename":"QuiltWrapperSection"},"footer":{"backgroundImageProps":{"assetName":null,"backgroundSize":"COVER","backgroundRepeat":"NO_REPEAT","backgroundPosition":"CENTER_CENTER","lastModified":null,"__typename":"BackgroundImageProps"},"backgroundColor":"var(--lia-bs-body-color)","items":[{"id":"custom.widget.Beta_Footer","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"custom.widget.Tag_Manager_Helper","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"custom.widget.Consent_Blackbar","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"}],"__typename":"QuiltWrapperSection"},"__typename":"QuiltWrapper","localOverride":false},"localOverride":false},"CachedAsset:text:en_US-components/common/ActionFeedback-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/common/ActionFeedback-1743097587452","value":{"joinedGroupHub.title":"Welcome","joinedGroupHub.message":"You are now a member of this group and are subscribed to updates.","groupHubInviteNotFound.title":"Invitation Not Found","groupHubInviteNotFound.message":"Sorry, we could not find your invitation to the group. The owner may have canceled the invite.","groupHubNotFound.title":"Group Not Found","groupHubNotFound.message":"The grouphub you tried to join does not exist. It may have been deleted.","existingGroupHubMember.title":"Already Joined","existingGroupHubMember.message":"You are already a member of this group.","accountLocked.title":"Account Locked","accountLocked.message":"Your account has been locked due to multiple failed attempts. Try again in {lockoutTime} minutes.","editedGroupHub.title":"Changes Saved","editedGroupHub.message":"Your group has been updated.","leftGroupHub.title":"Goodbye","leftGroupHub.message":"You are no longer a member of this group and will not receive future updates.","deletedGroupHub.title":"Deleted","deletedGroupHub.message":"The group has been deleted.","groupHubCreated.title":"Group Created","groupHubCreated.message":"{groupHubName} is ready to use","accountClosed.title":"Account Closed","accountClosed.message":"The account has been closed and you will now be redirected to the homepage","resetTokenExpired.title":"Reset Password Link has Expired","resetTokenExpired.message":"Try resetting your password again","invalidUrl.title":"Invalid URL","invalidUrl.message":"The URL you're using is not recognized. Verify your URL and try again.","accountClosedForUser.title":"Account Closed","accountClosedForUser.message":"{userName}'s account is closed","inviteTokenInvalid.title":"Invitation Invalid","inviteTokenInvalid.message":"Your invitation to the community has been canceled or expired.","inviteTokenError.title":"Invitation Verification Failed","inviteTokenError.message":"The url you are utilizing is not recognized. Verify your URL and try again","pageNotFound.title":"Access Denied","pageNotFound.message":"You do not have access to this area of the community or it doesn't exist","eventAttending.title":"Responded as Attending","eventAttending.message":"You'll be notified when there's new activity and reminded as the event approaches","eventInterested.title":"Responded as Interested","eventInterested.message":"You'll be notified when there's new activity and reminded as the event approaches","eventNotFound.title":"Event Not Found","eventNotFound.message":"The event you tried to respond to does not exist.","redirectToRelatedPage.title":"Showing Related Content","redirectToRelatedPageForBaseUsers.title":"Showing Related Content","redirectToRelatedPageForBaseUsers.message":"The content you are trying to access is archived","redirectToRelatedPage.message":"The content you are trying to access is archived","relatedUrl.archivalLink.flyoutMessage":"The content you are trying to access is archived View Archived Content"},"localOverride":false},"CachedAsset:component:custom.widget.Beta_MetaNav-en-1742463042522":{"__typename":"CachedAsset","id":"component:custom.widget.Beta_MetaNav-en-1742463042522","value":{"component":{"id":"custom.widget.Beta_MetaNav","template":{"id":"Beta_MetaNav","markupLanguage":"HANDLEBARS","style":null,"texts":null,"defaults":{"config":{"applicablePages":[],"description":"MetaNav menu at the top of every page.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Beta_MetaNav","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"CUSTOM","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"MetaNav menu at the top of every page.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Beta_Footer-en-1742463042522":{"__typename":"CachedAsset","id":"component:custom.widget.Beta_Footer-en-1742463042522","value":{"component":{"id":"custom.widget.Beta_Footer","template":{"id":"Beta_Footer","markupLanguage":"HANDLEBARS","style":null,"texts":null,"defaults":{"config":{"applicablePages":[],"description":"DevCentral´s custom footer.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Beta_Footer","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"CUSTOM","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"DevCentral´s custom footer.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Tag_Manager_Helper-en-1742463042522":{"__typename":"CachedAsset","id":"component:custom.widget.Tag_Manager_Helper-en-1742463042522","value":{"component":{"id":"custom.widget.Tag_Manager_Helper","template":{"id":"Tag_Manager_Helper","markupLanguage":"HANDLEBARS","style":null,"texts":null,"defaults":{"config":{"applicablePages":[],"description":"Helper widget to inject Tag Manager scripts into head element","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Tag_Manager_Helper","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"CUSTOM","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"Helper widget to inject Tag Manager scripts into head element","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Consent_Blackbar-en-1742463042522":{"__typename":"CachedAsset","id":"component:custom.widget.Consent_Blackbar-en-1742463042522","value":{"component":{"id":"custom.widget.Consent_Blackbar","template":{"id":"Consent_Blackbar","markupLanguage":"HTML","style":null,"texts":null,"defaults":{"config":{"applicablePages":[],"description":"","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Consent_Blackbar","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"TEXTHTML","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:text:en_US-components/community/Breadcrumb-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/community/Breadcrumb-1743097587452","value":{"navLabel":"Breadcrumbs","dropdown":"Additional parent page navigation"},"localOverride":false},"CachedAsset:text:en_US-components/tags/TagsHeaderWidget-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/tags/TagsHeaderWidget-1743097587452","value":{"tag":"{tagName}","topicsCount":"{count} {count, plural, one {Topic} other {Topics}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageListForNodeByRecentActivityWidget-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageListForNodeByRecentActivityWidget-1743097587452","value":{"title@userScope:other":"Recent Content","title@userScope:self":"Contributions","title@board:FORUM@userScope:other":"Recent Discussions","title@board:BLOG@userScope:other":"Recent Blogs","emptyDescription":"No content to show","MessageListForNodeByRecentActivityWidgetEditor.nodeScope.label":"Scope","title@instance:1706288370055":"Content Feed","title@instance:1743095186784":"Most Recent Updates","title@instance:1704317906837":"Content Feed","title@instance:1743095018194":"Most Recent Updates","title@instance:1702668293472":"Community Feed","title@instance:1743095117047":"Most Recent Updates","title@instance:1704319314827":"Blog Feed","title@instance:1743095235555":"Most Recent Updates","title@instance:1704320290851":"My Contributions","title@instance:1703720491809":"Forum Feed","title@instance:1743095311723":"Most Recent Updates","title@instance:1703028709746":"Group Content Feed","title@instance:VTsglH":"Content Feed"},"localOverride":false},"Category:category:Forums":{"__typename":"Category","id":"category:Forums","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Forum:board:TechnicalForum":{"__typename":"Forum","id":"board:TechnicalForum","forumPolicies":{"__typename":"ForumPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Forum:board:WaterCooler":{"__typename":"Forum","id":"board:WaterCooler","forumPolicies":{"__typename":"ForumPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:DevCentralNews":{"__typename":"Tkb","id":"board:DevCentralNews","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:GroupsCategory":{"__typename":"Category","id":"category:GroupsCategory","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:F5-Groups":{"__typename":"Category","id":"category:F5-Groups","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:CommunityGroups":{"__typename":"Category","id":"category:CommunityGroups","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Occasion:board:Events":{"__typename":"Occasion","id":"board:Events","boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"occasionPolicies":{"__typename":"OccasionPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Idea:board:Suggestions":{"__typename":"Idea","id":"board:Suggestions","boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"ideaPolicies":{"__typename":"IdeaPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:CrowdSRC":{"__typename":"Category","id":"category:CrowdSRC","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:codeshare":{"__typename":"Tkb","id":"board:codeshare","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:communityarticles":{"__typename":"Tkb","id":"board:communityarticles","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:security-insights":{"__typename":"Tkb","id":"board:security-insights","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:article-series":{"__typename":"Tkb","id":"board:article-series","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Conversation:conversation:306550":{"__typename":"Conversation","id":"conversation:306550","topic":{"__typename":"TkbTopicMessage","uid":306550},"lastPostingActivityTime":"2025-01-15T10:02:24.982-08:00","solved":false},"User:user:242856":{"__typename":"User","uid":242856,"login":"MichaelOLeary","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0yNDI4NTYtMjA2NzVpMjAwQzU1OUQzMEFFMDM2RQ"},"id":"user:242856"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE5MzdpMDUxQjQzRTRDMTIyRTZFNQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE5MzdpMDUxQjQzRTRDMTIyRTZFNQ?revision=68","title":"k8s.jpg","associationType":"COVER","width":500,"height":500,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1MDZpNjFFNDAwMzE1RkI2MTAwQQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1MDZpNjFFNDAwMzE1RkI2MTAwQQ?revision=68","title":"k8s-arch-options.png","associationType":"BODY","width":699,"height":263,"altText":"k8s-arch-options.png"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzNpQjkyNDZCQjVGMzc1RUQ0MQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzNpQjkyNDZCQjVGMzc1RUQ0MQ?revision=68","title":"secure-k8s-gw.PNG","associationType":"BODY","width":2508,"height":1201,"altText":"secure-k8s-gw.PNG"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzZpNUNEOUUzOUMxNEFCOTI2NQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzZpNUNEOUUzOUMxNEFCOTI2NQ?revision=68","title":"k8s-site.PNG","associationType":"BODY","width":2518,"height":1183,"altText":"k8s-site.PNG"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1NzFpQUQ1NUIyM0U2MDcyMEMyOQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1NzFpQUQ1NUIyM0U2MDcyMEMyOQ?revision=68","title":"vk8s.PNG","associationType":"BODY","width":2165,"height":1284,"altText":"vk8s.PNG"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1OTBpMkMwRDFFNTcxNjI1NUQ4Qg?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1OTBpMkMwRDFFNTcxNjI1NUQ4Qg?revision=68","title":"mk8s.PNG","associationType":"BODY","width":2359,"height":1339,"altText":"mk8s.PNG"},"TkbTopicMessage:message:306550":{"__typename":"TkbTopicMessage","subject":"Kubernetes architecture options with F5 Distributed Cloud Services","conversation":{"__ref":"Conversation:conversation:306550"},"id":"message:306550","revisionNum":68,"uid":306550,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:242856"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":9651},"postTime":"2023-01-11T09:22:19.033-08:00","lastPublishTime":"2025-01-15T10:02:24.982-08:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" \n Summary \n F5 Distributed Cloud Services (F5 XC) can both integrate with your existing Kubernetes (K8s) clusters and/or host a K8s workload itself. Within these distinctions, we have multiple architecture options. This article explores four major architectures in ascending order of sophistication and advantages. \n \n Architecture #1: External Load Balancer (Secure K8s Gateway) \n Architecture #2: CE as a pod (K8s site) \n Architecture #3: Managed Namespace (vK8s) \n Architecture #4: Managed K8s (mK8s) \n \n Kubernetes Architecture Options \n As K8s continues to grow, options for how we run K8s and integrate with existing K8s platforms continue to grow. F5 XC can both integrate with your existing K8s clusters and/or run a managed K8s platform itself. Multiple architectures exist within these offerings too, so I was thoroughly confused when I first heard about these possibilities. \n A colleague recently laid it out for me in a conversation: \n \"Michael, listen up: XC can either integrate with your K8s platform, run inside your K8s platform, host virtual K8s (Namespace-aaS), or run a K8s platform in your environment.\" I replied, \"That's great. Now I have a mental model for differentiating between architecture options.\" \n This article will overview these architectures and provide 101-level context: when, how, and why would you implement these options? \n Side note 1: F5 XC concepts and terms \n F5 XC is a global platform that can provide networking and app delivery services, as well as compute (K8s workloads). We call each of our global PoP's a Regional Edge (RE). RE's are highly meshed to form the backbone of the global platform. They connect your sites, they can expose your services to the Internet, and they can run workloads. This platform is extensible into your data center by running one or more XC Nodes in your network, also called a Customer Edge (CE). A CE is a compute node in your network that registers to our global control plane and is then managed by a customer as SaaS. \n The registration of one or more CE's creates a customer site in F5 XC. A CE can run on a hypervisor (VMWare/KVM/Etc), a Hyperscaler (AWS, Azure, GCP, etc), baremetal, or even as a k8s pod, and can be deployed in HA clusters. \n XC Mesh functionality provides connectivity between sites, security services, and observability. Optionally, in addition, XC App Stack functionality allows a large and arbitrary number of managed clusters to be logically grouped into a virtual site with a single K8s mgmt interface. So where Mesh services provide the networking, App Stack services provide the Kubernetes compute mgmt. Our first 2 architectures require Mesh services only, and our last two require App Stack. \n Side note 2: Service-to-service communication \n I'm often asked how to allow services between clusters to communicate with each other. This is possible and easy with XC. Each site can publish services to every other site, including K8s sites. This means that any K8s service can be reachable from other sites you choose. And this can be true in any of the architectures below, although more granular controls are possible with the more sophisticated architectures. I'll explore this common question more in a separate article. \n Architecture 1: External Load Balancer (Secure K8s Gateway) \n In a Secure Kubernetes Gateway architecture, you have integration with your existing K8s platform, using the XC node as the external load balancer for your K8s cluster. In this scenario, you create a ServiceAccount and kubeconfig file to configure XC. The XC node then performs service discovery against your K8s API server. I've covered this process in a previous article, but the advantage is that you can integrate with existing K8s platforms. This allows exposing both NodePort and ClusterIP services via the XC node. \n \n XC is not hosting any workloads in this architecture, but it is exposing your services to your local network, or remote sites, or the Internet. In the diagram above, I show a web application being accesssed from a remote site (and/or the Internet) where the origin pool is a NodePort service discovered in a K8s cluster. \n Architecture 2: Run a site within a K8s cluster (K8s site type) \n Creating a K8s site is easy - just deploy a single manifest found here. This file deploys multiple resources in your cluster, and together these resources work to provide the services of a CE, and create a customer site. I've heard this referred to as \"running a CE inside of K8s\" or \"running your CE as a pod\". However, when I say \"CE node\" I'm usually referring to a discreet compute node like a VM or piece of hardware; this architecture is actually a group of pods and related resources that run within K8s to create a XC customer site. \n With XC running inside your existing cluster, you can expose services within the cluster by DNS name because the site will resolve these from within the cluster. Your service can then be exposed anywhere by the F5 XC platform. This is similar to Architecture 1 above, but with this model, your site is simply a group of pods within K8s. An advantage here is the ability to expose services of other types (e.g. ClusterIP). \n A site deployed into a K8s cluster will only support Mesh functionality and does not support AppStack functionality (i.e., you cannot run a cluster within your cluster). In this architecture, XC acts as a K8s ingress controller with built-in application security. It also enables Mesh features, such as publishing of other sites' services on this site, and publishing of this site's discovered services on other sites. \n Architecture 3: vK8s (Namespace-as-a-Service) \n If the services you use include AppStack capabilities, then architectures #3 and #4 are possible for you. In these scenarios, our XC node actually runs your K8s on your workloads. We are no longer integrating XC with your existing K8s platform. XC is the platform. \n A simple way to run K8s workloads is to use a virtual k8s (vK8s) architecture. This could be referred to as a \"managed Namespace\" because by creating a vK8s object in XC you get a single namespace in a virtual cluster. \n Your Namespace can be fully hosted (deployed to RE's) or run on your VM's (CE's), or both. Your kubeconfig file will allow access to your Namespace via the hosted API server. Via your regular kubectl CLI (or via the web console) you can create/delete/manage K8s resources (Deployments, Services, Secrets, ServiceAccounts, etc) and view application resource metrics. \n This is great if you have workloads that you want to deploy to remote regions where you do not have infrastructure and would prefer to run in F5's RE's, or if you have disparate clusters across multiple sites and you'd like to manage multiple K8s clusters via a single centralized, virtual cluster. \n Best practice guard rails for vK8s \n With a vK8s architecture, you don't have your own cluster, but rather a managed Namespace. So there are some restrictions (for example, you cannot run a container as root, bind to a privileged port, or to the Host network). You cannot create CRD's, ClusterRoles, PodSecurityPolicies, or Namespaces, so K8s operators are not supported. In short, you don't have a managed cluster, but a managed Namespace on a virtual cluster. \n Architecture 4: mK8s (Managed K8s) \n In managed k8s (mk8s, also known as physical K8s or pk8s) deployment, we have an enterprise-level K8s distribution that is run at your site. This means you can use XC to deploy/manage/upgrade K8s infrastructure, but you manage the Kubernetes resources. The benefits include what is typical for 3rd-party K8s mgmt solutions, but also some key differentiators: \n \n multi-cloud, with automation for Azure, AWS, and GCP environments \n consumed by you as SaaS \n enterprise-level traffic control natively \n allows a large and arbitrary number of managed clusters to be logically managed with a single K8s mgmt interface \n \n You can enable kubectl access against your local cluster and disable the hosted API server, so your kubeconfig file can point to a global URL or a local endpoint on-prem. \n Another benefit of mK8s is that you are running a full K8s cluster at your site, not just a Namespace in a virtual cluster. The restrictions that apply to vK8s (see above) do not apply to mK8s, so you could run privileged pods if required, use Operators that make use of ClusterRoles and CRDs, and perform other tasks that require cluster-wide access. \n Traffic management controls with mK8s \n Because your workloads run in a cluster managed by XC, we can apply more sophisticated and native policies to K8s traffic than non-managed clusters in earlier architectures: \n \n Service isolation can be enforced within the cluster, so that pods in a given namespace cannot communicate with services outside of that namespace, by default. \n More service-to-service controls exist so that you can decide which services can reach with other services with more granularity. \n Egress control can be natively enforced for outbound traffic from the cluster, by namespace, labels, IP ranges, or other methods. E.g.: Svc A can reach myapi.example.com but no other Internet service. \n WAF policies, bot defense, L3/4 policies, etc—all of these policies that you have typically applied with network firewalls, WAF's, etc—can be applied natively within the platform. \n \n This architecture took me a long time to understand, and longer to fully appreciate. But once you have run your workloads natively on a managed K8s platform that is connected to a global backbone and capable of performing network and application delivery within the platform, the security and traffic mgmt benefits become very compelling. \n Conclusion: \n As K8s continues to expand, management solutions of your clusters make it possible to secure your K8s services, whether they are managed by XC or exist in disparate clusters. With F5 XC as a global platform consumed as a service—not a discreet installation managed by you—the available architectures here are unique and therefore can accommodate the diverse (and changing!) ways we see K8s run today. \n Related Articles \n \n Securely connecting Kubernetes Microservices with F5 Distributed Cloud \n Multi-cluster Multi-cloud Networking for K8s with F5 Distributed Cloud - Architecture Pattern \n Multiple Kubernetes Clusters and Path-Based Routing with F5 Distributed Cloud \n ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"10582","kudosSumWeight":29,"repliesCount":5,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE5MzdpMDUxQjQzRTRDMTIyRTZFNQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1MDZpNjFFNDAwMzE1RkI2MTAwQQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzNpQjkyNDZCQjVGMzc1RUQ0MQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzZpNUNEOUUzOUMxNEFCOTI2NQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1NzFpQUQ1NUIyM0U2MDcyMEMyOQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1OTBpMkMwRDFFNTcxNjI1NUQ4Qg?revision=68\"}"}}],"totalCount":6,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:337953":{"__typename":"Conversation","id":"conversation:337953","topic":{"__typename":"TkbTopicMessage","uid":337953},"lastPostingActivityTime":"2024-12-30T05:00:00.049-08:00","solved":false},"User:user:430188":{"__typename":"User","uid":430188,"login":"mpstefan","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS00MzAxODgtWU9DT3BL?image-coordinates=0%2C0%2C200%2C200"},"id":"user:430188"},"TkbTopicMessage:message:337953":{"__typename":"TkbTopicMessage","subject":"Announcing F5 NGINX Gateway Fabric 1.5.0 with NGINX Code Snippets","conversation":{"__ref":"Conversation:conversation:337953"},"id":"message:337953","revisionNum":3,"uid":337953,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:430188"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":100},"postTime":"2024-12-30T05:00:00.049-08:00","lastPublishTime":"2024-12-30T05:00:00.049-08:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Today we are announcing the release of F5 NGINX Gateway Fabric 1.5.0 which comes with one of the most prominent features from F5 NGINX Ingress Controller: code snippets. As always, we have made this feature available through an extension of the Gateway API, a new resource called a SnippetFilter. With it, you can define custom NGINX configuration that can be applied to any Route rule! \n In addition to SnippetFilters, some other highlights of this release include: \n \n Ability to retain client IP information via Proxy Pass or XForwardedFor header \n \n \n Configurable NGINX error log level and reduced “info” log verbosity \n \n \n A new Getting Started guide for installing NGF for the first time \n \n \n F5 NGINX Plus R33 support \n \n \n Bug fixes! \n \n You can see the full changelog with details here! \n NGINX Code Snippets via SnippetFilters \n With a 20 year old legacy, there is a massive range of features that F5 NGINX provides that are not yet explicitly exposed through NGINX Gateway Fabric via the Gateway API. If you are familiar with NGINX directives and configuration and just want to use a native NGINX feature, you were unable to because our control plane would overwrite any configuration you wrote. We needed a way for your directives to merge with the configuration we pushed. \n With SnippetFilters, we have not only enabled you to append directives to whatever context you need them in but also built them as an extension to the Gateway API. Once you define a SnippetFilter, any route rule can then use that snippet filter to apply NGINX configuration for the context you supplied. You can even change higher level contexts using these snippets, such as main, to do things like importing modules – just make sure you coordinate with everyone when making these changes! \n As SnippetFilters do open the opportunity for misconfiguration in a way that may impact other application teams using the same Gateway, they do have to be explicitly enabled. But if you find yourself needing a feature from NGINX that is not yet available in NGINX Gateway Fabric, or you want to include your own NGINX logic, the SnippetFilter will fill that gap between our current first-class Gateway API features and the full power of NGINX. \n What’s Next \n Next release we will be focusing on a big architectural shift in NGINX Gateway Fabric that we have been planning for a while: separating our data and control planes. \n Currently, both our control plane (what configures NGINX for us) and our data plane (NGINX itself) are contained within the same pod. Ideally, these should be separate for security, performance, and scalability reasons. As we are also looking at supporting multiple Gateway resources for a single NGINX Gateway Fabric installation, we thought this was a good time to make that change, prompting our next release to be 2.0! \n In 2.0, you’ll see the control plane start as its own pod. As you create Gateways, or as you scale existing ones, the control plane will spin up new instances of NGINX in their own pods. This translates into less overhead, better RBAC security, and the ability to provision multiple Gateway resources. \n For more information, check out our design on the separation! \n Resources \n For the complete changelog for NGINX Gateway Fabric 1.5.0, see the Release Notes. To try NGINX Gateway Fabric for Kubernetes with NGINX Plus, start your free 30-day trial today or contact us to discuss your use cases. \n If you would like to get involved, see what is coming next, or see the source code for NGINX Gateway Fabric, check out our repository on GitHub! \n We have weekly community meetings on Tuesdays at 9:30AM Pacific/12:30PM Eastern/5:30PM GMT. The meeting link, updates, agenda, and notes are on the NGINX Gateway Fabric Meeting Calendar. Links are also always available from our GitHub README. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"3931","kudosSumWeight":0,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:333091":{"__typename":"Conversation","id":"conversation:333091","topic":{"__typename":"TkbTopicMessage","uid":333091},"lastPostingActivityTime":"2024-11-07T05:00:00.052-08:00","solved":false},"User:user:419147":{"__typename":"User","uid":419147,"login":"aconley","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS00MTkxNDctME4wMDV0?image-coordinates=0%2C0%2C2400%2C2400"},"id":"user:419147"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtVzJITnpu?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtVzJITnpu?revision=11","title":"Screenshot 2024-09-17 at 1.25.56 PM.png","associationType":"BODY","width":1186,"height":803,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtR3lhR1hl?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtR3lhR1hl?revision=11","title":"Screenshot 2024-10-15 at 2.31.58 PM.png","associationType":"BODY","width":1755,"height":919,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtOVNnSGpZ?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtOVNnSGpZ?revision=11","title":"Screenshot 2024-10-15 at 2.39.14 PM.png","associationType":"BODY","width":1625,"height":960,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEteUNzTFRZ?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEteUNzTFRZ?revision=11","title":"Screenshot 2024-10-15 at 2.59.06 PM.png","associationType":"BODY","width":1734,"height":544,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtRzhENVhi?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtRzhENVhi?revision=11","title":"Screenshot 2024-10-15 at 3.02.15 PM.png","associationType":"BODY","width":1748,"height":704,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtVHlURThs?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtVHlURThs?revision=11","title":"Screenshot 2024-10-15 at 3.11.01 PM.png","associationType":"BODY","width":1642,"height":704,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtbk5qam9E?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtbk5qam9E?revision=11","title":"Screenshot 2024-10-15 at 3.13.25 PM.png","associationType":"BODY","width":1628,"height":608,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtV0puQm9T?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtV0puQm9T?revision=11","title":"Screenshot 2024-10-15 at 3.15.59 PM.png","associationType":"BODY","width":1618,"height":953,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtT2ZCNVhM?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtT2ZCNVhM?revision=11","title":"Screenshot 2024-10-15 at 3.21.02 PM.png","associationType":"BODY","width":1600,"height":742,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtTllXNWcx?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtTllXNWcx?revision=11","title":"Screenshot 2024-10-15 at 3.22.30 PM.png","associationType":"BODY","width":1604,"height":466,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtNnFPaVVB?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtNnFPaVVB?revision=11","title":"Screenshot 2024-10-15 at 3.52.55 PM.png","associationType":"BODY","width":1746,"height":934,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtYm5qeXdo?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtYm5qeXdo?revision=11","title":"Screenshot 2024-10-15 at 3.54.55 PM.png","associationType":"BODY","width":1742,"height":231,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtYzd4NmRG?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtYzd4NmRG?revision=11","title":"Screenshot 2024-10-15 at 3.59.20 PM.png","associationType":"BODY","width":1632,"height":890,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtTmZlU2l3?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtTmZlU2l3?revision=11","title":"Screenshot 2024-10-15 at 4.02.15 PM.png","associationType":"BODY","width":1617,"height":491,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtcGI1TDlG?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtcGI1TDlG?revision=11","title":"Screenshot 2024-10-15 at 4.29.57 PM.png","associationType":"BODY","width":1629,"height":591,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEta1FQMFVG?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEta1FQMFVG?revision=11","title":"Screenshot 2024-10-15 at 4.07.57 PM.png","associationType":"BODY","width":1625,"height":632,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtcnFLUEl4?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtcnFLUEl4?revision=11","title":"Screenshot 2024-10-15 at 4.10.45 PM.png","associationType":"BODY","width":1100,"height":627,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtdDEwdXo5?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtdDEwdXo5?revision=11","title":"Screenshot 2024-10-15 at 4.13.51 PM.png","associationType":"BODY","width":1595,"height":594,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtN3p2OTMz?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtN3p2OTMz?revision=11","title":"Screenshot 2024-10-15 at 4.17.34 PM.png","associationType":"BODY","width":1624,"height":776,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtT2VpVjV1?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtT2VpVjV1?revision=11","title":"Screenshot 2024-10-15 at 4.19.31 PM.png","associationType":"BODY","width":1618,"height":649,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtMDNmQ0JK?revision=11\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtMDNmQ0JK?revision=11","title":"Screenshot 2024-10-15 at 4.22.10 PM.png","associationType":"BODY","width":746,"height":273,"altText":""},"TkbTopicMessage:message:333091":{"__typename":"TkbTopicMessage","subject":"NGINX Unit running in Distributed Cloud vK8s","conversation":{"__ref":"Conversation:conversation:333091"},"id":"message:333091","revisionNum":11,"uid":333091,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:419147"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":161},"postTime":"2024-11-07T05:00:00.052-08:00","lastPublishTime":"2024-11-07T05:00:00.052-08:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" F5 Distributed Cloud provides a mechanism to easily deploy applications using virtual Kubernetes (vK8s) across a global network. This results in applications running closer to the end user. \n This article will demonstrate creating a NGINX Unit container to run a simple server-side WebAssembly (Wasm) application and deliver it via Distributed Cloud Regional Edge (RE) sites. Distributed Cloud provides a vK8s infrastructure for deploying modern apps as well as load balancing and security services to securely and reliably deliver applications at the edge. \n \n NGINX Unit Pod \n A Kubernetes pod is the smallest execution unit that can be deployed in Kubernetes. A pod is a single instance of an application and may contain single or multiple containers. For this article, a single NGINX Unit container will make up the application pod and replicas of the pod will be deployed to each of the configured Distributed Cloud Points of Presence (PoPs). \n Building the NGINX Unit Container \n The first step is to build a server-side WebAssemby (Wasm) application that Unit can execute. Instructions on creating a simple Hello World Wasm application are available here. After the Hello World Wasm component is built, a NGINX Unit Container needs to be created to execute the application. \n To build the NGINX Unit container, download the Unit Wasm Dockerfile that is available on the NGINX Unit github repo. The Dockerfile needs to be modified to allow the container to run as a non-root user in the Distributed Cloud virtual Kubernetes (vK8s) environment. Below is the Dockerfile that I used for this demonstration. \n FROM debian:bullseye-slim \n\nLABEL org.opencontainers.image.title=\"Unit (wasm)\" \nLABEL org.opencontainers.image.description=\"Official build of Unit for Docker.\" \nLABEL org.opencontainers.image.url=\"https://unit.nginx.org\" \nLABEL org.opencontainers.image.source=\"https://github.com/nginx/unit\" \nLABEL org.opencontainers.image.documentation=\"https://unit.nginx.org/installation/#docker-images\" \nLABEL org.opencontainers.image.vendor=\"NGINX Docker Maintainers <docker-maint@nginx.com>\" \nLABEL org.opencontainers.image.version=\"1.32.1\" \n\nRUN set -ex \\ \n && savedAptMark=\"$(apt-mark showmanual)\" \\ \n && apt-get update \\ \n && apt-get install --no-install-recommends --no-install-suggests -y ca-certificates git build-essential libssl-dev libpcre2-dev curl pkg-config \\ \n && mkdir -p /usr/lib/unit/modules /usr/lib/unit/debug-modules \\ \n && mkdir -p /usr/src/unit \\ \n && mkdir -p /unit/var/lib/unit \\ \n && mkdir -p /unit/var/run \\ \n && mkdir -p /unit/var/log \\ \n && mkdir -p /unit/var/tmp \\ \n && mkdir /app \\ \n && chmod -R 777 /unit/var/lib/unit \\ \n && chmod -R 777 /unit/var/run \\ \n && chmod -R 777 /unit/var/log \\ \n && chmod -R 777 /unit/var/tmp \\ \n && chmod 777 /app \\ \n && cd /usr/src/unit \\ \n && git clone --depth 1 -b 1.32.1-1 https://github.com/nginx/unit \\ \n && cd unit \\ \n && NCPU=\"$(getconf _NPROCESSORS_ONLN)\" \\ \n && DEB_HOST_MULTIARCH=\"$(dpkg-architecture -q DEB_HOST_MULTIARCH)\" \\ \n && CC_OPT=\"$(DEB_BUILD_MAINT_OPTIONS=\"hardening=+all,-pie\" DEB_CFLAGS_MAINT_APPEND=\"-Wp,-D_FORTIFY_SOURCE=2 -fPIC\" dpkg-buildflags --get CFLAGS)\" \\ \n && LD_OPT=\"$(DEB_BUILD_MAINT_OPTIONS=\"hardening=+all,-pie\" DEB_LDFLAGS_MAINT_APPEND=\"-Wl,--as-needed -pie\" dpkg-buildflags --get LDFLAGS)\" \\ \n && CONFIGURE_ARGS_MODULES=\"--prefix=/usr \\ \n --statedir=/unit/var/lib/unit \\ \n --debug \\ \n --control=unix:/unit/var/run/control.unit.sock \\ \n --runstatedir=/unit/var/run \\ \n --pid=/unit/var/run/unit.pid \\ \n --logdir=/unit/var/log \\ \n --log=/unit/var/log/unit.log \\ \n --tmpdir=/unit/var/tmp \\ \n --openssl \\ \n --libdir=/usr/lib/$DEB_HOST_MULTIARCH\" \\ \n && CONFIGURE_ARGS=\"$CONFIGURE_ARGS_MODULES \\ \n --njs\" \\ \n && make -j $NCPU -C pkg/contrib .njs \\ \n && export PKG_CONFIG_PATH=$(pwd)/pkg/contrib/njs/build \\ \n && ./configure $CONFIGURE_ARGS --cc-opt=\"$CC_OPT\" --ld-opt=\"$LD_OPT\" --modulesdir=/usr/lib/unit/debug-modules --debug \\ \n && make -j $NCPU unitd \\ \n && install -pm755 build/sbin/unitd /usr/sbin/unitd-debug \\ \n && make clean \\ \n && ./configure $CONFIGURE_ARGS --cc-opt=\"$CC_OPT\" --ld-opt=\"$LD_OPT\" --modulesdir=/usr/lib/unit/modules \\ \n && make -j $NCPU unitd \\ \n && install -pm755 build/sbin/unitd /usr/sbin/unitd \\ \n && make clean \\ \n && apt-get install --no-install-recommends --no-install-suggests -y libclang-dev \\ \n && export RUST_VERSION=1.76.0 \\ \n && export RUSTUP_HOME=/usr/src/unit/rustup \\ \n && export CARGO_HOME=/usr/src/unit/cargo \\ \n && export PATH=/usr/src/unit/cargo/bin:$PATH \\ \n && dpkgArch=\"$(dpkg --print-architecture)\" \\ \n && case \"${dpkgArch##*-}\" in \\ \n amd64) rustArch=\"x86_64-unknown-linux-gnu\"; rustupSha256=\"0b2f6c8f85a3d02fde2efc0ced4657869d73fccfce59defb4e8d29233116e6db\" ;; \\ \n arm64) rustArch=\"aarch64-unknown-linux-gnu\"; rustupSha256=\"673e336c81c65e6b16dcdede33f4cc9ed0f08bde1dbe7a935f113605292dc800\" ;; \\ \n *) echo >&2 \"unsupported architecture: ${dpkgArch}\"; exit 1 ;; \\ \n esac \\ \n && url=\"https://static.rust-lang.org/rustup/archive/1.26.0/${rustArch}/rustup-init\" \\ \n && curl -L -O \"$url\" \\ \n && echo \"${rustupSha256} *rustup-init\" | sha256sum -c - \\ \n && chmod +x rustup-init \\ \n && ./rustup-init -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION --default-host ${rustArch} \\ \n && rm rustup-init \\ \n && rustup --version \\ \n && cargo --version \\ \n && rustc --version \\ \n && make -C pkg/contrib .wasmtime \\ \n && install -pm 755 pkg/contrib/wasmtime/target/release/libwasmtime.so /usr/lib/$(dpkg-architecture -q DEB_HOST_MULTIARCH)/ \\ \n && ./configure $CONFIGURE_ARGS_MODULES --cc-opt=\"$CC_OPT\" --modulesdir=/usr/lib/unit/debug-modules --debug \\ \n && ./configure wasm --include-path=`pwd`/pkg/contrib/wasmtime/crates/c-api/include --lib-path=/usr/lib/$(dpkg-architecture -q DEB_HOST_MULTIARCH)/ && ./configure wasm-wasi-component \\ \n && make -j $NCPU wasm-install wasm-wasi-component-install \\ \n && make clean \\ \n && ./configure $CONFIGURE_ARGS_MODULES --cc-opt=\"$CC_OPT\" --modulesdir=/usr/lib/unit/modules \\ \n && ./configure wasm --include-path=`pwd`/pkg/contrib/wasmtime/crates/c-api/include --lib-path=/usr/lib/$(dpkg-architecture -q DEB_HOST_MULTIARCH)/ && ./configure wasm-wasi-component \\ \n && make -j $NCPU wasm-install wasm-wasi-component-install \\ \n && cd \\ \n && rm -rf /usr/src/unit \\ \n && for f in /usr/sbin/unitd /usr/lib/unit/modules/*.unit.so; do \\ \n ldd $f | awk '/=>/{print $(NF-1)}' | while read n; do dpkg-query -S $n; done | sed 's/^\\([^:]\\+\\):.*$/\\1/' | sort | uniq >> /requirements.apt; \\ \n done \\ \n && apt-mark showmanual | xargs apt-mark auto > /dev/null \\ \n && { [ -z \"$savedAptMark\" ] || apt-mark manual $savedAptMark; } \\ \n && /bin/true \\ \n && mkdir -p /var/lib/unit/ \\ \n && mkdir -p /docker-entrypoint.d/ \\ \n && apt-get update \\ \n && apt-get --no-install-recommends --no-install-suggests -y install curl $(cat /requirements.apt) \\ \n && apt-get purge -y --auto-remove build-essential \\ \n && rm -rf /var/lib/apt/lists/* \\ \n && rm -f /requirements.apt \\ \n && ln -sf /dev/stderr /var/log/unit.log \n\nWORKDIR /app \n\nCOPY --chmod=755 hello_wasi_http.wasm /app \nCOPY ./*.json /docker-entrypoint.d/ \nCOPY --chmod=755 docker-entrypoint.sh /usr/local/bin/ \nCOPY welcome.* /usr/share/unit/welcome/ \n\nSTOPSIGNAL SIGTERM \n\nENTRYPOINT [\"/usr/local/bin/docker-entrypoint.sh\"] \nEXPOSE 8000 \nCMD [\"unitd\", \"--no-daemon\", \"--control\", \"unix:/unit/var/run/control.unit.sock\"] \n The changes to the Dockerfile are due to the non-root user not being able to write to the /var directory. Rather than modify the permissions on /var and its subdirectories, this docker file creates a /unit directory that is used for storing process information and logs. \n The docker-entrypoint.sh script also needed to be modified to use the /unit directory structure. The updated docker-entrypoint.sh script is shown below. \n #!/bin/sh \n\nset -e \n\nWAITLOOPS=5 \nSLEEPSEC=1 \n\ncurl_put() \n{ \n RET=$(/usr/bin/curl -s -w '%{http_code}' -X PUT --data-binary @$1 --unix-socket /unit/var/run/control.unit.sock http://localhost/$2) \n RET_BODY=$(echo $RET | /bin/sed '$ s/...$//') \n RET_STATUS=$(echo $RET | /usr/bin/tail -c 4) \n if [ \"$RET_STATUS\" -ne \"200\" ]; then \n echo \"$0: Error: HTTP response status code is '$RET_STATUS'\" \n echo \"$RET_BODY\" \n return 1 \n else \n echo \"$0: OK: HTTP response status code is '$RET_STATUS'\" \n echo \"$RET_BODY\"\n fi \n return 0 \n} \n\nif [ \"$1\" = \"unitd\" ] || [ \"$1\" = \"unitd-debug\" ]; then \n if /usr/bin/find \"/unit/var/lib/unit/\" -mindepth 1 -print -quit 2>/dev/null | /bin/grep -q .; then \n echo \"$0: /unit/var/lib/unit/ is not empty, skipping initial configuration...\" \n else \n echo \"$0: Launching Unit daemon to perform initial configuration...\" \n /usr/sbin/$1 --control unix:/unit/var/run/control.unit.sock \n\n for i in $(/usr/bin/seq $WAITLOOPS); do \n if [ ! -S /unit/var/run/control.unit.sock ]; then \n echo \"$0: Waiting for control socket to be created...\" \n /bin/sleep $SLEEPSEC \n else \n break \n fi \n done \n # even when the control socket exists, it does not mean unit has finished initialisation \n # this curl call will get a reply once unit is fully launched \n /usr/bin/curl -s -X GET --unix-socket /unit/var/run/control.unit.sock http://localhost/\n \n if /usr/bin/find \"/docker-entrypoint.d/\" -mindepth 1 -print -quit 2>/dev/null | /bin/grep -q .; then \n echo \"$0: /docker-entrypoint.d/ is not empty, applying initial configuration...\" \n\n echo \"$0: Looking for certificate bundles in /docker-entrypoint.d/...\" \n for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name \"*.pem\"); do \n echo \"$0: Uploading certificates bundle: $f\" \n curl_put $f \"certificates/$(basename $f .pem)\" \n done\n \n echo \"$0: Looking for JavaScript modules in /docker-entrypoint.d/...\" \n for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name \"*.js\"); do \n echo \"$0: Uploading JavaScript module: $f\" \n curl_put $f \"js_modules/$(basename $f .js)\" \n done\n \n echo \"$0: Looking for configuration snippets in /docker-entrypoint.d/...\" \n for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name \"*.json\"); do \n echo \"$0: Applying configuration $f\"; \n curl_put $f \"config\" \n done\n \n echo \"$0: Looking for shell scripts in /docker-entrypoint.d/...\" \n for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -name \"*.sh\"); do \n echo \"$0: Launching $f\"; \n \"$f\" \n done\n \n # warn on filetypes we don't know what to do with \n for f in $(/usr/bin/find /docker-entrypoint.d/ -type f -not -name \"*.sh\" -not -name \"*.json\" -not -name \"*.pem\" -not -name \"*.js\"); do \n echo \"$0: Ignoring $f\"; \n done \n else \n echo \"$0: /docker-entrypoint.d/ is empty, creating 'welcome' configuration...\" \n curl_put /usr/share/unit/welcome/welcome.json \"config\" \n fi\n \n echo \"$0: Stopping Unit daemon after initial configuration...\" \n kill -TERM $(/bin/cat /unit/var/run/unit.pid) \n\n for i in $(/usr/bin/seq $WAITLOOPS); do \n if [ -S /unit/var/run/control.unit.sock ]; then \n echo \"$0: Waiting for control socket to be removed...\" \n /bin/sleep $SLEEPSEC \n else \n break \n fi \n done \n if [ -S /unit/var/run/control.unit.sock ]; then \n kill -KILL $(/bin/cat /unit/var/run/unit.pid) \n rm -f /unit/var/run/control.unit.sock \n fi\n \n echo \n echo \"$0: Unit initial configuration complete; ready for start up...\" \n echo \n fi \nfi \n\nexec \"$@\" \n The Dockerfile copies all files ending in .json to a directory named /docker-entrypoint.d. The docker-entrypoint.sh script checks the /docker-entrypoint.d directory for configuration files that are used to configure NGINX Unit. The following config.json file was created in the same directory as the Dockerfile and docker-entrypoint.sh script. \n {\n \"listeners\": {\n \"*:8000\": {\n \"pass\": \"applications/wasm\"\n }\n },\n \"applications\": {\n \"wasm\": { \n \"type\": \"wasm-wasi-component\",\n \"component\": \"/app/hello_wasi_http.wasm\"\n }\n }\n} \n This config file instructs NGINX unit to listen on port 8000 and pass any client request to the Wasm application component located at /app/hello_wasi_http.wasm. \n The last step before building the container is to copy the Wasm application component into the same directory as the Dockerfile, docker-entrypoint.sh, and config.json files. With these four files in the same directory, the docker build command is used to create a container: \n docker build --no-cache --platform linux/amd64 -t hello-wasi-xc:1.0 . \n Here I use the --platform option to specify that the container will run on a linux/amd64 platform. I also use -t to give the image a name. The portion of the name after the \":\" can be used for versioning. For example, if I added a new feature to my application, I could create a new image with -t hello-wasi-xc:1.1 to represent version 1.1 of the hello-wasi-xc application. \n Once the image is built, it can be uploaded to the container registry of your choice. I have access to an Azure Container Registry, so that is what I chose to use. Distributed Cloud supports pulling images from both public and private container registries. To push the image to my registry, I had to first tag the image with the registry location ({{registry_name}} should be replaced with the name of your registry): \n docker tag hello-wasi-xc:1.0 {{registry_name}}/examples/hello-wasi-xc:1.0 \n Next, I logged into my Azure registry: \n az login az acr login --name {{registry_name}} \n I then pushed the image to the registry: \n docker push {{registry_name}}/examples/hello-wasi-xc:1.0 \n F5 Distributed Cloud Virtual Kubernetes \n F5 Distributed Cloud Services support a Kubernetes compatible API for centralized orchestration of applications across a fleet of sites (customer sites or F5 Distributed Cloud Regional Edges). Distributed Cloud utilizes a distributed control plane to manage scheduling and scaling of applications across multiple sites. \n Kubernetes Objects \n Virtual Kubernetes supports the following Kubernetes objects: Deployments, StatefulSets, Jobs, CronJob, DaemonSet, Service, ConfigMap, Secrets, PersistentVolumeClaim, ServiceAccount, Role, and Role Binding. For this article I will focus on the Deployments object. \n A deployment is commonly used for stateless applications like web-servers, front-end web-ui, etc. A deployment works well for this use case because the NGINX Unit container is serving up a stateless web application. \n Workload Object \n A workload within Distributed Cloud is used to configure and deploy components of an application in Virtual Kubernetes. Workload encapsulates all the operational characteristics of Kubernetes workload, storage, and network objects (deployments, statefulsets, jobs, persistent volume claims, configmaps, secrets, and services) configuration, as well as configuration related to where the workload is deployed and how it is advertised using L7 or L4 load balancers. Within Distributed Cloud there are four types of workloads: Simple Service, Service, Stateful Service, and Job. \n Namespaces \n In Distributed Cloud, tenant configuration objects are grouped under namespaces. Namespaces can be thought of as administrative domains. Within each Namespace, a user can create an object called vK8s and use that for application management. Each Namespace can have a maximum of one vK8s object. \n Distributed Cloud vK8s Configuration \n vK8s is configured under the Distributed Apps tile within the Distributed Cloud console. \n \n After clicking on that tile, I next created a new Virtual K8s. By clicking Add Virtual K8s from the Virtual K8s menu under Applications. \n \n This brings up the Virtual K8s configuration form. I provided a name for my vK8s site and click Save and Exit. \n \n This initiates the vK8s build. \n Deploy the Application \n Once the vK8s cluster is ready, I click on the name of my vk8s to configure the deployment of my application. \n \n For this demo, I am going to deploy my application via Workloads. I click on Workloads and then Add VK8s Workload. \n \n This brings up the Workload configuration form. I provide my Workload a name, select Service for Workload type and click Configure. Using Service allows me to have more granular controls on how my application will be advertised. \n \n Next, I click Add Item under containers to specify to Distribtued Cloud where it can pull my container image from. \n \n In the resulting form, I supply a name for my container, supply the public registry FQDN along with the container name and version and click Apply. \n \n Next I configure where to advertise the application. I want to manually configure my HTTP LB, so I chose to advertise in Cluster. Manually configuring the LB allows me to configure additional options such as Web Application Firewall (WAF). \n \n Within the Advertise in Cluster form, I specify port 8000, because that is the port my container is listening on. \n \n Click Apply and then click Save and Exit. \n Create an HTTP Load Balancer to Advertise the Application \n The next step is to create an HTTP Load Balancer to expose the application to the Internet. This can be done by clicking Manage, selecting Load Balancers, and clicking on HTTP Load Balancers. \n \n Next, click on Add HTTP Load Balancer to create a new load balancer. \n \n I gave the LB a Name, Domain, Load Balancer Type, and Checked the box to Automatically Manage DNS Records. \n \n Then I configured the Origin Pool by clicking Add Item in the Origin Pool section and then clicking Add Item under the Origin Pool. \n \n On the resulting form, I created a name for my Origin Pool and then clicked Add Item under Origin Servers to add an origin Server. \n \n In the Origin Server form, I selected K8s Service Name of Origin Server on given Sites, Service Name, and provided my vK8s workload name along with my namespace. \n \n I also selected Virtual Site and created a Virtual Site for the REs I wanted to use to access my application and clicked Continue. \n \n Back on the Origin Server form, I selected vK8s Networks on Site from the Select Network on the site drop down and then clicked Apply. \n \n On the Origin Pool form, I specified port 8000 for my origin server Port and then clicked Continue. \n \n I then clicked Apply to return to the LB configuration form. \n I added a WAF policy to protect my application and then clicked Save and Exit. \n \n At this point my application is now accessible on the domain name I specified in the HTTP Load Balancer config (http://aconley-wasm.amer-ent.f5demos.com/). \n \n Conclusion \n This is a basic demo of using NGINX Unit and Distributed Cloud Virtual Kubernetes to deploy a server-side WASM application. The principals in this demo could be expanded to build more complex applications. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"20442","kudosSumWeight":0,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtVzJITnpu?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtR3lhR1hl?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtOVNnSGpZ?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEteUNzTFRZ?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtRzhENVhi?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtVHlURThs?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtbk5qam9E?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDg","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtV0puQm9T?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDk","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtT2ZCNVhM?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDEw","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtTllXNWcx?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDEx","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtNnFPaVVB?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDEy","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtYm5qeXdo?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDEz","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtYzd4NmRG?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE0","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtTmZlU2l3?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE1","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtcGI1TDlG?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE2","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEta1FQMFVG?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE3","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtcnFLUEl4?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE4","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtdDEwdXo5?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE5","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtN3p2OTMz?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDIw","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtT2VpVjV1?revision=11\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDIx","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzMwOTEtMDNmQ0JK?revision=11\"}"}}],"totalCount":21,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:300547":{"__typename":"Conversation","id":"conversation:300547","topic":{"__typename":"TkbTopicMessage","uid":300547},"lastPostingActivityTime":"2024-10-27T14:23:00.033-07:00","solved":false},"User:user:303102":{"__typename":"User","uid":303102,"login":"Ulises_Alonso","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0zMDMxMDItNnNGUVZV?image-coordinates=90%2C0%2C517%2C427"},"id":"user:303102"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNzdpMDI3MDVFRkJEN0I1QTYxOA?revision=12\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNzdpMDI3MDVFRkJEN0I1QTYxOA?revision=12","title":"Break or Extend Kubernetes.png","associationType":"TEASER","width":2088,"height":690,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNjVpNEI4MzY2QTI1OEFGQTBDRg?revision=12\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNjVpNEI4MzY2QTI1OEFGQTBDRg?revision=12","title":"Break or Extend Kubernetes.png","associationType":"BODY","width":2088,"height":690,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNjhpODdDQTI2MTg2MkNDQTY4Qg?revision=12\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNjhpODdDQTI2MTg2MkNDQTY4Qg?revision=12","title":"SPK-software-components.png","associationType":"BODY","width":2048,"height":1152,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNjZpN0FFQThCMkE5Q0E3QTNCRg?revision=12\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNjZpN0FFQThCMkE5Q0E3QTNCRg?revision=12","title":"SPK-general-architecture.png","associationType":"BODY","width":2086,"height":1172,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNzVpQUYzNTM1RTlGQjgyQTA2NA?revision=12\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNzVpQUYzNTM1RTlGQjgyQTA2NA?revision=12","title":"SPK-traffic-flows.png","associationType":"BODY","width":2048,"height":1128,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNzZpRUUyODNDMTMzMDc1NDEwMQ?revision=12\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNzZpRUUyODNDMTMzMDc1NDEwMQ?revision=12","title":"SPK-physical.png","associationType":"BODY","width":2048,"height":1128,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyOTRpREUxNUNENUI4RjM3NjJDNw?revision=12\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyOTRpREUxNUNENUI4RjM3NjJDNw?revision=12","title":"Data plane architecture - security.png","associationType":"BODY","width":2048,"height":1128,"altText":null},"TkbTopicMessage:message:300547":{"__typename":"TkbTopicMessage","subject":"BIG-IP Next SPK: a Kubernetes native ingress and egress gateway for Telco workloads","conversation":{"__ref":"Conversation:conversation:300547"},"id":"message:300547","revisionNum":12,"uid":300547,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:303102"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":" BIG-IP Next SPK: a Kubernetes native ingress and egress gateway for Telco workloads that is CNF agnostic. \n ","introduction":"","metrics":{"__typename":"MessageMetrics","views":2824},"postTime":"2022-09-25T18:00:00.028-07:00","lastPublishTime":"2022-09-25T18:00:00.028-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" \n Kubernetes has a simplified networking model which was designed for general IT workloads which only use TCP/HTTP protocols, and a simplified networking model with a single IP address per POD (the smallest deployable unit of computing in Kubernetes) and a single external gateway. On the other hand, Telco deployments require: \n \n 3GPP protocols support. \n Transitional 4G to 5G facilities. \n Network capabilities to match Telco's networks by providing multiple external network connectivity, allowing different paths depending on the Network Function (NF), dynamic routing, etc... \n \n Hopefully, Kubernetes has been designed to be extensible yet it's up to the software and infrastructure architects to design solutions with good practices by following Kubernetes patterns. It´s usual that NF vendors use additional interfaces (multus CNI) for each NF PODs. By doing this, there is no dynamic advertising of addresses as the PODs in the deployment change or a good way to track these addresses outside the cluster by network elements such as firewalls. \n Alternatively, NF vendors try to hide this complexity by selling turn-key solutions with dedicated Kubernetes clusters for each NF or vendor. Therefore the customer ends up with multiple clusters typically disimilar, ultimately defeating the whole purpose of Kubernetes which aims to homogenize application's environment in a single platform. In a way, this this approach is equivalent to having a load balancer for each application. Also, the management of addresses by external network elements is still inaddecuate because identification of NFs is done in a coarse manner by identifying clusters' addresses. \n These two approaches break Kuberentes patterns by adding complexity in the form of non-homogenous networking to the different NFs. \n In this post we introduce F5 BIG-IP Next Service Proxy Kubernetes (SPK) -- BIG-IP Next SPK for short -- architecture which overcomes these limitations while being Network Functions-agnostic. We will use Red Hat's Openshift as reference platform. \n BIG-IP Next SPK software architecture \n BIG-IP Next SPK is a cloud-native solution which runs inside the Kubernetes cluster and is made out of independent components which can be scaled-out. It's headless software (no graphical UI) and it's managed using the Kubernetes API. The major software components are shown next. \n \n BIG-IP Next SPK's data plane makes use of the widely trusted BIG-IP’s Traffic Management Microkernel (TMM) data plane. This allows for a high performance, dependable product from the start. A dynamic routing component configures the BGP peering with the upstream routers for ECMP load distribution. The BFD feature for fast failure detection is available. The session persistence is a distributed database which allows to store connection related state such as pool member persistence, SNATs, NAT46 translations, etc... This database is backed in Kubernetes Persistent Volume which allows this information to be available even after POD restarts. The controller is the component which interacts with the Kubernetes API that customers use to configure BIG-IP Next SPK. Fluentd is a high-performance industry standard for exposing BIG-IP Next SPK metrics and logs to external tools. \n BIG-IP Next SPK network architecture \n The overall network architecture is shown next. Out of this picture we would like to emphasise the following items: \n \n Independent BIG-IP Next SPK instances, with completely different external network config, can handle ingress & egress traffic for each namespace individually. \n BIG-IP Next SPK is highly scalable at POD level (1-24 cores) and at cluster level, limited by the upstream ECMP capabilities. \n \n A more detailed view of the network path is shown next. From this diagram we want to emphasise: \n \n PODs make use of BIG-IP Next SPK transparently by continue using the CNI as usual. \n BIG-IP Next SPK is a single tier ingress/egress solution not requiring external LB. \n BIG-IP Next SPK has direct POD IP visibility, there is no kube-proxy or other IP-translating mechanism in between. \n \n \n As depicted above, BIG-IP Next SPK has two types of interfaces: external facing the upstream routers and internal facing the Kubernetes networking. Openshift's networking facilitates that using BIG-IP Next SPK in a cluster is optional in a per-namespace basis and this is done transparently to the applications. No change or configuration needs to be done in the applications. Openshift uses for its networking the OVNKubernetes CNI. It can be seen in the picture that the applications continue using the OVNKubernetes router as default and only gateway. \n Finally we will show a L2 view of the networks in a cluster with BIG-IP Next SPK. From this diagram we want to emphasise: \n \n How regular nodes hosting applications have no modifications either. \n How BIG-IP Next SPK is typically setup with link aggregations and SR-IOV wire-speed interfaces. \n How the L3 path between BIG-IP Next SPK and the application's nodes is validated by means of using BFD. \n \n \n Using BIG-IP Next SPK \n In order to use BIG-IP Next SPK no changes need to be done in the applications or in the namespace hosting the applications. At BIG-IP Next SPK configuration time we will instruct which namespace we want BIG-IP Next SPK to handle and voilà: BIG-IP Next SPK becomes the next-hop of OVNKubernetes router for that namespace. No labels or other artifacts need to be configured manually. \n Defining BIG-IP Next SPK services configurations is done thorugh the Kubernetes API using Custom Resource Definitions (CRDs). At time of this writting the following resources are avaiilable: \n \n \n \n \n F5SPKIngressTCP Manages ingress layer 4 TCP application traffic. \n F5SPKIngressUDP Manages ingress layer 4 UDP application traffic. \n F5SPKIngressDiameter Manages Diameter traffic unifying ingress and egress traffic using either TCP or SCTP and keeps sessions persistent using the SESSION-ID attribute value pair (AVP) by default. \n F5SPKIngressNGAP Balances ingress datagram loads for SCTP or NG application protocol (NGAP) signaling. \n F5SPKEgress Enables egress traffic for pods using SNAT or DNS/NAT46. DNS cache and rate limiting parameters can be configured. \n F5SPKSnatpool Allocates IP addresses for egress pod connections. \n F5SPKDNSCache Provides high-performance, transparent DNS resolution and caching for the F5SPKEgress resources. \n F5SPKPortListandF5SPKAddressList Creates sets of ports and addresses, respectively, to make creating and updating services easier. \n \n Besides the above resource list we would like to highlight 3 functionalities that should not be overlooked: \n \n IPv6 support\n \n \n \n BIG-IP Next SPK fully supports IPv4/IPv6 dual-stack networking as implemented in Kubernetes v1.21 or later. BIG-IP Next SPK’s DNS46/NAT46 feature, however, does not rely on Kubernetes IPv4/IPv6 dual-stack and therefore, it can be used with earlier versions of Kubernetes. \n \n \n \n \n DNS46/NAT46 translation \n \n \n The adoption of IPv6 in new 5G deployments has created a need to interact with older IPv4 single stack components and services. BIG-IP Next SPK’s DNS46/NAT46 provides this interoperability, easing the adoption and transition between IPv4 and IPv6 services. This solution allows IPv4 applications to access any IPv6 application on demand, without requiring reconfiguration. \n \n \n \n Application hairpinning The application hairpinning feature is used to differentiate between internal and external clients. A selected set of internal clients accesses an BIG-IP Next SPK Service with the same domain name or IP address as that of another BIG-IP Next SPK Service, which is used by external clients using different configurations. The key difference between the two types of connections is that internal clients are connected using SNAT and external clients are not. This is done by installing two BIG-IP Next SPK CRs of the same type, for example F5SPKIngressTCP, with each CR enabled on a selected VLAN or VLAN list. \n \n ´BIG-IP Next SPKs roadmap \n In the upcoming releases BIG-IP Next SPK will continue expanding its traffic management capabilities by exposing more TMM capabilities through the Kubernetes API, noticiably HTTP/2. \n Also, BIG-IP Next SPK will be gaining more security oriented features. At present it is being targeted the following features: \n \n Firewall \n DDoS protection \n WAF \n \n These security features are specially useful because BIG-IP Next SPK constitutes a security boundary with respect of all the workloads in the cluster and kubernetes itself (CNI, API, basic node management). Although BIG-IP Next SPK is running inside the Kubernetes cluster, it is the only software that manages the external network interfaces at L3. This is depicted in the next figure. \n \n \n These security features have been available for long time in BIG-IP products and at present we are capturing customers´ input to design the best APIs possible for exposing these functionalities following Kubernetes patterns. \n Conclusion \n \n \n \n This article introduces a scalable and dependable high performance gateway solution that delivers the granular ingress and egress controls in Kubernetes-based deployments that Telcos need. It builds on the unique potential of OpenShift external gateways by making full use of OpenShift capabilities—an industry first. Use cases that particularly benefit include 5GC and MEC. Plus, the BIG-IP Next SPK solution can dynamically translate IPv4 to IPv6 network addresses, which solves the problem of mixed IPv4 and IPv6 deployments. The result is a gateway solution flexible enough to adapt to new and evolving Telco needs while offering interoperability with pre-5G services. For additional information please check the RedHat & F5 co-written white paper F5 Telco Gateway for Red Hat OpenShift and the official BIG-IP Next SPK documentation. \n \n \n \n \n \n \n \n ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"10027","kudosSumWeight":5,"repliesCount":1,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNzdpMDI3MDVFRkJEN0I1QTYxOA?revision=12\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNjVpNEI4MzY2QTI1OEFGQTBDRg?revision=12\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNjhpODdDQTI2MTg2MkNDQTY4Qg?revision=12\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNjZpN0FFQThCMkE5Q0E3QTNCRg?revision=12\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNzVpQUYzNTM1RTlGQjgyQTA2NA?revision=12\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyNzZpRUUyODNDMTMzMDc1NDEwMQ?revision=12\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDA1NDctMTkyOTRpREUxNUNENUI4RjM3NjJDNw?revision=12\"}"}}],"totalCount":7,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:334737":{"__typename":"Conversation","id":"conversation:334737","topic":{"__typename":"TkbTopicMessage","uid":334737},"lastPostingActivityTime":"2024-09-25T11:44:09.060-07:00","solved":false},"User:user:49695":{"__typename":"User","uid":49695,"login":"John_Gruber","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS00OTY5NS1LOUc1TGg?image-coordinates=11%2C0%2C703%2C691"},"id":"user:49695"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctalZXd1pO?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctalZXd1pO?revision=2","title":"image.png","associationType":"BODY","width":440,"height":313,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctUXpYdnJ4?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctUXpYdnJ4?revision=2","title":"image.png","associationType":"BODY","width":180,"height":316,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctTXE0dnA5?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctTXE0dnA5?revision=2","title":"image.png","associationType":"BODY","width":304,"height":306,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctS3pyR1Nn?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctS3pyR1Nn?revision=2","title":"image.png","associationType":"BODY","width":491,"height":438,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctbkRJTnpr?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctbkRJTnpr?revision=2","title":"image.png","associationType":"BODY","width":487,"height":337,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctSWhMM1Rj?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctSWhMM1Rj?revision=2","title":"image.png","associationType":"BODY","width":738,"height":414,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctUVpESFdL?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctUVpESFdL?revision=2","title":"image.png","associationType":"BODY","width":290,"height":559,"altText":""},"TkbTopicMessage:message:334737":{"__typename":"TkbTopicMessage","subject":"How to Prepare Your Network Infrastructure to Add HPC Clusters for AI to Your Data Center","conversation":{"__ref":"Conversation:conversation:334737"},"id":"message:334737","revisionNum":2,"uid":334737,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:49695"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":" HPC AI clusters are getting deployed as highly-engineered 'lego blocks' which are opaque to established data center operations and standards. By taking advantage of established Kubernetes based networking solutions that provide high-speed intelligent networking, you can save yourself from expensive cost overruns, data center re-auditing, and delays. By using Kubernetes based solutions which take advantage of the high-speed networking solutions already required by HP AI deployments, you further optimize your investment in AI. ","introduction":"","metrics":{"__typename":"MessageMetrics","views":249},"postTime":"2024-09-25T11:44:09.060-07:00","lastPublishTime":"2024-09-25T11:44:09.060-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" High Performance Computing (HPC) AI cluster infrastructures are increasingly finding their way into enterprise data centers. There are things that should be considered to avoid rearchitecting data center operations, monitoring, and security to accommodate these complex clusters. \n Introducing HPC clusters into data centers increases both the potential and danger of shifting the entire infrastructure ecosystem. When integrating AI data, tools, and policies into existing infrastructure, careful consideration must be given to maintaining operational standards for monitoring and reliability. The Kubernetes network infrastructure may also require additional scrutiny. This will help avoid outages caused by scalability limitations or security vulnerabilities. \n Many decision-makers are rushing to capture GPU-powered compute clusters to deliver AI model training and inferencing capabilities, so they don’t get left behind during this hype cycle. However, these accelerated GPU hardware and requisite new integrations can inadvertently cause unplanned cascading re-architecture and require incremental auditing of their infrastructure operations and personnel. This can ultimately add unplanned and long-term costs, impacting both CapEx and OpEx. \n Network Segmentation risk when placing HPC AI clusters into multi-tenant data centers \n The costs to operationalize HPC AI clusters require evaluation beyond power and cooling requirements. Adding the new complex high-performance infrastructure elements within HPC clusters into established data centers requires normalizing their services to the methods and practices needed to support the enterprise overall. \n One primary area which needs to be normalized is how network segmentation is required within the larger data center. Network segmentation forms the basis of data center scale of monitoring, service levels, and security. Here, global hyperscalers have invested significant resources within their infrastructures—both in products and services as well as staffing resources—to ensure basic assurance of service tenancy expected for all modern systems. \n But rushing to place HPC Kubernetes-orchestrated AI clusters within multi-tenant data centers creates problems without providing a solution to provide network-level tenancy compatible with existing systems. If compatibility is not maintained, a complete infrastructure design and audit to ensure AI service performance and security would be required. This opens the doors to incremental delays, added cost, new vulnerabilities, and potential service disruptions. Know what to look for and plan ahead. \n Let's break this down into three topics in the data center: \n \n Required Kubernetes ingress \n Data center challenges introduced by Kubernetes multi-tenancy \n Optimizing GPU performance with DPU/IPU offloading \n \n We will end with a common case study in high-performance Kubernetes ingress load balancing to demonstrate immediate value for AI training and inferencing. \n (1) Kubernetes network ingress adds new challenges to the data center \n One area of standardization already embedded within HPC AI cluster operations is the use of Kubernetes to orchestrate infrastructure services. Kubernetes networking was purposely designed to allow containerized processes to communicate within the cluster directly. To accomplish this, a flat network structure is maintained across each cluster. This means whenever processes want to provide a service to applications outside their cluster (or present a service to other segregated applications within their own cluster), another Kubernetes service resource is required to facilitate that ingress communication. \n Figure 1. diagram showing Kubernetes ingress into a data center \n\n Note: Kubernetes network design requires an infrastructure component to provide network traffic ingress for clustered applications—ingress is not a default function for network traffic. This management and integration of incoming service traffic to the Kubernetes cluster is determined by the infrastructure provider. So, depending on your configuration and service partners, some components within the data center network must be added to perform this ingress role for each cluster. \n Kubernetes Ingress Requires Orchestration and New Security Policy Control \n So, it follows that Kubernetes ingress requires an element within EACH cluster’s control plane to allocate and configure (or orchestrate) the necessary infrastructure resources. (Context: some organizations currently manage up to 50 production clusters, per the CNCF.org 2023 annual user survey dataset). This provides an ingress path to the in-cluster containerized endpoints. To service this need, there is an entire market of third-party products to provide Ingress, called Kubernetes Ingress Controllers. Adding even more complexity, each Kubernetes Ingress Controller vendor brings its own set of service delivery, scaling and data center networking integrations, which requires support by the infrastructure NetOps (network operations) teams. As ingress represents a strategic control point of policy for all the service orchestrated within the Kubernetes clusters, security best practice calls for the SecOps (security operations) team to secure the ingress orchestration by inserting firewall and monitoring capabilities leveraging the scope of the rest of the data center infrastructure. Kubernetes Ingress Controller solutions are operationalized through the DevOp (developer operations) teams declaration of Kubernetes service requirements as part of their CI/CD pipelines. The standard Kubernetes service declarations for ingress are defined by the following standard Kubernetes resources: \n \n \n \n \n Kubernetes API \n Data Center Networking Services \n \n \n LoadBalancer \n Port-based service delivery (L4 TCP) using network address translation to provide access to service endpoint ports within the cluster \n \n \n Ingress \n A proxy-based service providing HTTP service delivery, which includes HTTP routing, TLS termination, and virtual hosting capabilities for service endpoints within the cluster \n \n \n Gateway \n A proxy-based, newly released standard, which provides a fully range of extensible application delivery services including role-based configurations for advanced routing of TCP, UDP, HTTP, HTTP/2 (gRPC) service traffic for service endpoints within the cluster \n \n \n \n \n F5’s expertise is in network load balancing through our F5 BIG-IP product suite. BIG-IP has evolved its networking infrastructure, software and hardware (appliances, chassis, virtual machines) to provide intelligent application delivery and security services. These proven network functions are now available in cloud-native containerized form factors and are currently in production managing traffic at the scale of hundreds of Gbps (gigabits per second) through direct integration with data center networking fabrics. Adding to the options of ingress vendors, BIG-IP offers two Kubernetes Ingress Controllers—BIG-IP Container Ingress is for hardware and VM deployments, and BIG-IP Next Service Proxy for Kubernetes is for cloud-native deployments. Both offerings provide secured ingress for Kubernetes clustered applications. \n Note: as Kubernetes networking ingress is a requirement for any Kubernetes cluster, some server OEMs who provide pre-bundled bill of materials (BOMs) for Kubernetes clusters often include F5 BIG-IP appliances and chassis-based products to fill the ingress implementer role. BIG-IP is a well-understood and accepted component in enterprise data centers for both NetOps and SecOps teams. BIG-IP forms the basis to normalize Kubernetes cluster ingress, networking and security to the larger data center. \n (2) Single-tenancy, and multi-tenancy in the data center: F5’s Kubernetes “Ball of Fire” \n Another set of hidden challenges becomes apparent when cluster operational complexity grows. Due to the way Kubernetes tenancy has typically been deployed in most data center environments, these complex challenges have thus far been masked. \n In Kubernetes, the nodes which run containerized applications provide inter-host routing for the cluster by having each node maintain a NodeIP, which is the IP address of the server treated like a node-- a network address which can be routed within the data center’s underlying network fabric. (NodeIP is different from each cluster’s internal ClusterIP, which are addresses facilitating the flat direct routing for services within a cluster.) In inter-host routing, when traffic needs to egress from a given node to another node, or to access resources outside the cluster, the traffic is sourced from the infrastructure routable NodeIP for whichever cluster node is hosting a particular application container instance. At first glance, this seems to be a good, distributed network design, but the issue is it removes a key control point required in most data center operations. \n \n Imagine designating network monitoring and security in the larger data center for Kubernetes-hosted applications. If a whole Kubernetes cluster can be allocated through bare-metal deployment of virtual machines for each segregated security tenant of the data center, we have no problem--all the NodeIP addresses for that dedicated cluster belong to one specific cluster owner, thus one data center tenant. \n \n In this simple model where each cluster is assigned to one tenant, NetOps teams understand how to allocate those addresses to that cluster owner. Firewall security and monitoring can identify the data center tenant simply by the network segmentation required to route their traffic within the infrastructure. SecOps teams can build monitoring and security based on this simple network allocation scheme. When each cluster represents one tenant, we maintain existing data center operations. In this configuration, using lots of clusters, while proliferating Kubernetes everywhere, keeps the resources distributed, scalable, and secure. \n However, what happens when the resources inside the Kubernetes cluster need to support multi-tenancy from a data center tenancy perspective? In this case, which is the security case for HPC AI clusters, multiple applications with different data center tenants are hosted within the same cluster. This means egress traffic for multiple data center tenants can now be sourced from the same NodeIP whenever traffic leaves any given host node. This fact masks the needed network details from generations of high-performance monitoring and security tools required in the data center infrastructure. It creates what F5 calls the “Ball of Fire”. \n \n \n How Telecoms Successfully Deliver Multi-Tenant Kubernetes Clusters in Data Centers \n Deploying multi-tenant Kubernetes clusters was solved by network service providers (telecoms) around the world—out of necessity—in order to adopt the standards for 5G services and applications. Critical network functions shifted form factors from VMs to cloud-native network functions (CNFs) such as CGNAT, DDoS, Firewall, Policy Manager, and aggregated from numerous vendors and sources. Each CNF from multiple vendors therefore adds another unique challenge to segregate and secure in the broader, external to the Kubernetes cluster, network context. \n While F5 BIG-IP has years of operational experience integrating at scale within their network fabrics for both their IT and telco clouds, there was no Kubernetes standard to handle egress tenancy. \n An F5 customer and early-adopter Tier 1 service provider requested changes to BIG-IP in order to build out a widescale 5G infrastructure on an aggressive time schedule, with new requirements: \n \n Distributed in a containerized form factor that itself could be managed and controlled by Kubernetes \n Additionally, support network and application protocols, application delivery features, and security functionality, which existing Kubernetes networking architecture does not address \n \n In short, they needed a Kubernetes networking infrastructure service, which would normalize their Kubernetes cluster deployments to their data center infrastructure, while at the same time maintaining the ‘swiss army knife’ scale and functionality, for both ingress and egress. This functionality set is already provided by legacy F5 BIG-IP appliances and chassis already in deployment and needed transference to the modern form factor. \n This containerization development of F5 BIG-IP functions generated a new iteration of services labeled F5 BIG-IP Next. A Kubernetes resources-based control plane was needed, along with a deeper infrastructure integration with the internal of the Kubernetes clusters themselves. This complex set of requirements did not exist in the industry prior to this customer request, so F5 developed BIG-IP Next Service Proxy for Kubernetes (SPK) to specifically fit this functional gap in Kubernetes. \n \n SPK uniquely provides a distributed implementation of BIG-IP, controlled as a Kubernetes resource, which understands both Kubernetes namespace-based tenancy and the network segregation tenancy required by the data center networking fabric. BIG-IP Next SPK lives both inside the Kubernetes clusters as well as inside the data center network fabric. SPK provides MAC (L2 networking) all the way to application (L7 networking) level control for all traffic ingress or egressing Kubernetes clusters. SPK functions not just as the required Kubernetes Ingress Controller, but also, through declared custom resource definitions (CRD) as a policy and security engine to normalize multi-tenant clustered application to the wider data center and global network at telecom speeds and scale. SPK was the key component for a global telecom to achieve multi-tenant scale and manage complexity in a Kubernetes framework to deliver 5G. \n Multiple teams can take advantage of this advanced functionality: \n \n DevOps teams can continue to use standard Kubernetes resource declarations to deploy application from their tested CI/CD pipelines \n NetOps teams in the data center can dictate that all traffic from a namespace within the cluster must egress from specific VLANs, VxLAN, interface VRFs, or IPv4 or IPv6 subnets. NetOps teams continue to define the required service levels based on this network segregation \n SecOps teams use the inherent security and monitoring found in BIG-IP in conjunction with their other security controls to ensure secure application delivery. \n \n This preserves data center operations. It did this without fundamentally breaking the Kubernetes networking model and forcing containers to live on underlying data center networks. It got rid of the need for new security implementations and the subsequent re-auditing process adopting them would require. This value cannot be overstated when it comes to deploying complex Kubernetes clusters into established data centers quickly. \n BIG-IP Next SPK is now in production for tens of millions of mobile subscribers’ traffic every day across global networks. With the scale and speed of such massive network deployments, managing outages in a critical piece as Kubernetes ingress or egress cannot be accomplished as a side project for the infrastructure team or as a feature add-on for a firewall vendor. Reliability, scaling, and load balancing must be in the core DNA of the network stack. And with these newly scaled capabilities for Kubernetes in distributed computing environments, it’s how SPK is ready to deliver for AI workloads. \n \n \n (3) Optimize GPU with DPU/IPU Offloads for HPC AI Kubernetes network ingress and egress services \n Our third infrastructure pain point is maximizing multiple GPU compute performance and scale as HPC AI clusters are introduced into IT data centers originally designed for typical web service and client server compute workloads. By design, to accommodate the super-HPC scale, these new HPC AI clusters have inter-service (east-west) networking requirements, which can reach the equivalent bandwidths needed to deliver mobile traffic for whole geographic continents. The networking bandwidths within HPC AI clusters are staggering. \n These networking requirements were introduced to facilitate the use of: \n \n Remote Direct Memory Access (RDMA) \n Nonvolatile memory express (NVMe) over Fabrics \n \n Protocols as a data busses between nodes (east-west). These protocols utilize very high-bandwidth, non-blocking network architectures that allow one computer to directly access data from another across the network without expensive OS stacks or CPU cycles being used to slowly keep track of things. This significantly lowers latency and ensures the fastest response times to data for AI workloads and allows clusters of GPUs to copy data between themselves using extensions to their own chip-to-chip data technologies. The network fabric is functioning as the new backplane for the whole HPC AI cluster. \n This HPC supercomputing cluster is opaque operationally as a ‘lego block’ within the larger data center. Not surprisingly, the technical requirements in HPC AI cluster design are very tight and non-negotiable when tied to specific hardware decisions. Extending RAM, storage, and proprietary chip-to-chip technology across the network is not a simple task and must be highly engineered. This is not news to the HPC community but is new for most enterprises or network service operator teams. While the protocols used certainly aren’t new, how they are implemented by specific hardware in HPC AI clusters is alarming in its growing scope. If HPC AI cluster proliferation is the new normal, then the opaque nature of their networking will be driving significant cost and operational challenges in the near future. \n Programmable SuperNIC data processing unit/interface processing unit (DPU/IPU) are replacing the HPC AI cluster node NICs to facilitate connectivity within these highly engineered network fabrics. These new DPUs don’t just include the necessary network switching technologies to connect to the 200Gbps/400Gbps ports on the non-blocking network switches, but also include hardware accelerators for nVME, connection, compression, encryption, and other offloads. But like their NIC predecessors, DPU/IPUs are still compatible with x86 and Arm hosts, which opens a new range of flexible functionality. \n Kubernetes host networking stacks are quickly being optimized to take advantage of the DPU/IPU accelerators. The de facto Open vSwitch (OVS) Linux networking stack has implementations of connection offloading for multiple DPU/IPU vendor accelerators, allowing for high-speed networking flows between Kubernetes ClusterIPs for east-west traffic. \n Implementing ingress and egress services for clusters is using 20–30% of the HPC AI node compute \n We’re observing that ingress and node-level service-to-service networking takes a significant amount of cluster compute resources when performed by software-based networking stacks running on each—or across a set of—cluster nodes. To optimize performance, networking software pins itself to specific processing cores and pre-allocates memory to process network flows. These resources appear totally consumed and unavailable to the HPC AI cluster host. It is not an understatement that between 20–30% of cluster host compute could be expended by network software simply getting traffic in and out of the cluster nodes. \n The compute footprint in the HPC AI data center should ideally be instead focused on AI application services—which requires the parallel stream and tensor core processing driving the deployment of expensive GPUs in the first place. For every CPU host cycle that is expended providing infrastructure services, like ingress/egress networking, we starve AI workloads that keep the expensive GPU resources busy. That’s when the TCO calculations, which justify the GPU hardware expenditure and new cluster expensive non-blocking networking components tip even more towards the red in terms of efficiency, cost, and ROI. (Somewhere a CFO just pulled some hair from their head.) \n Kubernetes ingress and egress services, and their security, are prime targets for DPU/IPU network accelerator offloads. The DPU/IPUs are being placed inside the HPC AI cluster for their own reasons, namely RDMA and NVMe offloads. However, the same offloads can be utilized for ingress/egress network processing, thus optimizing compute for the efficient utilization of GPUs. \n \n Customer Use Case: High-Performance and Scalable Kubernetes Load Balancing for S3 HPC AI Cluster Storage Access \n Even before the proliferation of HPC AI clusters hit the data center world, there was already a fundamental AI use case which demonstrates the value of accelerated and intelligent BIG-IP application-level delivery. AI model training, or retraining, requires data, lots of data. Moving data into HPC AI cluster storage is largely handled through the use of object storage APIs. Data is replicated from various tiers of object storage sources and copied into clustered file technology, which can provide high-speed access to data for GPU stream processing. \n The most widely deployed object storage API is S3 (Simple Storage Service), a cloud object storage API pioneered by AWS. S3 uses HTTP REST API methods where HTTP objects represent file buckets (folders) and files. The S3 services translate HTTP requests to storage requests, which maintain efficient reading and writing of data across devices, as well as security permissions. There are numerous implementations of S3-compatible APIs available, either as containerized HTTP microservices, which front attached storage devices. Or as hosted HTTP endpoints in storage vendor’s appliance arrays. \n \n F5 BIG-IP hardware-accelerated appliances and chassis already load balance many S3 deployments, allowing for intelligent routing of storage object requests. This is typically done through the publishing of multiple service endpoints, where each is represented by a separate hostname. Resiliency and scale is handled by L4 accelerated connection load balancing. This is the simplest and highest-scale solution, but not the only one available in BIG-IP. Alternatively, S3 HTTP requests can be processed with BIG-IP evaluating the HTTP Host header, path, and query parameters. All this intelligence can be used as ways to load-balance S3 traffic to specific endpoints. TLS offload is also an obvious choice because of hardware acceleration. \n There is another point of value for the AI S3 use case. The S3 client libraries are built to support high concurrency through threading. The load-balancing solution must therefore also be able to handle very high levels of connection concurrency efficiently. This is all part of understanding the task of load balancing S3, and both BIG-IP for L4 connections or L7 HTTP request load balancing of S3 traffic support the highest scale in the industry. S3 load balancing is a task BIG-IP was purpose-built to perform. \n \n Distributed Application Delivery for HPC AI Clusters is Available Today \n The ability to hardware accelerate HPC, AI cluster ingress and egress network services on deployed DPU/IPUs is available in BIG-IP Next SPK today. The DPU/IPU accelerated solution is not a new limited version of F5’s data plane, but rather the full BIG-IP stack. That means access to a wide range of functions for simplified AI service deployments with BIG-IP—for both reverse proxy ingress and forward proxy egress—is available as a key functional component of your HPC AI cluster deployment. These application delivery and security functions are automatically inline and efficient as they are part of the same network stack that is providing the required ingress functionality. \n An additional benefit of locating offload capability for the ingress and egress networking so close to the HPC AI cluster network hardware itself is that external services, running on more traditional and less costly computing services, and can also be injected into the AI application path without complex service chaining orchestrations. This provides an obvious point of network integration for data observability features needed for privacy and compliance, AI API gateway features, and new security points. Because BIG-IP Next SPK can map the HPC AI cluster namespace tenancy to data center network tenancy, these external products from F5 and others can be placed inline without requiring deep integration into the HPC AI clusters themselves. Policies can be based on the network segmentation provided by F5 for the cluster, not restricted to the specifics of a given GPU-generation of HPC AI cluster. \n Using hardware offload capabilities for networking, application delivery, and security can be complicated and requires significant levels of testing to ensure scale and support. F5 remains committed to a vision of a more open infrastructure for offload services through their work in the Open Programmable Infrastructure (OPI) initiative, which F5 helped found as part of the Linux Foundation in 2022. OPI’s goal remains the open-source democratization of APIs and programmable SmartNICs for acceleration to promote wider adoption of hardware acceleration for the broader software community. \n The reality, however, is that differentiated hardware offloads with proprietary APIs will continue to forge the cutting edge of the performance computing market. No one understands this better than the HPC community. Integrating a dedicated ingress and egress architecture early, which is proven at scale and is headed by a vendor that is constantly engaged in this market sets a direction which can steer your HPC AI cluster deployments away from both data center and financial obstacles to avoid slowing down your AI application rollouts and adoption. \n To talk to an F5 representative, Contact Us and put in the text box note you’d like to discuss AI HPC clusters. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"26043","kudosSumWeight":4,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctalZXd1pO?revision=2\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctUXpYdnJ4?revision=2\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctTXE0dnA5?revision=2\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctS3pyR1Nn?revision=2\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctbkRJTnpr?revision=2\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctSWhMM1Rj?revision=2\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ3MzctUVpESFdL?revision=2\"}"}}],"totalCount":7,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:333871":{"__typename":"Conversation","id":"conversation:333871","topic":{"__typename":"TkbTopicMessage","uid":333871},"lastPostingActivityTime":"2024-09-20T05:00:00.042-07:00","solved":false},"TkbTopicMessage:message:333871":{"__typename":"TkbTopicMessage","subject":"Announcing F5 NGINX Gateway Fabric 1.4.0 with IPv6 and TLS Passthrough","conversation":{"__ref":"Conversation:conversation:333871"},"id":"message:333871","revisionNum":3,"uid":333871,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:430188"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":131},"postTime":"2024-09-20T05:00:00.042-07:00","lastPublishTime":"2024-09-20T05:00:00.042-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" We announced the next release of F5 NGINX Gateway Fabric version 1.4.0 which includes a lot of smaller but very necessary features. This allows us to dedicate more time to advancing our non-functional testing framework and ensuring we maintain top performance across releases. Nevertheless, we have some great highlights of this release: \n \n IPv6 support \n \n \n TLS passthrough (via TLSRoute) \n \n \n Server zone metrics \n \n \n Ability to add custom pod annotations \n \n \n Plenty of bug fixes! \n \n During this release cycle, we discovered a bug around our custom policies that occurred when you had the same path for more than one Route: The policy would not be applied to either Route. For this release, we’ve decided to enforce a restriction so that policies cannot be applied when two or more routes share the same path. However, we are pursuing a long-term solution to lift this restriction on this edge case, as we understand that use cases that route based on header, query parameter, or other request attributes on the same path do exist. \n \n IPv6 Support \n While most Kubernetes clusters are still utilizing IPv4, we recognized that anyone employing a IPv6 cluster would have no ability to deploy NGINX Gateway Fabric. Thus, we implemented a simple feature to dual IPv4/IPv6 networking for NGINX Gateway Fabric. This option is enabled by default, so you can simply install as normal on an IPv6 cluster. \n \n TLS Passthrough \n New with 1.4 is TLSRoute support. This Route type enables the TLS Passthrough use case and is similar to setting up an HTTPRoute. This allows you to pass encrypted traffic through NGINX Gateway Fabric where it is terminated by your backend application, ensuring end-to-end encryption. As most information passes through NGINX Gateway Fabric with this route, setup is easy. You can enable TLS passthrough for any application using our guide available here. \n \n Non-Functional Testing \n This release marks the completion of automating our non-functional testing that we execute before each release. If you are unfamiliar with these tests, our team runs NGINX Gateway Fabric through a series of scenarios, non-functional tests, to test if our performance is regressing or improving from previous releases. As an infrastructure product that you rely on, it is our top priority to ensure that stability and performance are not compromised as new features are released. \n The results of all non-functional testing are available in the GitHub repository for anyone to see and should give you an idea of how well NGINX Gateway Fabric performs in general and across releases. \n \n What’s Next \n NGINX Gateway Fabric 1.5.0 will bring NGINX code snippets to the Gateway API with a first-class Upstream Settings policy to configure keepalive connections and NGINX zone size. If you are familiar with NGINX or find that you need to use a feature that NGINX provides that is not yet available via a Gateway API extension, you can put a NGINX code snippet within a SnippetFilter to apply NGINX configuration to a Route rule. You will even be able to use the feature to load other modules NGINX provides and leverage the vast wealth of NGINX functionality. \n We will still be providing many NGINX features via first-class policies and filters, such as the Upstream Settings policy, as they allow us to handle much of the complexity of translating to Gateway API for you. These custom policies and filters allow us to handle a lot of the complexity of applying NGINX config across the Gateway API framework for you. \n The Upstream Settings policy can set upstream management directives that are unable to be applied via snippets effectively. We will continue to deliver these custom policies and filters across all of our releases, in addition to new Gateway API resources and NGINX Gateway Fabric specific features. \n You can see a preview of the full snippet design here, though not all features may be implemented in one release cycle. For more information on our strategy towards first-class NGINX customization via Gateway API extensions, see our full enhancement proposal here. \n Resources \n For the complete changelog for NGINX Gateway Fabric 1.4.0, see the Release Notes. To try NGINX Gateway Fabric for Kubernetes with NGINX Plus, start your free 30-day trial today or contact us to discuss your use cases. \n If you would like to get involved, see what is coming next, or see the source code for NGINX Gateway Fabric, check out our repository on GitHub! \n We have weekly community meetings on Tuesdays at 9:30AM Pacific/12:30PM Eastern/5:30PM GMT. The meeting link, updates, agenda, and notes are on the NGINX Gateway Fabric Meeting Calendar. Links are also always available from our GitHub readme. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"4891","kudosSumWeight":1,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:332495":{"__typename":"Conversation","id":"conversation:332495","topic":{"__typename":"TkbTopicMessage","uid":332495},"lastPostingActivityTime":"2024-08-08T05:00:00.035-07:00","solved":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzI0OTUtSjV1MUFk?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzI0OTUtSjV1MUFk?revision=2","title":"nginx-functionality-groups.png","associationType":"BODY","width":2043,"height":1585,"altText":""},"TkbTopicMessage:message:332495":{"__typename":"TkbTopicMessage","subject":"Announcing F5 NGINX Gateway Fabric 1.3.0 with Tracing, GRPCRoute, and Client Settings","conversation":{"__ref":"Conversation:conversation:332495"},"id":"message:332495","revisionNum":2,"uid":332495,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:430188"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":" The release of NGINX Gateway Fabric version 1.3.0, introduces plenty of highly requested features and improvements. GRPCRoutes are now supported to manage gRPC traffic, similar to the handling of HTTPRoute. The update includes new custom policies like ClientSettingsPolicy for client request configurations and ObservabilityPolicy for enabling application tracing with OpenTelemetry support. The GRPCRoute allows for efficient routing, header modifications, traffic weighting, and error conversion from HTTP to gRPC. \n We will explain how to set up NGINX Gateway Fabric to manage gRPC traffic using a Gateway and a GRPCRoute, providing a detailed example of the setup. It also outlines how to enable tracing through the NginxProxy resource and ObservabilityPolicy, emphasizing a selective approach to tracing to avoid data overload. \n Additionally, the ClientSettingsPolicy allows for the customization of NGINX directives at the Gateway or Route level, giving users control over certain NGINX behaviors with the possibility of overriding Gateway defaults at the Route level. \n Looking ahead, the NGINX Gateway Fabric team plans to work on TLS Passthrough, IPv6, and improvements to the testing suite, while preparing for larger updates like NGINX directive customization and separation of data and control planes. Check the end of the article to see how to get involved in the development process through GitHub and participate in bi-weekly community meetings. Further resources and links are also provided within. ","introduction":"","metrics":{"__typename":"MessageMetrics","views":255},"postTime":"2024-08-08T05:00:00.035-07:00","lastPublishTime":"2024-08-08T05:00:00.035-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Today we are announcing the next release of F5 NGINX Gateway Fabric version 1.3.0, which adds our next route type, GRPCRoute, to the product. As the name suggests, this enables gRPC traffic ingestion and management within NGINX Gateway Fabric the same way you would with an HTTPRoute. gRPC is a powerful protocol to use when you need additional performance or streaming over traditional HTTP REST APIs most often used across the web. \n This is the first release where we have extended the Gateway API with custom policies that directly expose advanced NGINX functionality and configuration otherwise not available in Gateway API resources such as Routes or Gateways. With our new ClientSettingsPolicy and ObservabilityPolicy you can enable and configure NGINX features using the framework the Gateway API provides by attaching them to Routes or Gateways. Through the ClientSettingsPolicy, you can set configuration around client request requirements and keepalive connections. With the ObservabilityPolicy and some configuration in NginxProxy, you can now enable tracing for your applications, although attachment is currently only possible on the Route for this release. \n \n GRPCRoute \n With the release of NGINX Gateway Fabric 1.3.0 comes core support for GRPCRoute. For those applications currently taking advantage of, or looking at the gRPC framework, you can now ingress traffic to your gRPC application with a GRPCRoute. \n Many are looking at using gRPC over traditional HTTP REST based APIs to take advantage of its data efficiency and latency benefits. Utilizing a binary data format for transmission, gRPC calls offer a lower latency over HTTP REST. It was for this reason that we wanted to ensure it was included in NGINX Gateway Fabric. \n To enable this, we utilize our data plane, NGINX, to provide http/2 transport. This provides optimal proxy performance and minimal overhead for gRPC traffic through NGINX Gateway Fabric. With core support for GRPCRoute, you can: \n \n Match traffic to be routed based on hostname, headers, or method \n \n \n Modify request or response headers \n \n \n Split traffic by weight \n \n \n Automatically convert http error responses from your application to gRPC \n \n For full and up to date compatibility details, see our compatibility page under gRPC here. \n Enabling gRPC traffic management through NGINX Gateway Fabric is straightforward via a GRPCRoute. For example, if we want to set up NGINX Gateway Fabric to forward gRPC traffic to a single application, we will just need a Gateway and a GRPCRoute. \n Gateway: \n apiVersion: gateway.networking.k8s.io/v1beta1 \nkind: Gateway \nmetadata: \n name: my-gateway \nspec: \n gatewayClassName: nginx \n listeners: \n - name: listener-1 \n port: 80 \n protocol: HTTP \n allowedRoutes: \n namespaces: \n from: Same \n kinds: \n - kind: gRPCroute \n hostname: bar.com \n \n GRPCRoute: \n apiVersion: gateway.networking.k8s.io/v1alpha2 \nkind: GRPCRoute \nmetadata: \n name: grpc-route-1 \nspec: \n parentRefs: \n - name: my-gateway \n sectionName: listener-1 \n rules: \n - matches: \n backendRefs: \n - name: gRPC-application-service \n port: 80 \n \n Once these two resources have been created, any gRPC traffic sent to “my-gateway” on port 80 for bar.com will be forwarded to the service “gRPC-application-service\". For other features and api reference, check out the Gateway API documentation available here. \n \n OpenTelemetry Tracing Support \n To enable tracing, we have not just bolted on the ability to add the relevant configuration to NGINX via some config, but built the feature into extensions of the Gateway API. By utilizing global configuration in NginxProxy and attaching our new ObservabilityPolicy to a route, you can configure NGINX Gateway Fabric to generate and forward traces to your tracing backend. \n These extensions align with our strategy to extend the Gateway API to create a cohesive configuration experience based on the roles defined in the Gateway API. As we’ve seen with Ingress, many custom configurations not defined in a standard way can lead to a lot of confusion we want to avoid. \n As a “cluster operator,” you would create the “NginxProxy” resource to attach to the GatewayClass. This defines where the traces should be sent to and what span name to include in the traces collected. An example of an NginxProxy resource is below: \n \n NGINXProxy: \n apiVersion: gateway.nginx.org/v1alpha1 \nkind: NginxProxy \nmetadata: \n name: NGINX Gateway Fabric-proxy-config \nspec: \n telemetry: \n exporter: \n endpoint: jaeger.tracing.svc:4317 \n spanAttributes: \n - key: cluster \n value: my-cluster \n \n While the above resource defines where traces go, no traces will be collected until an ObservabilityPolicy is created. Usually, this will be created by an application developer and attached to one of their own routes. See below for an example of an ObservabilityPolicy with a pre-existing route, “coffee”: \n \n ObservabilityPolicy: \n apiVersion: gateway.nginx.org/v1alpha1 \nkind: ObservabilityPolicy\nmetadata: \n name: coffee-tracing \nspec: \n targetRefs: \n - group: gateway.networking.k8s.io \n kind: HTTPRoute \n name: coffee \n tracing: \n strategy: ratio \n ratio: 50 \n spanAttributes: \n - key: route \n value: coffee \n \n In the above example, we can also choose to omit the ratio for a default 100% capture rate. As always, when implementing in your cluster, make sure to check the status of any resources you create! \n We elected to skip a setting to enable tracing globally because as your cluster scales, the amount of tracing data it generates will become very large. A global option will likely generate far more data than you can use, so we recommend enabling tracing only for those endpoints that need it. But as always, let us know on GitHub if you have a strong use case for this feature! \n For the full documentation on how to configure tracing for your environment, check here. \n \n Client Settings Policy \n As the name implies, NGINX Gateway Fabric uses NGINX OSS or NGINX Plus as its data plane. This means we can leverage all the features NGINX provides as we develop NGINX Gateway Fabric, but we also want to expose that configuration to our users. One aspect of our strategy towards exposing NGINX functionality is to promote frequently used settings into extensions of the Gateway API. The ClientSettingsPolicy is our first step towards doing that. \n The ClientSettingsPolicy currently allows you to configure 5 NGINX directives at either the Gateway or Route: \n \n client_max_body_size \n \n \n client_body_timeout \n \n \n keepalive_requests \n \n \n keepalive_time \n \n \n keepalive_timeout \n \n As the names of the directives imply, you can use these to change default nginx behavior. Below is an example of a ClientSettingsPolicy attached to a Gateway: \n apiVersion: gateway.nginx.org/v1alpha1 \nkind: ClientSettingsPolicy \nmetadata: \n name: gateway-client-settings \n namespace: default \nspec: \n targetRef: \n group: gateway.networking.k8s.io \n kind: Gateway \n name: my-gateway \n body: \n maxSize: 5m \n timeout: 30s \n keepAlive: \n requests: 3000 \n time: 5s \n timeout: \n server: 2s \n header: 1s \n \n If you were to attach this policy at both the Gateway and Route scopes, any fields specified in the Route will be overwritten over the Gateway defined values. That way, the cluster operator can set appropriate defaults, but can be overridden when needed for specific applications. A short guide on inherited policy attachment can be found here. \n \n For the full documentation on how to configure every aspect of the ClientSettingsPolicy, check it out here! \n What’s Next \n This next release cycle, we will be working on some smaller features such as TLS Passthrough and IPv6 while we improve our testing suite and follow up on smaller features and bugs. At the same time, we will be laying the groundwork for our bigger features for our next release, including NGINX directive customization, and separating our data and control planes. \n As many experienced users of NGINX know, there are a lot of features NGINX Gateway Fabric can tap into just by providing an interface to NGINX directives directly. Thus, instead of trying to expose absolutely everything via the Gateway API, we are going to provide an interface more akin to NGINX Ingress' snippets; a method to insert your own directives with your own values into specific places in the configuration NGINX Gateway Fabric sends to NGINX. \n While we will also provide a first-class method of configuration via Gateway API extensions, this feature will ensure our users can leverage existing directives now rather than when they are implemented as their own extension. Over time, we will be implementing the map below of NGINX functionality into our own extensions. \n \n For more information on our strategy towards NGINX customization, see our full enhancement proposal here. \n Resources \n For the complete changelog for NGINX Gateway Fabric 1.3.0, see the Release Notes. To try NGINX Gateway Fabric for Kubernetes with NGINX Plus, start your free 30-day trial today or contact us to discuss your use cases. \n If you would like to get involved, see what is coming next, or see the source code for NGINX Gateway Fabric, check out our repository on GitHub! \n We have bi-weekly community meetings on Mondays at 9AM Pacific/5PM GMT. The meeting link, updates, agenda, and notes are on the NGINX Gateway Fabric Meeting Calendar. Links are also always available from our GitHub readme. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"9929","kudosSumWeight":0,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzI0OTUtSjV1MUFk?revision=2\"}"}}],"totalCount":1,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:330280":{"__typename":"Conversation","id":"conversation:330280","topic":{"__typename":"TkbTopicMessage","uid":330280},"lastPostingActivityTime":"2024-06-18T14:53:53.142-07:00","solved":false},"User:user:427941":{"__typename":"User","uid":427941,"login":"Rawdata","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS00Mjc5NDEtSlY5T3hr?image-coordinates=1099%2C0%2C5099%2C4000"},"id":"user:427941"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtRHRFcklr?revision=6\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtRHRFcklr?revision=6","title":"Screenshot 2024-06-18 at 2.28.21 PM.png","associationType":"BODY","width":1360,"height":950,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtYkd2Qlgx?revision=6\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtYkd2Qlgx?revision=6","title":"Screenshot 2024-06-03 at 1.35.09 PM.png","associationType":"BODY","width":513,"height":430,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtWEhGTGlw?revision=6\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtWEhGTGlw?revision=6","title":"Screenshot 2024-06-04 at 11.10.29 AM.png","associationType":"BODY","width":524,"height":499,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtdXNRZVho?revision=6\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtdXNRZVho?revision=6","title":"Screenshot 2024-06-04 at 11.12.06 AM.png","associationType":"BODY","width":524,"height":144,"altText":""},"TkbTopicMessage:message:330280":{"__typename":"TkbTopicMessage","subject":"Securing and Scaling Hybrid Apps with F5/NGINX (Part 3)","conversation":{"__ref":"Conversation:conversation:330280"},"id":"message:330280","revisionNum":6,"uid":330280,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:427941"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":303},"postTime":"2024-06-18T09:27:05.996-07:00","lastPublishTime":"2024-06-18T14:53:53.142-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" \n In part 2 of our series, I demonstrated how to configure ZT (Zero Trust) use cases centering around authentication with NGINX Plus in hybrid environments. We deployed NGINX Plus as the external LB to route and authenticate users connecting to my Kubernetes applications. \n \n \n In this article, we explore other areas of the ZT spectrum configurable on the External LB Service, including: \n \n Authorization and Access \n Encryption mTLS \n Monitoring/Auditing \n \n ZT Use case #1: Authorization \n Many people think that authentication and authorization can be used interchangeably. However, they both mean different things. Authentication involves the process of verifying user identities based on the credentials presented. \n Even though authenticated users are verified by the system, they do not necessarily have the authority to access protected applications. That is where authorization comes into play. Authorization involves the process of verifying the authority of an identity before granting access to application. \n Authorization in the context of OIDC authentication involves retrieving claims from user ID tokens and setting conditions to validate whether the user is authorized to enter the system. \n An authenticated user is granted an ID token from the IdP with specific user information through JWT claims. The configuration of these claims is typically set from the IdP. Revisiting the OIDC auth use case configured in the previous section, we can retrieve the ID tokens of authenticated users from the NGINX key-value store. \n $ curl -i http://localhost:8010/api/9/http/keyvals/oidc_acess_tokens \n Then we can view the decoded value of the ID token using jwt.io. Below is an example of decoded payload data from the ID token. \n {\n\"exp\": 1716219261, \n\"iat\": 1716219201, \n\"admin\": true, \n\"name\": \"Micash\", \n\"zone_info\": \"America/Los_Angeles\" \n\"jti\": \"9f8ff4bd-4857-4e12-9634-e5876f786f98\", \n\"iss\": \"http://idp.f5lab.com:8080/auth/realms/master\", \n\"aud\": \"account\", \"typ\": \"Bearer\",\n\"azp\": \"appworld2024\", \n\"nonce\": \"gMNK3tu06j6tp5-jGa3aRhkj4F0P-Z3e04UfcFeqbes\"\n} \n NGINX Plus has access to these claims as embedded variables. They are accessed by prefixing $jwt_claim_ to the desired field (for example, $jwt_claim_admin for the admin claim). We can easily set conditions on these claims and block unauthorized users before they even reach the back-end applications. \n Going back to our frontend.conf file in the previous part of our series. We can set $jwt_flag variable to 0 or 1 based on the value of the admin JWT claim. We then use the jwt_claim_require directive to validate the ID token. ID tokens with admin claims set to false will be rejected. \n map $jwt_claim_admin $jwt_status { \n \"true\" 1;\n default 0; \n} \nserver { \n include conf.d/openid_connect.server_conf; # Authorization code flow and Relying Party processing \n error_log /var/log/nginx/error.log debug; # Reduce severity level as required \n listen [::]:443 ssl ipv6only=on;\n listen 443 ssl; \n server_name example.work.gd; \n ssl_certificate /etc/ssl/nginx/default.crt; # self-signed for example only\n ssl_certificate_key /etc/ssl/nginx/default.key;\n\n location / { \n # This site is protected with OpenID Connect\n auth_jwt \"\" token=$session_jwt;\n error_page 401 = @do_oidc_flow;\n auth_jwt_key_request /_jwks_uri; # Enable when using URL\n auth_jwt_require $jwt_status;\n proxy_pass https://cluster1-https; # The backend site/app\n }\n} \n Note: Authorization with NGINX Plus is not restricted to only JWT tokens. You can technically set conditions on a variety of attributes, such as: \n \n Session cookies \n HTTP headers \n Source/Destination IP addresses \n \n ZT use case #2: Mutual TLS Authentication (mTLS) \n When it comes to ZT, mTLS is one of the mainstream use cases falling under the Zero Trust umbrella. For example, enterprises are using Service Mesh technologies to stay compliant with ZT standards. This is because Service Mesh technologies aim to secure service to service communication using mTLS. \n In many ways, mTLS is similar to the OIDC use case we implemented in the previous section. Only here, we are leveraging digital certificates to encrypt and authenticate traffic. This underlying framework is defined by PKI (Public Key Infrastructure). \n To explain this framework in simple terms we can refer to a simple example; the driver's license you carry in your wallet. Your driver’s license can be used to validate your identity, the same way digital certificates can be used to validate the identity of applications. Similarly, only the state can issue valid driver's licenses, the same way only Certificate Authorities (CAs) can issue valid certificates to applications. It is also important that only the state can issue valid certificates. Therefore, every CA must have a private secure key to sign and issue valid certificates. \n Configuring mTLS with NGINX can be broken down in two parts: \n \n Ingress mTLS; Securing SSL client traffic and validating client certificates against a trusted CA. \n Egress mTLS; securing SSL upstream traffic and offloading authentication of TLS material to a trusted HTTPS back-end server. \n \n Ingress mTLS \n You can configure ingress mTLS on the NLK deployment by simply referencing the trusted certificate authority adding the ssl_client_certificate directive in the server context. This will configure NGINX to validate client certificates with the referenced CA. \n Note: If you do not have a CA, you can create one using OpenSSL or Cloudflare PKI and TLS toolkits \n server {\n listen 443 ssl;\n status_zone https://cafe.example.com;\n server_name cafe.example.com;\n ssl_certificate /etc/ssl/nginx/default.crt;\n ssl_certificate_key /etc/ssl/nginx/default.key;\n ssl_client_certificate /etc/ssl/ca.crt;\n} \n Egress mTLS \n Egress mTLS is a slight alternative to ingress mTLS where NGINX verifies certificates of upstream applications rather than certificates originating from clients. This feature can be enabled by adding the proxy_ssl_trusted_certificate directive to the server context. You can reference the same trusted CA we used for verification when configuring ingress mTLS or reference a different CA. \n In addition to verifying server certificates, NGINX as a reverse-proxy can pass over certs/keys and offload verification to HTTPS upstream applications. This can be done by adding the proxy_ssl_certificate and proxy_ssl_certificate_key directives in the server context. \n server {\n listen 443 ssl;\n status_zone https://cafe.example.com;\n server_name cafe.example.com;\n ssl_certificate /etc/ssl/nginx/default.crt;\n ssl_certificate_key /etc/ssl/nginx/default.key;\n\n #Ingress mTLS\n ssl_client_certificate /etc/ssl/ca.crt;\n\n #Egress mTLS\n proxy_ssl_certificate /etc/nginx/secrets/default-egress.crt;\n proxy_ssl_certificate_key /etc/nginx/secrets/default-egress.key;\n proxy_ssl_trusted_certificate /etc/nginx/secrets/default-egress-ca.crt;\n} \n ZT use case #3: Secure Assertion Markup Language (SAML) \n SAML (Security Assertion Markup Language) is an alternative SSO solution to OIDC. Many organizations may choose between SAML and OIDC depending on requirements and IdPs they currently run in production. SAML requires a SP (Service Provider) to exchange XML messages via HTTP POST binding to a SAML IdP. Once exchanges between the SP and IdP are successful, the user will have session access to the protected backed applications with one set of user credentials. \n In this section, we will configure NGINX Plus as the SP and enable SAML with the IdP. This will be like how we configured NGINX Plus as the relying party in an OIDC authorization code flow (See ZT Use case #1). \n Setting up the IdP \n The one prerequisite is setting up your IdP. In our example, we will set up the Microsoft Entra ID on Azure. You can use the SAML IdP of your choosing. Once the SAML application is created in your IdP, you can access the SSO fields necessary to link your SP (NGINX Plus) to your IdP (Microsoft Entra ID). \n \n \n You will need to edit the basic SAML configuration by clicking on the pencil icon next to Edit in Basic SAML Configuration, as seen in the figure above. \n Add the following values and click Save: \n \n Identifier (Entity ID) -- https://fourth.run.place \n Reply URL (Assertion Consumer Service URL) -- https://fourth.run.place/saml/acs \n Sign on URL: https://fourth.run.place \n Logout URL (Optional): https://fourth.run.place/saml/sls \n \n Finally download the Certificate (Raw) from Microsoft Entra ID and save it to your NGINX Plus instance. This certificate is used to verify signed SAML assertions received from the IdP. Once the certificate is saved on the NGINX Plus instance, extract the public key from the downloaded certificate and convert it to SPKI format. We will use this certificate later when we configure NGINX Plus in the next section. \n $ openssl x509 -in demo-nginx.der -outform DER -out demo-nginx.der\n$ openssl x509 -inform DER -in demo-nginx.der -pubkey -noout > demo-nginx.spki \n Configuring NGINX Plus as the SAML Service Provider \n After the IdP is setup, we can configure NGINX Plus as the SP to exchange and validate XML messages with the IdP. Once logged into the NGINX Plus instance, simply clone the nginx SAML GitHub repo. \n $ git clone https://github.com/nginxinc/nginx-saml.git && cd nginx-saml \n Copy the config files into the /etc/nginx/conf.d directory. \n $ cp frontend.conf saml_sp.js saml_sp.server_conf saml_sp_configuration.conf /etc/nginx/conf.d/ \n Notice that by default, frontend.conf listens on port 8010 with clear text http. You can merge kube_lb.conf into frontend.conf to enable TLS termination and update the upstream context with application endpoints you wish to protect with SAML. \n Finally we will need to edit the saml_sp_configuration.conf file and update variables in the map context based on the parameters of your SP and IdP: \n \n $saml_sp_entity_id; https://fourth.run.place \n $saml_sp_acs_url; https://fourth.run.place/saml/acs \n $saml_sp_sign_authn; false \n $saml_sp_want_signed_response; false \n $saml_sp_want_signed_assertion; true \n $saml_sp_want_encrypted_assertion; false \n $saml_idp_entity_id; Unique identifier that identifies the IdP to the SP. This field is retrieved from your IdP \n $saml_idp_sso_url; This is the login URL and is also retrieved from the IdP \n $saml_idp_verification_certificate; Variable referencing the certificate downloaded from the previous section when setting up the IdP. This certificate will verify signed assertions received from the IdP. Use the full directory (/etc/nginx/conf.d/demo-nginx.spki) \n $saml_sp_slo_url; https://fourth.run.place/saml/sls \n $saml_idp_slo_url; This is the logout URL retrieved from the IdP \n $saml_sp_want_signed_slo; true \n \n The remaining variables defined in saml_sp_configuration.conf can be left unchanged, unless there is a specific requirement for enabling them. Once the variables are set appropriately, we can reload NGINX Plus. \n $ nginx -s reload \n Testing \n Now we will verify the SAML flow. open your browser and enter https://fourth.run.place in the address bar. This should redirect me to the IDP login page. \n \n \n Once you login with your credentials, I should be granted access to my protected application \n \n ZT use case #4: Monitoring/Auditing \n NGINX logs/metrics can be exported to a variety of 3rd party providers including: Splunk, Prometheus/Grafana, cloud providers (AWS CloudWatch and Azure Monitor Logs), Datadog, ELK stack, and more. \n You can monitor NGINX metrics and logs natively with NGINX Instance Manager or NGINX SaaS. The NGINX Plus API provides me a lot of flexibility by exporting metrics to any third-party tool that accepts JSON. For example, you can export NGINX Plus API metrics to our native real-time dashboard from part 1. \n native real-time dashboard from part 1 \n Whichever tool I chose, monitoring/auditing my data generated from my IT systems is key to understanding and optimizing my applications. \n Conclusion \n Cloud providers offer a convenient way to expose Kubernetes Services to the internet. Simply create Kubernetes Service of type: LoadBalancer and external users connect to your services via public entry point. However, cloud load balancers do nothing more than basic TCP/HTTP load balancing. You can configure NGINX Plus with many Zero Trust capabilities as you scale out your environment to multiple clusters in different regions, which is what we will cover in the next part of our series. \n ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"12826","kudosSumWeight":2,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtRHRFcklr?revision=6\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtYkd2Qlgx?revision=6\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtWEhGTGlw?revision=6\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzAyODAtdXNRZVho?revision=6\"}"}}],"totalCount":4,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:329878":{"__typename":"Conversation","id":"conversation:329878","topic":{"__typename":"TkbTopicMessage","uid":329878},"lastPostingActivityTime":"2024-05-23T05:00:00.028-07:00","solved":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk4NzgtWXhDc0hN?revision=3\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk4NzgtWXhDc0hN?revision=3","title":"clipboard_image-1-1715978696421.png","associationType":"BODY","width":1402,"height":1364,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk4NzgtTmhZdkFF?revision=3\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk4NzgtTmhZdkFF?revision=3","title":"clipboard_image-2-1715978696404.png","associationType":"BODY","width":1415,"height":1356,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk4NzgtQUJibm9i?revision=3\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk4NzgtQUJibm9i?revision=3","title":"clipboard_image-4-1715978696448.png","associationType":"BODY","width":1403,"height":1380,"altText":""},"TkbTopicMessage:message:329878":{"__typename":"TkbTopicMessage","subject":"Use topology labels to reduce cross-AZ ingress traffic with F5 CIS and EKS","conversation":{"__ref":"Conversation:conversation:329878"},"id":"message:329878","revisionNum":3,"uid":329878,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:242856"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":" This solution outlines how to use topology labels and multiple CIS instances to reduce cross-AZ traffic on AWS EKS cluster ingress. ","introduction":"","metrics":{"__typename":"MessageMetrics","views":524},"postTime":"2024-05-23T05:00:00.028-07:00","lastPublishTime":"2024-05-23T05:00:00.028-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" This solution outlines how to use topology labels and multiple Container Ingress Services (CIS) instances to reduce cross-AZ traffic on AWS EKS cluster ingress. \n Requirements and background \n Recently I had a customer with the following environment: \n \n The need to load-balance traffic from Internet-based clients into EKS \n The need for mTLS termination and other functionality that required BIG-IP \n AWS EKS cluster deployed in 3x Availability Zones (AZ’s) \n \n Simple enough, right? But there’s some other problems that add an interesting twist to the requirements: \n \n Reduce or eliminate cross-AZ traffic between any external load balancer and EKS (due to cost of cross-AZ traffic at the planned high throughput) \n Do not use NLB if possible (NLB throughput cost alone is significant) \n While mTLS/other functionality could be performed inside the cluster (eg NGINX Ingress Controller), we want these functions performed external to the cluster, for other reasons. \n \n Let’s solve this! \n F5 CIS with a typical default installation \n Typically, a customer will use F5 CIS to dynamically update the configuration of an HA pair of BIG-IP’s, sending traffic directly to pods running inside Kubernetes (K8s). \n This typical HA-pair deployment would be easy, but it would not meet all requirements. \n\n Cross-AZ traffic with a typical CIS deployment \n The above diagram is a valid deployment, but cross-AZ traffic is very likely. The BIG-IP’s are unaware of K8s topology, and will load-balance equally across AZ’s. \n \n Since only 1x BIG-IP is active in the pair, ingressing to pods in 2 out of 3 AZ’s requires cross-AZ traffic \n Ingress to pods in AZ 3 will always generate cross-AZ traffic, regardless of which BIG-IP is active \n \n Multiple active BIG-IP’s and node-label selector with CIS \n CIS can use the node-label-selector argument to limit load-balancing to select nodes. We will use this to keep ingress traffic local to an AZ. \n To deploy an architecture like the following diagram: \n \n Find or create your topology labels. In my example using EKS, I see my nodes have labels such as topology.kubernetes.io/zone=us-east-1a\n \n You can also create your own labels on nodes for this purpose \n \n \n Deploy 3x standalone BIG-IP’s \n Deploy 3x CIS instances\n \n Use the node-label-selector argument so that each CIS instance only watches for pods on select nodes \n \n \n \n Notice that this design uses 3x standalone BIG-IPs and limits ingress traffic within an AZ. \n\n Here is an example of a CIS deployment with a node-label-selector (line 32): \n apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: f5cis1\n namespace: kube-system\nspec:\n replicas: 1\n selector:\n matchLabels:\n app: k8s-bigip-ctlr-deployment\n template:\n metadata:\n labels:\n app: k8s-bigip-ctlr-deployment\n spec:\n containers:\n - name: k8s-bigip-ctlr\n image: \"f5networks/k8s-bigip-ctlr:2.16.1\"\n env:\n - name: BIGIP_USERNAME\n valueFrom:\n secretKeyRef:\n name: bigip-login\n key: username\n - name: BIGIP_PASSWORD\n valueFrom:\n secretKeyRef:\n name: bigip-login\n key: password\n command: [\"/app/bin/k8s-bigip-ctlr\"]\n args: [\n \"--node-label-selector=topology.kubernetes.io/zone=us-east-1a\",\n \"--bigip-username=$(BIGIP_USERNAME)\",\n \"--bigip-password=$(BIGIP_PASSWORD)\",\n \"--bigip-url=10.0.0.11\",\n \"--bigip-partition=kubernetes\",\n \"--pool-member-type=cluster\",\n \"--insecure\",\n \"--custom-resource-mode=true\",\n \"--log-level=DEBUG\",\n \"--disable-teems=true\"\n ]\n serviceAccount: bigip-ctlr\n serviceAccountName: bigip-ctlr\n imagePullSecrets:\n - name: bigip-login \n \n Other pointers in this solution \n \n NLB’s are not required. To avoid NLB throughput costs, use DNS Load Balancing (GSLB) to spread traffic evenly across your Internet-facing BIG-IP’s. Other methods to disaggregate traffic exist also but are not the focus of this solution. \n K8s labels can be arbitrary. I have used existing topology labels in this case, but you could use your own node labels for the same outcome. \n \n NodePort vs Cluster mode \n It is worth noting that the diagrams above have assumed the CIS deployment is in ClusterIP mode, and not NodePort mode. If you were sending traffic to K8s nodes and relying on kube-proxy to distribute traffic evenly across pods, you would almost certainly generate cross-AZ traffic between nodes. \n \n Further reading about topology and routing in K8s \n Topology Aware Routing is a K8s concept that uses labels and annotations to allow an administrator to define preferences for keeping traffic within regions or zones where possible. This solution does not take full advantage of this concept, but merely uses these labels to limit which pods CIS will configure on BIG-IP. Today, CIS and BIG-IP are not fully aware of the Kubernetes topology in which they operate. \n It is worth noting the AWS EKS Best Practices Guide, specifically as it discusses AWS Load balancers. When you use AWS ALB or NLB, you can choose between instance mode (somewhat similar to the NodePort example above) and ip mode (similar to the ClusterIP example above). If you’re planning to do something similar to what I’ve done above using AWS native load balancers, make sure you understand the difference and configure accordingly. \n Finally, the above solution does not apply only to AWS EKS. The concept of regions and zones exists in most clouds, and the concept of topologies in Kubernetes were created to account for these. However, it’s the cross-AZ traffic charges in AWS that make this solution pressing for users of AWS EKS. \n Conclusion \n Consider if your external load balancer is causing cross-AZ traffic charges for ingress traffic. At very high throughput levels, the cost of cross-AZ traffic can become significant. If using F5 CIS to populate pods as pool members in BIG-IP, consider using the node-label-selector argument and multiple active BIG-IP’s to keep ingress traffic to pods within a single Availability Zone. \n Thanks for reading! ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"6339","kudosSumWeight":0,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk4NzgtWXhDc0hN?revision=3\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk4NzgtTmhZdkFF?revision=3\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk4NzgtQUJibm9i?revision=3\"}"}}],"totalCount":3,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:328875":{"__typename":"Conversation","id":"conversation:328875","topic":{"__typename":"TkbTopicMessage","uid":328875},"lastPostingActivityTime":"2024-04-16T13:51:45.180-07:00","solved":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjg4NzUtdEpuWllx?revision=6\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjg4NzUtdEpuWllx?revision=6","title":"NKL W_ NGINX Plus Ingress Controller.png","associationType":"BODY","width":1425,"height":800,"altText":"null"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjg4NzUtNjlneXRX?revision=6\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjg4NzUtNjlneXRX?revision=6","title":"Screenshot 2024-04-16 at 8.17.48 AM.png","associationType":"BODY","width":852,"height":467,"altText":""},"TkbTopicMessage:message:328875":{"__typename":"TkbTopicMessage","subject":"Securing and Scaling Hybrid Application with F5 NGINX (Part 1)","conversation":{"__ref":"Conversation:conversation:328875"},"id":"message:328875","revisionNum":6,"uid":328875,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:427941"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":723},"postTime":"2024-04-08T05:00:00.032-07:00","lastPublishTime":"2024-04-16T13:51:45.180-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" If you are using Kubernetes in production, then you are likely using an ingress controller. The ingress controller is the core engine managing traffic entering and exiting the Kubernetes cluster. Because the ingress controller is a deployment running inside the cluster, how do you route traffic to the ingress controller? How do you route external traffic to internal Kubernetes Services? \n Cloud providers offer a simple convenient way to expose Kubernetes Services using an external load balancer. Simply deploy a Managed Kubernetes Service (EKS, GKE, AKS) and create a Kubernetes Service of type LoadBalancer. The cloud providers will host and deploy a load balancer providing a public IP address. External users can connect to Kubernetes Services using this public entry point. \n However, this integration only applies to Managed Kubernetes Services hosted by cloud providers. If you are deploying Kubernetes in private cloud/on-prem environments, you will need to deploy your own load balancer and integrate it with the Kubernetes cluster. Furthermore, Kubernetes Load Balancing integrations in the cloud are limited to TCP Load Balancing and generally lack visibility into metrics, logs, and traces. \n We propose: \n \n A solution that applies regardless of the underlying infrastructure running your workloads \n Guidance around sizing to avoid bottlenecks from high traffic volumes \n Application delivery use cases that go beyond basic TCP/HTTP load balancing \n \n In the solution depicted below, I deploy NGINX Plus as the external LB service for Kubernetes and route traffic to the NGINX Ingress Controller. The NGINX Ingress Controller will then route the traffic to the application backends. The NLK (NGINX Load Balancer for Kubernetes) deployment is a new controller by NGINX that monitors specified Kubernetes Services and sends API calls to manage upstream endpoints of the NGINX External Load Balancer \n \n In this article, I will deploy the components both inside the Kubernetes cluster and NGINX Plus as the external load balancer. \n \n Note: I like to deploy both the NLK and Kubernetes cluster in the same subnet to avoid network issues. This is not a hard requirement. \n \n Prerequisites \n The blog assumes you have experience operating in Kubernetes environments. In addition, you have the following: \n \n Access to a Kubernetes environment; Bare Metal, Rancher Kubernetes Engine (RKE), VMWare Tanzu Kubernetes (VTK), Amazon Elastic Kubernetes (EKS), Google Kubernetes Engine (GKE), Microsoft Azure Kubernetes Service (AKS), and RedHat OpenShift \n NGINX Ingress Controller – Deploy NGINX Ingress Controller in the Kubernetes cluster. Installation instructions can be found in the documentation. \n NGINX Plus – Deploy NGINX Plus on VM or bare metal with SSH access. This will be the external LB service for the Kubernetes cluster. Installation instructions can be found in the documentation. You must have a valid license for NGINX Plus. You can get started today by requesting a 30-day free trial. \n \n Setting up the Kubernetes environment \n I start with deploying the back-end applications. You can deploy your own applications, or you can deploy our basic café application as an example. \n $ kubectl apply –f cafe.yaml \n Now I will configure routes and TLS settings for the ingress controller \n $ kubectl apply –f cafe-secret.yaml $ kubectl apply –f cafe-virtualserver.yaml \n To ensure the ingress rules are successfully applied, you can examine the output of kubectl get vs. \n The VirtualServer definition should be in the Valid state. \n NAMESPACE NAME STATE HOST IP PORTS \ndefault cafe-vs Valid cafe.example.com \n \n Setting up NGINX Plus as the external LB \n A fresh install of NGINX Plus will provide the default.conf file in the /etc/nginx/conf.d directory. We will add two additional files into this directory. Simply copy the bulleted files into your /etc/nginx/conf.d directory \n \n dashboard.conf; This will enable the real-time monitoring dashboard for NGINX Plus \n kube_lb.conf; The nginx configuration as the external load balancer for Kubernetes. You can change the configuration file to fit your requirements. In part 1 of this series, we enabled basic routing and TLS for one cluster. \n \n You will also need to generate TLS cert/keys and place them in the /etc/ssl/nginx folder of the NGINX Plus instance. For the sake of this example, we will generate a self-signed certificate with openssl. \n $ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout default.key -out default.crt -subj \"/CN=NLK\" \n \n \n Note: Using self-signed certificates is for testing purposes only. In a live production environment, we recommend using a secure vault that will hold your secrets and trusted CAs (Certificate Authorities). \n \n Now I can validate the configuration and reload nginx for the changes to take effect. \n $ nginx –t\n$ nginx –s reload \n I can now connect to the NGINX Plus dashboard by opening a browser and entering \n http://<external-ip-nginx>:9000/dashboard.html#upstreams \n \n \n The HTTP upstream table should be empty as we have not deployed the NLK Controller yet. \n We will do that in the next section. \n Installing the NLK Controller \n You can install the NLK Controller as a Kubernetes deployment that will configure upstream endpoints for the external load balancer using the NGINX Plus API. First, we will create the NLK namespace \n $ kubectl create ns nlk \n And apply the RBAC settings for the NKL deployment \n $ kubectl apply -f serviceaccount.yaml $ kubectl apply -f clusterrole.yaml $ kubectl apply -f clusterrolebinding.yaml $ kubectl apply -f secret.yaml \n The next step is to create a ConfigMap defining the API endpoint of the NGINX Plus external load balancer. The API endpoint is used by the NLK Controller to configure the NGINX Plus upstream endpoints. We simply modify the nginx-hosts field in the manifest from our GitHub repository to the IP address of the NGINX external load balancer. \n nginx-hosts: \n\n http://<nginx-plus-external-ip>:9000/api \n Apply the updated ConfigMap and deploy the NLK controller \n $ kubectl apply –f nkl-configmap.yaml $ kubectl apply –f nkl-deployment \n I can verify the NLK controller deployment is running and the ConfigMap data is applied. \n $ kubectl get pods –o wide –n nlk \t\n$ kubectl describe cm nginx-config –n nlk \n \t\t \n \n \n You should see the NLK deployment in status Running and the URL should be defined under nginx-hosts. The URL is the NGINX Plus API endpoint of the external load Balancer. Now that the NKL Controller is successfully deployed, the external load balancer is ready to route traffic to the cluster. The final step is deploying a Kubernetes Service type NodePort to expose the Kubernetes cluster to NGINX Plus. \n $ kubectl apply –f nodeport.yaml \n There are a couple things to note about the NodePort Service manifest. Fields on line 7 and 14 are required for the NLK deployment to configure the external load balancer appropriately: \n \n The nginxinc.io/nkl-cluster annotation \n The port name matching the upstream block definition in the NGINX Plus configuration (See line 42 in kube_lb.conf) and preceding nkl- \n \n \t\t \n apiVersion: v1 \nkind: Service \nmetadata: \n name: nginx-ingress \n namespace: nginx-ingress \n annotations: \n nginxinc.io/nlk-cluster1-https: \"http\" # Must be added \nspec: \n type: NodePort \n ports: \n - port: 443 \n targetPort: 443 \n protocol: TCP \n name: nlk-cluster1-https \n selector: \n app: nginx-ingress \n Once the service is applied, you can note down the assigned nodeport selecting the NGINX Ingress Controller deployment. In this example, that node port is 32222. \n $ kubectl get svc –o wide –n nginx-ingress\n\t\t\nNAME TYPE CLUSTER-IP PORT(S) SELECTOR\n\nnginx-ingress NodePort x.x.x.x 443:32222/TCP app=nginx-ingress \n \n If I reconnect to my NGINX Pus dashboard, the upstream tab should be populated with the worker node IPs of the Kubernetes cluster and matching the node port of the nginx-ingress Service (32222). \n \n You can list the node IPs of your cluster to make sure they match the IPs in the dashboard upstream tab. \n $ kubectl get nodes -o wide | awk '{print $6}' \n\nINTERNAL-IP \n\n10.224.0.6 \n10.224.0.5 \n10.224.0.4 \n Now I can connect to the Kubernetes application from our local machine. The hostname we used in our example (cafe.example.com) should resolve to the IP address of the NGINX Plus load balancer. \n \n Wrapping it up \n Most enterprises deploying Kubernetes in production will install an ingress controller. It is the DeFacto standard for application delivery in container orchestrators like Kubernetes. DevOps/NetOps engineers are now looking for guidance on how to scale out their Kubernetes footprint in the most efficient way possible. Because enterprises are embracing the hybrid approach, they will need to implement their own integrations outside of cloud providers. The solution we propose: \n \n is applicable to hybrid environments (particularly on-prem) \n Sizing information to avoid bottlenecks from large traffic volumes \n Enterprise Load Balancing capabilities that stretch beyond a TCP LoadBalancing Service \n \n In the next part of our series, I will dive into the third bullet point into much more detail and cover Zero Trust use cases with NGINX Plus, providing extra later of security in your hybrid model. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"9565","kudosSumWeight":0,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjg4NzUtdEpuWllx?revision=6\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjg4NzUtNjlneXRX?revision=6\"}"}}],"totalCount":2,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"CachedAsset:text:en_US-components/community/Navbar-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/community/Navbar-1743097587452","value":{"community":"Community Home","inbox":"Inbox","manageContent":"Manage Content","tos":"Terms of Service","forgotPassword":"Forgot Password","themeEditor":"Theme Editor","edit":"Edit Navigation Bar","skipContent":"Skip to content","migrated-link-9":"Groups","migrated-link-7":"Technical Articles","migrated-link-8":"DevCentral News","migrated-link-1":"Technical Forum","migrated-link-10":"Community Groups","migrated-link-2":"Water Cooler","migrated-link-11":"F5 Groups","Common-external-link":"How Do I...?","migrated-link-0":"Forums","article-series":"Article Series","migrated-link-5":"Community Articles","migrated-link-6":"Articles","security-insights":"Security Insights","migrated-link-3":"CrowdSRC","migrated-link-4":"CodeShare","migrated-link-12":"Events","migrated-link-13":"Suggestions"},"localOverride":false},"CachedAsset:text:en_US-components/community/NavbarHamburgerDropdown-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/community/NavbarHamburgerDropdown-1743097587452","value":{"hamburgerLabel":"Side Menu"},"localOverride":false},"CachedAsset:text:en_US-components/community/BrandLogo-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/community/BrandLogo-1743097587452","value":{"logoAlt":"Khoros","themeLogoAlt":"Brand Logo"},"localOverride":false},"CachedAsset:text:en_US-components/community/NavbarTextLinks-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/community/NavbarTextLinks-1743097587452","value":{"more":"More"},"localOverride":false},"CachedAsset:text:en_US-components/authentication/AuthenticationLink-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/authentication/AuthenticationLink-1743097587452","value":{"title.login":"Sign In","title.registration":"Register","title.forgotPassword":"Forgot Password","title.multiAuthLogin":"Sign In"},"localOverride":false},"CachedAsset:text:en_US-components/nodes/NodeLink-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/nodes/NodeLink-1743097587452","value":{"place":"Place {name}"},"localOverride":false},"CachedAsset:text:en_US-components/tags/TagSubscriptionAction-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/tags/TagSubscriptionAction-1743097587452","value":{"success.follow.title":"Following Tag","success.unfollow.title":"Unfollowed Tag","success.follow.message.followAcrossCommunity":"You will be notified when this tag is used anywhere across the community","success.unfollowtag.message":"You will no longer be notified when this tag is used anywhere in this place","success.unfollowtagAcrossCommunity.message":"You will no longer be notified when this tag is used anywhere across the community","unexpected.error.title":"Error - Action Failed","unexpected.error.message":"An unidentified problem occurred during the action you took. Please try again later.","buttonTitle":"{isSubscribed, select, true {Unfollow} false {Follow} other{}}","unfollow":"Unfollow"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageListTabs-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageListTabs-1743097587452","value":{"mostKudoed":"{value, select, IDEA {Most Votes} other {Most Likes}}","mostReplies":"Most Replies","mostViewed":"Most Viewed","newest":"{value, select, IDEA {Newest Ideas} OCCASION {Newest Events} other {Newest Topics}}","newestOccasions":"Newest Events","mostRecent":"Most Recent","noReplies":"No Replies Yet","noSolutions":"No Solutions Yet","solutions":"Solutions","mostRecentUserContent":"Most Recent","trending":"Trending","draft":"Drafts","spam":"Spam","abuse":"Abuse","moderation":"Moderation","tags":"Tags","PAST":"Past","UPCOMING":"Upcoming","sortBymostRecent":"Sort By Most Recent","sortBymostRecentUserContent":"Sort By Most Recent","sortBymostKudoed":"Sort By Most Likes","sortBymostReplies":"Sort By Most Replies","sortBymostViewed":"Sort By Most Viewed","sortBynewest":"Sort By Newest Topics","sortBynewestOccasions":"Sort By Newest Events","otherTabs":" Messages list in the {tab} for {conversationStyle}","guides":"Guides","archives":"Archives"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/QueryHandler-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/QueryHandler-1743097587452","value":{"title":"Query Handler"},"localOverride":false},"CachedAsset:text:en_US-components/community/NavbarDropdownToggle-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/community/NavbarDropdownToggle-1743097587452","value":{"ariaLabelClosed":"Press the down arrow to open the menu"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/OverflowNav-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/OverflowNav-1743097587452","value":{"toggleText":"More"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageView/MessageViewInline-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageView/MessageViewInline-1743097587452","value":{"bylineAuthor":"{bylineAuthor}","bylineBoard":"{bylineBoard}","anonymous":"Anonymous","place":"Place {bylineBoard}","gotoParent":"Go to parent {name}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/Pager/PagerLoadMore-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/Pager/PagerLoadMore-1743097587452","value":{"loadMore":"Show More"},"localOverride":false},"CachedAsset:text:en_US-components/customComponent/CustomComponent-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/customComponent/CustomComponent-1743097587452","value":{"errorMessage":"Error rendering component id: {customComponentId}","bannerTitle":"Video provider requires cookies to play the video. Accept to continue or {url} it directly on the provider's site.","buttonTitle":"Accept","urlText":"watch"},"localOverride":false},"CachedAsset:text:en_US-components/users/UserLink-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/users/UserLink-1743097587452","value":{"authorName":"View Profile: {author}","anonymous":"Anonymous"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageSubject-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageSubject-1743097587452","value":{"noSubject":"(no subject)"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageBody-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageBody-1743097587452","value":{"showMessageBody":"Show More","mentionsErrorTitle":"{mentionsType, select, board {Board} user {User} message {Message} other {}} No Longer Available","mentionsErrorMessage":"The {mentionsType} you are trying to view has been removed from the community.","videoProcessing":"Video is being processed. Please try again in a few minutes.","bannerTitle":"Video provider requires cookies to play the video. Accept to continue or {url} it directly on the provider's site.","buttonTitle":"Accept","urlText":"watch"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageTime-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageTime-1743097587452","value":{"postTime":"Published: {time}","lastPublishTime":"Last Update: {time}","conversation.lastPostingActivityTime":"Last posting activity time: {time}","conversation.lastPostTime":"Last post time: {time}","moderationData.rejectTime":"Rejected time: {time}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/nodes/NodeIcon-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/nodes/NodeIcon-1743097587452","value":{"contentType":"Content Type {style, select, FORUM {Forum} BLOG {Blog} TKB {Knowledge Base} IDEA {Ideas} OCCASION {Events} other {}} icon"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageUnreadCount-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageUnreadCount-1743097587452","value":{"unread":"{count} unread","comments":"{count, plural, one { unread comment} other{ unread comments}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageViewCount-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageViewCount-1743097587452","value":{"textTitle":"{count, plural,one {View} other{Views}}","views":"{count, plural, one{View} other{Views}}"},"localOverride":false},"CachedAsset:text:en_US-components/kudos/KudosCount-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/kudos/KudosCount-1743097587452","value":{"textTitle":"{count, plural,one {{messageType, select, IDEA{Vote} other{Like}}} other{{messageType, select, IDEA{Votes} other{Likes}}}}","likes":"{count, plural, one{like} other{likes}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageRepliesCount-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageRepliesCount-1743097587452","value":{"textTitle":"{count, plural,one {{conversationStyle, select, IDEA{Comment} OCCASION{Comment} other{Reply}}} other{{conversationStyle, select, IDEA{Comments} OCCASION{Comments} other{Replies}}}}","comments":"{count, plural, one{Comment} other{Comments}}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/users/UserAvatar-1743097587452":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/users/UserAvatar-1743097587452","value":{"altText":"{login}'s avatar","altTextGeneric":"User's avatar"},"localOverride":false}}}},"page":"/tags/TagPage/TagPage","query":{"nodeId":"board:TechnicalArticles","tagName":"kubernetes"},"buildId":"q_bLpq2mflH0BeZigxpj6","runtimeConfig":{"buildInformationVisible":false,"logLevelApp":"info","logLevelMetrics":"info","openTelemetryClientEnabled":false,"openTelemetryConfigName":"f5","openTelemetryServiceVersion":"25.2.0","openTelemetryUniverse":"prod","openTelemetryCollector":"http://localhost:4318","openTelemetryRouteChangeAllowedTime":"5000","apolloDevToolsEnabled":false,"inboxMuteWipFeatureEnabled":false},"isFallback":false,"isExperimentalCompile":false,"dynamicIds":["./components/customComponent/CustomComponent/CustomComponent.tsx","./components/community/Navbar/NavbarWidget.tsx","./components/community/Breadcrumb/BreadcrumbWidget.tsx","./components/tags/TagsHeaderWidget/TagsHeaderWidget.tsx","./components/messages/MessageListForNodeByRecentActivityWidget/MessageListForNodeByRecentActivityWidget.tsx","./components/tags/TagSubscriptionAction/TagSubscriptionAction.tsx","./components/customComponent/CustomComponentContent/TemplateContent.tsx","../shared/client/components/common/List/ListGroup/ListGroup.tsx","./components/messages/MessageView/MessageView.tsx","./components/messages/MessageView/MessageViewInline/MessageViewInline.tsx","../shared/client/components/common/Pager/PagerLoadMore/PagerLoadMore.tsx","./components/customComponent/CustomComponentContent/HtmlContent.tsx","./components/customComponent/CustomComponentContent/CustomComponentScripts.tsx"],"appGip":true,"scriptLoader":[]}