kubernetes

71 Topics

Kubernetes architecture options with F5 Distributed Cloud Services
Summary F5 Distributed Cloud Services (F5 XC) can both integrate with your existing Kubernetes (K8s) clusters and/or host a K8s workload itself. Within these distinctions, we have multiple architecture options. This article explores four major architectures in ascending order of sophistication and advantages. Architecture #1: External Load Balancer (Secure K8s Gateway) Architecture #2: CE as a pod (K8s site) Architecture #3: Managed Namespace (vK8s) Architecture #4: Managed K8s (mK8s) Kubernetes Architecture Options As K8s continues to grow, options for how we run K8s and integrate with existing K8s platforms continue to grow. F5 XC can both integrate with your existing K8s clusters and/or run a managed K8s platform itself. Multiple architectures exist within these offerings too, so I was thoroughly confused when I first heard about these possibilities. A colleague recently laid it out for me in a conversation: "Michael, listen up: XC can either integrate with your K8s platform, run inside your K8s platform, host virtual K8s (Namespace-aaS), or run a K8s platform in your environment." I replied, "That's great. Now I have a mental model for differentiating between architecture options." This article will overview these architectures and provide 101-level context: when, how, and why would you implement these options? Side note 1: F5 XC concepts and terms F5 XC is a global platform that can provide networking and app delivery services, as well as compute (K8s workloads). We call each of our global PoP's a Regional Edge (RE). RE's are highly meshed to form the backbone of the global platform. They connect your sites, they can expose your services to the Internet, and they can run workloads. This platform is extensible into your data center by running one or more XC Nodes in your network, also called a Customer Edge (CE). A CE is a compute node in your network that registers to our global control plane and is then managed by a customer as SaaS. The registration of one or more CE's creates a customer site in F5 XC. A CE can run on a hypervisor (VMWare/KVM/Etc), a Hyperscaler (AWS, Azure, GCP, etc), baremetal, or even as a k8s pod, and can be deployed in HA clusters. XC Mesh functionality provides connectivity between sites, security services, and observability. Optionally, in addition, XC App Stack functionality allows a large and arbitrary number of managed clusters to be logically grouped into a virtual site with a single K8s mgmt interface. So where Mesh services provide the networking, App Stack services provide the Kubernetes compute mgmt. Our first 2 architectures require Mesh services only, and our last two require App Stack. Side note 2: Service-to-service communication I'm often asked how to allow services between clusters to communicate with each other. This is possible and easy with XC. Each site can publish services to every other site, including K8s sites. This means that any K8s service can be reachable from other sites you choose. And this can be true in any of the architectures below, although more granular controls are possible with the more sophisticated architectures. I'll explore this common question more in a separate article. Architecture 1: External Load Balancer (Secure K8s Gateway) In a Secure Kubernetes Gateway architecture, you have integration with your existing K8s platform, using the XC node as the external load balancer for your K8s cluster. In this scenario, you create a ServiceAccount and kubeconfig file to configure XC. The XC node then performs service discovery against your K8s API server. I've covered this process in a previous article, but the advantage is that you can integrate with existing K8s platforms. This allows exposing both NodePort and ClusterIP services via the XC node. XC is not hosting any workloads in this architecture, but it is exposing your services to your local network, or remote sites, or the Internet. In the diagram above, I show a web application being accesssed from a remote site (and/or the Internet) where the origin pool is a NodePort service discovered in a K8s cluster. Architecture 2: Run a site within a K8s cluster (K8s site type) Creating a K8s site is easy - just deploy a single manifest found here. This file deploys multiple resources in your cluster, and together these resources work to provide the services of a CE, and create a customer site. I've heard this referred to as "running a CE inside of K8s" or "running your CE as a pod". However, when I say "CE node" I'm usually referring to a discreet compute node like a VM or piece of hardware; this architecture is actually a group of pods and related resources that run within K8s to create a XC customer site. With XC running inside your existing cluster, you can expose services within the cluster by DNS name because the site will resolve these from within the cluster. Your service can then be exposed anywhere by the F5 XC platform. This is similar to Architecture 1 above, but with this model, your site is simply a group of pods within K8s. An advantage here is the ability to expose services of other types (e.g. ClusterIP). A site deployed into a K8s cluster will only support Mesh functionality and does not support AppStack functionality (i.e., you cannot run a cluster within your cluster). In this architecture, XC acts as a K8s ingress controller with built-in application security. It also enables Mesh features, such as publishing of other sites' services on this site, and publishing of this site's discovered services on other sites. Architecture 3: vK8s (Namespace-as-a-Service) If the services you use include AppStack capabilities, then architectures #3 and #4 are possible for you. In these scenarios, our XC node actually runs your K8s on your workloads. We are no longer integrating XC with your existing K8s platform. XC is the platform. A simple way to run K8s workloads is to use a virtual k8s (vK8s) architecture. This could be referred to as a "managed Namespace" because by creating a vK8s object in XC you get a single namespace in a virtual cluster. Your Namespace can be fully hosted (deployed to RE's) or run on your VM's (CE's), or both. Your kubeconfig file will allow access to your Namespace via the hosted API server. Via your regular kubectl CLI (or via the web console) you can create/delete/manage K8s resources (Deployments, Services, Secrets, ServiceAccounts, etc) and view application resource metrics. This is great if you have workloads that you want to deploy to remote regions where you do not have infrastructure and would prefer to run in F5's RE's, or if you have disparate clusters across multiple sites and you'd like to manage multiple K8s clusters via a single centralized, virtual cluster. Best practice guard rails for vK8s With a vK8s architecture, you don't have your own cluster, but rather a managed Namespace. So there are some restrictions (for example, you cannot run a container as root, bind to a privileged port, or to the Host network). You cannot create CRD's, ClusterRoles, PodSecurityPolicies, or Namespaces, so K8s operators are not supported. In short, you don't have a managed cluster, but a managed Namespace on a virtual cluster. Architecture 4: mK8s (Managed K8s) In managed k8s (mk8s, also known as physical K8s or pk8s) deployment, we have an enterprise-level K8s distribution that is run at your site. This means you can use XC to deploy/manage/upgrade K8s infrastructure, but you manage the Kubernetes resources. The benefits include what is typical for 3rd-party K8s mgmt solutions, but also some key differentiators: multi-cloud, with automation for Azure, AWS, and GCP environments consumed by you as SaaS enterprise-level traffic control natively allows a large and arbitrary number of managed clusters to be logically managed with a single K8s mgmt interface You can enable kubectl access against your local cluster and disable the hosted API server, so your kubeconfig file can point to a global URL or a local endpoint on-prem. Another benefit of mK8s is that you are running a full K8s cluster at your site, not just a Namespace in a virtual cluster. The restrictions that apply to vK8s (see above) do not apply to mK8s, so you could run privileged pods if required, use Operators that make use of ClusterRoles and CRDs, and perform other tasks that require cluster-wide access. Traffic management controls with mK8s Because your workloads run in a cluster managed by XC, we can apply more sophisticated and native policies to K8s traffic than non-managed clusters in earlier architectures: Service isolation can be enforced within the cluster, so that pods in a given namespace cannot communicate with services outside of that namespace, by default. More service-to-service controls exist so that you can decide which services can reach with other services with more granularity. Egress control can be natively enforced for outbound traffic from the cluster, by namespace, labels, IP ranges, or other methods. E.g.: Svc A can reach myapi.example.com but no other Internet service. WAF policies, bot defense, L3/4 policies, etc—all of these policies that you have typically applied with network firewalls, WAF's, etc—can be applied natively within the platform. This architecture took me a long time to understand, and longer to fully appreciate. But once you have run your workloads natively on a managed K8s platform that is connected to a global backbone and capable of performing network and application delivery within the platform, the security and traffic mgmt benefits become very compelling. Conclusion: As K8s continues to expand, management solutions of your clusters make it possible to secure your K8s services, whether they are managed by XC or exist in disparate clusters. With F5 XC as a global platform consumed as a service—not a discreet installation managed by you—the available architectures here are unique and therefore can accommodate the diverse (and changing!) ways we see K8s run today. Related Articles Securely connecting Kubernetes Microservices with F5 Distributed Cloud Multi-cluster Multi-cloud Networking for K8s with F5 Distributed Cloud - Architecture Pattern Multiple Kubernetes Clusters and Path-Based Routing with F5 Distributed Cloud
MichaelOLeary
Jan 11, 2023 Place Technical Articles
9.7KViews
29likes
5Comments
3 Ways to use F5 BIG-IP with OpenShift 4
F5 BIG-IP can provide key infrastructure and application services in a RedHat OpenShift 4 environment. Examples include providing core load balancing for the OpenShift API and Router, DNS services for the cluster, a supplement or replacement for the OpenShift Router, and security protection for the OpenShift management and application services. #1. Core Services OpenShift 4 requires a method to provide high availability to the OpenShift API (port 6443), MachineConfig (22623), and Router services (80/443). BIG-IP Local Traffic Manager (LTM) can provide these trusted services easily. OpenShift also requires several DNS records that the BIG-IP can provide accelerated responses as a DNS cache and/or providing Global Server Load Balancing of cluster DNS records. Additional documentation about OpenShift 4 Network Requirements (RedHat) Networking Requirements for user-provisioned infrastructure #2 OpenShift Router RedHat provides their own OpenShift Router for L7 load balancing, but the F5 BIG-IP can also provide these services using Container Ingress Services. Instead of deploying load balancing resources on the same nodes that are hosting OpenShift workloads; F5 BIG-IP provides these services outside of the cluster on either hardware or Virtual Edition platforms. Container Ingress Services can run either as an auxiliary router to the included router or a replacement. Additional articles that are related to Container Ingress Services • Using F5 BIG-IP Controller for OpenShift #3 Security F5 can help filter, authenticate, and validate requests that are going into or out of an OpenShift cluster. LTM can be used to host sensitive SSL resources outside of the cluster (including on a hardware HSM if necessary) as well as filtering of requests (i.e. disallow requests to internal resources like the management console). Advanced Web Application Firewall (AWAF) policies can be deployed to stymie bad actors from reaching sensitive applications. Access Policy Manager can provide OpenID Connect services for the OpenShift management console and help with providing identity services for applications and microservices that are running on OpenShift (i.e. converting BasicAuth request into a JWT token for a microservice). Additional documentation related to attaching a security policy to an OpenShift Route • AS3 Override Where Can I Try This? The environment that was used to write this article and create the companion video can be found at: https://github.com/f5devcentral/f5-k8s-demo/tree/ocp4/ocp4. For folks that are part of F5 you can access this in our Unified Demo Framework and can schedule labs with customers/partners (search for "OpenShift 4.3 with CIS"). I plan on publishing a version of this demo environment that can run natively in AWS. Check back to this article for any updates. Thanks!
Eric_Chen
Jun 29, 2020 Place Technical Articles
8.7KViews
6likes
3Comments
F5 Distributed Cloud - Regional Decryption with Virtual Sites
In this article we discuss how the F5 Distributed Cloud can be configured to support regulatory demands for TLS termination of traffic to specific regions around the world. The article provides insight into the F5 Distributed Cloud global backbone and application delivery network (ADN). The article goes on to inspect how the F5 Distriubted Cloud is able to achieve these custom topologies in a multi-tenant architecture while adhearing to the "rules of the internet" for route summarization. Read on to learn about the flexibility of F5's SaaS platform providing application delivery and security solutions for your applications.
MattHarmon
Jan 17, 2023 Place Technical Articles
6.4KViews
17likes
2Comments
Better together - F5 Container Ingress Services and NGINX Plus Ingress Controller Integration
Introduction The F5 Container Ingress Services (CIS) can be integrated with the NGINX Plus Ingress Controllers (NIC) within a Kubernetes (k8s) environment. The benefits are getting the best of both worlds, with the BIG-IP providing comprehensive L4 ~ L7 security services, while leveraging NGINX Plus as the de facto standard for micro services solution. This architecture is depicted below. The integration is made fluid via the CIS, a k8s pod that listens to events in the cluster and dynamically populates the BIG-IP pool pointing to the NIC's as they scale. There are a few components need to be stitched together to support this integration, each of which is discussed in detail over the proceeding sections. NGINX Plus Ingress Controller Follow this (https://docs.nginx.com/nginx-ingress-controller/installation/building-ingress-controller-image/) to build the NIC image. The NIC can be deployed using the Manifests either as a Daemon-Set or a Service. See this ( https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/ ). A sample Deployment file deploying NIC as a Service is shown below, apiVersion: apps/v1 kind: Deployment metadata: name: nginx-ingress namespace: nginx-ingress spec: replicas: 3 selector: matchLabels: app: nginx-ingress template: metadata: labels: app: nginx-ingress #annotations: #prometheus.io/scrape: "true" #prometheus.io/port: "9113" spec: serviceAccountName: nginx-ingress imagePullSecrets: - name: abgmbh.azurecr.io containers: - image: abgmbh.azurecr.io/nginx-plus-ingress:edge name: nginx-plus-ingress ports: - name: http containerPort: 80 - name: https containerPort: 443 #- name: prometheus #containerPort: 9113 securityContext: allowPrivilegeEscalation: true runAsUser: 101 #nginx capabilities: drop: - ALL add: - NET_BIND_SERVICE env: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name args: - -nginx-plus - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret - -ingress-class=sock-shop #- -v=3 # Enables extensive logging. Useful for troubleshooting. #- -report-ingress-status #- -external-service=nginx-ingress #- -enable-leader-election #- -enable-prometheus-metrics Notice the ‘- -ingress-class=sock-shop’ argument, it means that the NIC will only work with an Ingress that is annotated with ‘sock-shop’. The absence of this annotation makes NIC the default for all Ingress created. Below shows the counterpart Ingress with the ‘sock-shop’ annotation. apiVersion: extensions/v1beta1 kind: Ingress metadata: name: sock-shop-ingress annotations: kubernetes.io/ingress.class: "sock-shop" spec: tls: - hosts: - socks.ab.gmbh secretName: wildcard.ab.gmbh rules: - host: socks.ab.gmbh http: paths: - path: / backend: serviceName: front-end servicePort: 80 This Ingress says if hostname is socks.ab.gmbh and path is ‘/’, send traffic to a service named ‘front-end’, which is part of the socks application itself. The above concludes Ingress configuration with the NIC. F5 Container Ingress Services The next step is to leverage the CIS to dynamically populate the BIG-IP pool with the NIC addresses. Follow this ( https://clouddocs.f5.com/containers/v2/kubernetes/kctlr-app-install.html ) to deploy the CIS. A sample Deployment file is shown below, apiVersion: extensions/v1beta1 kind: Deployment metadata: name: k8s-bigip-ctlr-deployment namespace: kube-system spec: # DO NOT INCREASE REPLICA COUNT replicas: 1 template: metadata: name: k8s-bigip-ctlr labels: app: k8s-bigip-ctlr spec: # Name of the Service Account bound to a Cluster Role with the required # permissions serviceAccountName: bigip-ctlr containers: - name: k8s-bigip-ctlr image: "f5networks/k8s-bigip-ctlr" env: - name: BIGIP_USERNAME valueFrom: secretKeyRef: # Replace with the name of the Secret containing your login # credentials name: bigip-login key: username - name: BIGIP_PASSWORD valueFrom: secretKeyRef: # Replace with the name of the Secret containing your login # credentials name: bigip-login key: password command: ["/app/bin/k8s-bigip-ctlr"] args: [ # See the k8s-bigip-ctlr documentation for information about # all config options # https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest "--bigip-username=$(BIGIP_USERNAME)", "--bigip-password=$(BIGIP_PASSWORD)", "--bigip-url=https://x.x.x.x:8443", "--bigip-partition=k8s", "--pool-member-type=cluster", "--agent=as3", "--manage-ingress=false", "--insecure=true", "--as3-validation=true", "--node-poll-interval=30", "--verify-interval=30", "--log-level=INFO" ] imagePullSecrets: # Secret that gives access to a private docker registry - name: f5-docker-images # Secret containing the BIG-IP system login credentials - name: bigip-login Notice the following arguments below. They tell the CIS to consume AS3 declaration to configure the BIG-IP. According to PM, CCCL (Common Controller Core Library) – used to orchestrate F5 BIG-IP, is getting removed this sprint for the CIS 2.0 release. '--manage-ingress=false' means CIS is not doing anything for Ingress resources defined within the k8s, this is because that CIS is not the Ingress Controller, NGINX Plus is, as far as k8s is concerned. The CIS will create a partition named k8s_AS3 on the BIG-IP, this is used to hold L4~7 configuration relating to the AS3 declaration. The best practice is also to manually create a partition named 'k8s' (in our example), where networking info will be stored (e.g., ARP, FDB). "--bigip-url=https://x.x.x.x:8443", "--bigip-partition=k8s", "--pool-member-type=cluster", "--agent=as3", "--manage-ingress=false", "--insecure=true", "--as3-validation=true", To apply AS3, the declaration is embedded within a ConfigMap applied to the CIS pod. kind: ConfigMap apiVersion: v1 metadata: name: as3-template namespace: kube-system labels: f5type: virtual-server as3: "true" data: template: | { "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "id":"1847a369-5a25-4d1b-8cad-5740988d4423", "schemaVersion": "3.16.0", "Nginx_IC": { "class": "Tenant", "Nginx_IC_vs": { "class": "Application", "template": "https", "serviceMain": { "class": "Service_HTTPS", "virtualAddresses": [ "10.1.0.14" ], "virtualPort": 443, "redirect80": false, "serverTLS": { "bigip": "/Common/clientssl" }, "clientTLS": { "bigip": "/Common/serverssl" }, "pool": "Nginx_IC_pool" }, "Nginx_IC_pool": { "class": "Pool", "monitors": [ "https" ], "members": [ { "servicePort": 443, "shareNodes": true, "serverAddresses": [] } ] } } } } } They are telling the BIG-IP to create a tenant called ‘Nginx_IC’, a virtual named ‘Nginx_IC_vs’ and a pool named ‘Nginx_IC_pool’. The CIS will update the serverAddresses with the NIC addresses dynamically. Now, create a Service to expose the NIC’s. apiVersion: v1 kind: Service metadata: name: nginx-ingress namespace: nginx-ingress labels: cis.f5.com/as3-tenant: Nginx_IC cis.f5.com/as3-app: Nginx_IC_vs cis.f5.com/as3-pool: Nginx_IC_pool spec: type: ClusterIP ports: - port: 443 targetPort: 443 protocol: TCP name: https selector: app: nginx-ingress Notice the labels, they match with the AS3 declaration and this allows the CIS to populate the NIC’s addresses to the correct pool. Also notice the kind of the manifest ‘Service’, this means only a Service is created, not an Ingress, as far as k8s is concerned. On the BIG-IP, the following should be created. The end product is below. Please note that this article is focused solely on control plane, that is, how to get the CIS to populate the BIG-IP with NIC's addresses. The specific mechanisms to deliver packets from the BIG-IP to the NIC's on the data plane is not discussed, as it is decoupled from control plane. For data plane specifics, please take a look here ( https://clouddocs.f5.com/containers/v2/ ). Hope this article helps to lift the veil on some integration mysteries.
Chris_Zhang
Mar 05, 2020 Place Technical Articles
5.5KViews
11likes
27Comments
Exploring Kubernetes API using Wireshark part 1: Creating, Listing and Deleting Pods
Related Articles: Exploring Kubernetes API using Wireshark part 2: Namespaces Exploring Kubernetes API using Wireshark part 3: Python Client API Quick Intro This article answers the following question: What happens when we create, list and delete pods under the hood? More specifically on the wire. I used these 3 commands: I'll show you on Wireshark the communication between kubectl client and master node (API) for each of the above commands. I used a proxy so we don't have to worry about TLS layer and focus on HTTP only. Creating NGINX pod pcap: creating_pod.pcap (use http filter on Wireshark) Here's our YAML file: Here's how we create this pod: Here's what we see on Wireshark: Behind the scenes, kubectl command sent an HTTP POST with our YAML file converted to JSON but notice the same thing was sent (kind, apiVersion, metadata, spec): You can even expand it if you want to but I didn't to keep it short. Then, Kubernetes master (API) responds with HTTP 201 Created to confirm our pod has been created: Notice that master node replies with similar data with the additional status column because after pod is created it's supposed to have a status too. Listing Pods pcap: listing_pods.pcap (use http filter on Wireshark) When we list pods, kubectl just sends a HTTP GET request instead of POST because we don't need to submit any data apart from headers: This is the full GET request: And here's the HTTP 200 OK with JSON file that contains all information about all pods from default's namespace: I just wanted to emphasise that when you list a pod the resource type that comes back is PodList and when we created our pod it was just Pod. Remember? The other thing I'd like to point out is that all of your pods' information should be listed under items. All kubectl does is to display some of the API's info in a humanly readable way. Deleting NGINX pod pcap: deleting_pod.pcap (use http filter on Wireshark) Behind the scenes, we're just sending an HTTP DELETE to Kubernetes master: Also notice that the pod's name is also included in the URI: /api/v1/namespaces/default/pods/nginx ← this is pods' name HTTP DELETE just like HTTP GET is pretty straightforward: Our master node replies with HTTP 200 OK as well as some json file with all the info about the pod, including about it's termination: It's also good to emphasise here that when our pod is deleted, master node returns JSON file with all information available about the pod. I highlighted some interesting info. For example, resource type is now just Pod (not PodList when we're just listing our pods).
Rodrigo_Albuque
Jun 05, 2019 Place Technical Articles
4.6KViews
3likes
0Comments
Service Discovery and authentication options for Kubernetes providers (EKS, AKS, GCP)
Summary Hosted Kubernetes (K8s) providers use different services for authenticating to your hosted K8s cluster. A kubeconfig file will define your cluster's API server address and your authentication to this server. Uploading this kubeconfig file is how you enable service discovery from another platform like F5 Distributed Cloud (F5 XC). What is Service Discovery in F5 XC? Service Discovery in this case means something outside the cluster is querying the K8s API and learning the services and pods running inside the cluster. In our case, we want to send traffic into K8s via a platform that can publish our k8s services internally or publicly, and also apply security to the applications exposed. How do I configure k8s Service Discovery? Follow these instructions to configure K8s Service Discovery in XC. You will notice that a kubeconfig file is typically required to be uploaded to XC. User authentication to AKS, EKS, and GKE In K8s, User authentication happens outside of the cluster. There is no "User" resource in K8s - you cannot create a User with kubectl. Unlike ServiceAccounts (SAs), which are created inside K8s and whose authentication secrets exist inside K8s, a User is authenticated with a system outside of K8s. There are multiple authentication schemes, and certificates and OAuth are very common to see. For this article, I'll review the typical authentication providers major providers: Azure's AKS, AWS's EKS, and Google's GKE. Azure AKS AKS authentication (authn) starts at Azure Active Directory, and authorization (authz) can be applied at AAD or k8s RBAC. For this article I am focused on authn (not authz). If you follow official instructions to create an AKS cluster via CLI, you will run the az aks get-credentials command to get the access credentials for an AKS cluster and merge them into your kubeconfig file. You will see that a client certificate and key is included in the User section of your kubeconfig file. This kubeconfig file can be uploaded to F5 XC for successful service discovery because it does not rely on any additional software to be on the kubectl client. (However, be cautious. Don't share the kubeconfig file further, since it contains credentials to AKS. Microsoft provides instructions to use Azure role-based access control (Azure RBAC) to control access to these credentials. These Azure roles let you define who can retrieve the kubeconfig file, and what permissions they then have within the cluster.) AWS EKS "Typical" EKS kubeconfig The AWS EKS cluster authentication process requires the use of an AWS IAM identity, and typically relies on an additional software being installed on the kubectl client machine. Instead of credentials being configured directly in the kubeconfig file, either the aws cli or aws-iam-authenticator are used locally on the kubectl client to generate a token. This token is sent to the k8s master API server, which is verified against AWS IAM (see steps 1-2 in the image below). Use a service account for EKS access The AWS user that created the cluster will have system:masters permissions to your cluster, but if you want to add another user to your cluster, you must edit the aws-auth ConfigMap within Kubernetes. AWS has instructions to do this so I won't repeat them here, but the implications should be clear for what we are trying to do: You don't want a 3rd party authenticating to EKS as your primary user account. If you created the EKS cluster with your primary account, you should create a different user for EKS access. Create an AWS IAM user for this purpose, and then add this user to the aws-auth ConfigMap within Kubernetes. EKS requires authentication from the kubectl client. Instead of hard-coding credentials into the kubeconfig file, the kubeconfig file tells the client to run one of the two following commands that rely on the existing of additional software: aws-iam-authenticator token -i [clustername] or aws eks get-token --region [region] --cluster-name [cluster] We can update our kubeconfig file so that it can use credentials directly in the kubeconfig file, so that we can upload this file to F5 XC for service discovery. Creating a kubeconfig file that uses aws cli or aws-iam-authenticator with provided credentials In the case of F5 XC, we can see from the instructions that "...,you must add AWS credentials in the kubeconfig file for successful service discovery". This means that to upload your kubeconfig to XC you will need to configure it to use the aws-iam-authenticator option, and provide your IAM aws_access_key_id and aws_secret_access_key as env vars in your kubeconfig. Example kubeconfig: .... <everything else in kubeconfig file> ... users: - name: arn:aws:eks:us-east-1:<my_aws_acct_number>:cluster/<my_eks_cluster_name> user: exec: apiVersion: client.authentication.k8s.io/v1alpha1 args: - token - -i - <my_eks_cluster_name> command: aws-iam-authenticator env: - name: AWS_ACCESS_KEY_ID value: "<my_aws_access_key_id>" - name: AWS_SECRET_ACCESS_KEY value: "<my_aws_secret_access_key>" I can use the above kubeconfig file for successful service discovery in XC. I did make some notes: In my testing, the aws cli method did not work in XC. It appears that the XC node has aws-iam-authenticator installed, but not aws cli. Notice the apiVersion above is v1alpha1. This works when uploaded to F5 XC, but when testing this file on my local workstation, I had to ensure my kubectl client was downgraded to kubectl v 1.23.6 or lower. v1alpha1 has been deprecated in newer kubectl versions, but was required for me to authenticate with the aws-iam-authenticator method. This is not ideal. It appears that the XC node has a version of kubectl that allows for v1alpha1 auth, but we know this will be deprecated eventually. These notes led me to asking: can I get a kubeconfig file to authenticate to AWS without 3rd party tools? Read on for how to do this, but first I'll cover Google's GKE. Google GKE Similar to EKS, GKE uses gcloud cli as a "credential helper" in the typical kubeconfig file for GKE cluster administration. This excellent article covers GKE authentication very well. This means we need to take a similar approach as we did for EKS, if we want to authenticate to the GKE API server from a location without gcloud installed. For GKE, I followed an approach I learnd here this blog post, but I'll summarize for you now: Create a Kubeconfig file that contains your cluster API server address and CA certificate. These are safe to commit in git because they do not need to be private. Create a Service Account in Google, generate a key for this service account and save it in JSON format. Export the variable GOOGLE_APPLICATION_CREDENTIALS so the value is the location of your .json file. Use kubectl commands. These should work because kubectl as a client is aware of the env var above. This article from Google explains the community's requirement that provider-specific code is removed from the OSS codebase in v1.25 (as of my writing, v1.24 is the most recent release). This requires an update to kubectl that will remove existing functionality that allows kubectl to authenticate to GKE. Soon, you will need to use a new credential helper from Google called gke-gcloud-auth-plugin. Soon, you'll be relying on a different 3rd party plugin for GKE authentication. So our question will come up again, as it did earlier: Can't I just have a kubeconfig file that doesn't rely on a 3rd party library? Creating a kubeconfig file that does not rely on other software What if we need to upload our kubeconfig to a machine that does not have the aws cli, aws-iam-authenticator, or any other 3rd party credential helpers installed? Sometimes you just need a kubeconfig that doesn't rely on 3rd party software or their IAM service, and this is what ServiceAccounts can achieve. I wrote an article with a script that you can use, focused on K8s SA's for service discovery with F5 XC. Since it's all documented in another article I'll summarize the steps here. Create a Service Account (SA) with permissions to discover services. Export the secret for that SA to use as an auth token. Create a kubeconfig file that targets your cluster and authenticates with your SA. This file can be uploaded to XC. Summary A kubeconfig file is relatively straightforward for most k8s admins. However, uploading a kubeconfig file into a hosted service, for the purpose of k8s service discovery, will sometimes require further analysis. Remember: Your kubeconfig file defines your authentication to the k8s API server. So use a service account (not your primary user account) If your kubeconfig file relies on a helper like aws cli, aws-iam-authenticator, or gcloud, you can probably work around this by editing your kubeconfig file to use a SA and token instead. This may be required if you upload the kubeconfig file to a hosted platform like F5 XC. All this might seem complex at first, but after service discovery is configured the ability to expose your services anywhere - publicly or privately - is incredibly easy and powerful. Good luck and reach out with feedback!
MichaelOLeary
Aug 31, 2022 Place Technical Articles
4.5KViews
3likes
3Comments
CIS and Kubernetes - Part 1: Install Kubernetes and Calico
Welcome to this series to see how to: Install Kubernetes and Calico (Part 1) Deploy F5 Container Ingress Services (F5 CIS) to tie applications lifecycle to our application services (Part 2) Here is the setup of our lab environment: BIG-IP Version: 15.0.1 Kubernetes component: Ubuntu 18.04 LTM We consider that your BIG-IPs are already setup and running: Licensed and setup as a cluster The networking setup is already done Part 1: Install Kubernetes and Calico Setup our systems before installing kubernetes Step1: Update our systems and install docker To run containers in Pods, Kubernetes uses a container runtime. We will use docker and follow the recommendation provided here As root on ALL Kubernetes components (Master and Node): # Install packages to allow apt to use a repository over HTTPS apt-get -y update && apt-get install -y apt-transport-https ca-certificates curl software-properties-common # Add Docker’s official GPG key curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - # Add Docker apt repository. add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" # Install Docker CE. apt-get -y update && apt-get install -y docker-ce=18.06.2~ce~3-0~ubuntu # Setup daemon. cat > /etc/docker/daemon.json <<EOF { "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "storage-driver": "overlay2" } EOF mkdir -p /etc/systemd/system/docker.service.d # Restart docker. systemctl daemon-reload systemctl restart docker We may do a quick test to ensure docker run as expected: docker run hello-world Step2: Setup Kubernetes tools (kubeadm, kubelet and kubectl) To setup Kubernetes, we will leverage the following tools: kubeadm: the command to bootstrap the cluster. kubelet: the component that runs on all of the machines in your cluster and does things like starting pods and containers. kubectl: the command line util to talk to your cluster. As root on ALL Kubernetes components (Master and Node): curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - cat <<EOF | tee /etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF apt-get -y update We can review which version of kubernetes is supported with F5 Container Ingress Services here At the time of this article, the latest supported version is v1.13.4. We'll make sure to install this specific version with our following step apt-get install -qy kubelet=1.13.4-00 kubeadm=1.13.4-00 kubectl=1.13.4-00 kubernetes-cni=0.6.0-00 apt-mark hold kubelet kubeadm kubectl Install Kubernetes Step1: Setup Kubernetes with kubeadm We will follow the steps provided in the documentation here As root on the MASTER node (make sure to update the api server address to reflect your master node IP): kubeadm init --apiserver-advertise-address=10.1.20.20 --pod-network-cidr=192.168.0.0/16 Note: SAVE somewhere the kubeadm join command. It is needed to "assimilate" the node later. In my example, it looks like the following (YOURS WILL BE DIFFERENT): kubeadm join 10.1.20.20:6443 --token rlbc20.va65z7eauz89mmuv --discovery-token-ca-cert-hash sha256:42eca5bf49c645ff143f972f6bc88a59468a30276f907bf40da3bcf5127c0375 Now you should NOT be ROOT anymore. Go back to your non root user. Since i use Ubuntu, i'll use the default "ubuntu" user Run the following commands as highlighted in the screenshot above: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Step2: Install the networking component of Kubernetes The last step is to setup the network related to our k8s infrastructure. In our kubeadm init command, we used --pod-network-cidr=192.168.0.0/16 in order to be able to setup next on network leveraging Calico as documented here kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml You may monitor the deployment by running the command: kubectl get pods --all-namespaces After some time (<1 min), everything should have a "Running" status. Make sure that CoreDNS started also properly. If everything is up and running, we have our master setup properly and can go to the node to setup k8s on it. Step3: Add the Node to our Kubernetes Cluster Now that the master is setup properly, we can assimilate the node. You need to retrieve the "kubeadmin join …" command that you received at the end of the "kubeadm init …" cmd. You must run the following command as ROOT on the Kubernetes NODE (remember that you got a different hash and token, the command below is an example): kubeadm join 10.1.20.20:6443 --token rlbc20.va65z7eauz89mmuv --discovery-token-ca-cert-hash sha256:42eca5bf49c645ff143f972f6bc88a59468a30276f907bf40da3bcf5127c0375 We can check the status of our node by running the following command on our MASTER (ubuntu user) kubectl get nodes Both component should have a "Ready" status. Last step is to setup Calico between our BIG-IPs and our Kubernetes cluster Setup Calico We need to setup Calico on our BIG-IPs and k8S components. We will setup our environment with the following AS Number: 64512 Step1: BIG-IPs Calico setup F5 has documented this procedure here We will use our self IPs on the internal network. Therefore we need to make sure of the following: The self IP has a portlock down setup to "Allow All" Or add a TCP custom port to the self IP: TCP port 179 You need to allow BGP on the default route domain 0 on your BIG-IPs. Connect to the BIG-IP GUI on go into Network > Route domain. Click on Route Domain "0" and allow BGP Click on "Update" Once this is done, connect via SSH and get into a bash shell on both BIG-IPs Run the following commands: #access the IMI Shell imish #Switch to enable mode enable #Enter configuration mode config terminal #Setup route bgp with AS Number 64512 router bgp 64512 #Create BGP Peer group neighbor calico-k8s peer-group #assign peer group as BGP neighbors neighbor calico-k8s remote-as 64512 #we need to add all the peers: the other BIG-IP, our k8s components neighbor 10.1.20.20 peer-group calico-k8s neighbor 10.1.20.21 peer-group calico-k8s #on BIG-IP1, run neighbor 10.1.20.12 peer-group calico-k8s #on BIG-IP2, run neighbor 10.1.20.11 peer-group calico-k8s #save configuration write #exit end You can review your setup with the command show ip bgp neighbors Note: your other BIG-IP should be identified with a router ID and have a BGP state of "Active". The k8s node won't have a router ID since BGP hasn't already been setup on those nodes. Keep your BIG-IP SSH sessions open. We'll re-use the imish terminal once our k8s components have Calico setup Step2: Kubernetes Calico setup On the MASTER node (not as root), we need to retrieve the calicoctl binary curl -O -L https://github.com/projectcalico/calicoctl/releases/download/v3.10.0/calicoctl chmod +x calicoctl sudo mv calicoctl /usr/local/bin We need to setup calicoctl as explained here sudo mkdir /etc/calico Create a file /etc/calico/calicoctl.cfg with your preferred editor (you'll need sudo privilegies). This file should contain the following apiVersion: projectcalico.org/v3 kind: CalicoAPIConfig metadata: spec: datastoreType: "kubernetes" kubeconfig: "/home/ubuntu/config" Note: you may have to change the path specified by the kubeconfig parameter based on the user you use to do kubectl command To make sure that calicoctl is properly setup, run the command calicoctl get nodes You should get a list of your Kubernetes nodes Now we can work on our Calico/BGP configuration as documented here On the MASTER node: cat << EOF | calicoctl create -f - apiVersion: projectcalico.org/v3 kind: BGPConfiguration metadata: name: default spec: logSeverityScreen: Info nodeToNodeMeshEnabled: true asNumber: 64512 EOF Note: Because we setup nodeToNodeMeshEnabled to True, the k8s node will receive the same config We may now setup our BIG-IP BGP peers. Replace the peerIP Value with the IP of your BIG-IPs cat << EOF | calicoctl create -f - apiVersion: projectcalico.org/v3 kind: BGPPeer metadata: name: bgppeer-global-bigip1 spec: peerIP: 10.1.20.11 asNumber: 64512 EOF cat << EOF | calicoctl create -f - apiVersion: projectcalico.org/v3 kind: BGPPeer metadata: name: bgppeer-global-bigip2 spec: peerIP: 10.1.20.12 asNumber: 64512 EOF Review your setup with the command: calicoctl get bgpPeer If you go back to your BIG-IP SSH connections, you may check that your Kubernetes nodes have a router ID now in your BGP configuration: imish show ip bgp neighbors Summary So far we have: Setup Kubernetes Setup Calico between our BIG-IPs and our Kubernetes cluster In the next article, we will setup F5 container Ingress Services (F5 CIS)
Nicolas_Menant
Nov 04, 2019 Place Technical Articles
4.4KViews
1like
1Comment
Understanding Modern Application Architecture - Part 1
This is part 1 of a series. Here are the other parts: Understanding Modern Application Architecture - Part 2 Understanding Modern Application Architecture - Part 3 Over the past decade, there has been a change taking place in how applications are built. As applications become more expansive in capabilities and more critical to how a business operates, (or in many cases, the application is the business itself) a new style of architecture has allowed for increased scalability, portability, resiliency, and agility. To support the goals of a modern application, the surrounding infrastructure has had to evolve as well. Platforms like Kubernetes have played a big role in unlocking the potential of modern applications and is a new paradigm in itself for how infrastructure is managed and served. To help our community transition the skillset they've built to deal with monolithic applications, we've put together a series of videos to drive home concepts around modern applications. This article highlights some of the details found within the video series. In these first three videos, we breakdown the definition of a Modern Application. One might think that by name only, a modern application is simply an application that is current. But we're actually speaking in comparison to a monolithic application. Monolithic applications are made up of a single, or a just few pieces. They are rigid in how they are deployed and fragile in their dependencies. Modern applications will instead incorporate microservices. Where a monolithic application might have all functions built into one broad encompassing service, microservices will break down the service into smaller functions that can be worked on separately. A modern application will also incorporate 4 main pillars. Scalability ensures that the application can handle the needs of a growing user base, both for surges as well as long term growth. Portability ensures that the application can be transportable from its underlying environment while still maintaining all of its functionality and management plane capabilities. Resiliency ensures that failures within the system go unnoticed or pose minimal disruption to users of the application. Agility ensures that the application can accommodate for rapid changes whether that be to code or to infrastructure. There are also 6 design principles of a modern application. Being agnostic will allow the application to have freedom to run on any platform. Leveraging open source software where it makes sense can often allow you to move quickly with an application but later be able to adopt commercial versions of that software when full support is needed. Defining by code allows for more uniformity of configuration and move away rigid interfaces that require specialized knowledge. Automated CI/CD processes ensures the quick integration and deployment of code so that improvements are constantly happening while any failures are minimized and contained. Secure development ensures that application security is integrated into the development process and code is tested thoroughly before being deployed into production. Distributed Storage and Infrastructure ensures that applications are not bound by any physical limitations and components can be located where they make the most sense. These videos should help set the foundation for what a modern application is. The next videos in the series will start to define the fundamental technical components for the platforms that bring together a modern application. Continued in Part 2
buulam
Jan 25, 2022 Place Technical Articles
4KViews
8likes
0Comments
Exploring Kubernetes API using Wireshark part 3: Python Client API
Quick Intro In this article, we continue our exploration of Kubernetes API but this time we're going to use Python along with Wireshark. I strongly advise you to go through Exploring Kubernetes API using Wireshark part 1: Creating, Listing and Deleting Pods and Exploring Kubernetes API using Wireshark part 2: Namespaces first. We're not creating any app here. I'll just show you how you can explore Kubernetes API using Python's client API and Wireshark output retrieved from kubectl get pods command: In case you want to follow along, I've set up gcloud tool as I'm using Google Cloud and GOOGLE_APPLICATION_CREDENTIALS environment variable on my Mac so I had no problem authenticating to Kubernetes API. I hope you understand that when I say Kubernetes API, I'm talking about the API on Kubernetes Master node on my Google Cloud's Kubernetes cluster. What I'm going to do here In this article, we're going to do the following (using Python): Authenticate to kube API Connect to /api/v1 and then move to /api/v1/namespaces/default/pods Add pod information to a variable named pods from variable pods, create a list of: pod names pod status At the end, we display the above info Authenticating/Authorising your code to contact Kube API Here we're loading auth and cluster info from kube-config file and storing into into our client API config: Connecting to /api/v1 We now store into v1 variable the API object. For the sake of simplicity, think of it as we were pointing to /api/v1/ folder and we haven't decided where to go yet: Connecting to /api/v1/namespaces/default/pods We now move to /api/v1/namespaces/default/pods which where we find information about pods that belong to default namespace (watch=False just means we will not be monitoring for changes, we're just retrieving it as one-off thing): Now, what we've stored in the above variable (ret) is the equivalent of moving to the root directory in our JSON tree (Object) in the output below: The output above was from kubectl get pods command which lists all pods from default namespace and that's equivalent to what we're doing here using Python's Kubernetes Client API. Python shows us similar options about where to go when I type ret. and hit tab: We've got kind, apiVersion, metadata and items as options to move to but items is what we're looking for because it's supposed to store pod's information. So let's move to items and store it in another variable called pods to make it easier to remember that it actually holds all pods' information from default namespace: Listing pods' names We're now in a comfortable place and ready to play! Let's keep pods variable as our placeholder! On Wireshark, pod's name is located in metadata.name: \ So let's create another variable called pods_names to store all pods' names: Listing Pods' status (phase) What we're looking for is in status.phase: Let's store it in status variable in Python: Displaying the output We can now display the output using built-in zip() method: If you want something pretty you can use prettytable:
Rodrigo_Albuque
Jul 08, 2019 Place Technical Articles
3.3KViews
1like
0Comments
Integrating with your IPv6 Kubernetes Cluster
Why important to focus on IPv6 IPv6 was first introduced as a standard in 1995, yet it is only in the last five years that adoption has accelerated due to the growing need to address the limitations of IPv4. In even some countries like Japan, the pool of IPv4 addresses has long been considered exhausted and many utilize NAT architectures to buy time. With the explosion of cloud-based applications and IoT, not even Kubernetes can escape the call for IPv6 support and all the caveats that come with it. For network engineers, the IPv6 is well known. However, one can argue that the container world is still very new to the protocol. Fortunately, this is changing and arguably with more speed now that dual-stack IPv4/IPv6 networking has reached general availability (GA) in Kubernetes v1.23 and single-stack IPv6 alpha support in Istio v1.9. IPv6 changes everything in Kubernetes When deploying IPv6-only clusters, your pod and service subnets will talk to each other on 128-bit addresses from a block of IPs you defined (default /122) and will therefore need NAT64 and DNS64 in place when calling to IPv4 only services such as DockerHub, GitHub, and other package libraries sitting internally or on the Internet. This also applies to libraries required when installing pre-requisites for K8S. Your CNI configuration, for example Calico, will also need specific variables set to enable IPv6. https://projectcalico.docs.tigera.io/networking/ipv6 Once you have your cluster set up, you can then focus on all the hard parts of IPv6 integration. For example, your BGP routers will need to ensure proper configuration for IPv6 peers, and everything will need to allow neighbor discovery (or NDP) to succeed or else you will fail to achieve the simplest of tests — the ICMP ping. All your containerized applications will also need to support communicating on IPv6 sockets. This seems to be the biggest challenge with most applications that are still migrating from the IPv4 world. Fortunately, F5 has made the ingress part easier. How can BIG-IP integrate with your IPv6-only cluster for reverse proxy and security services? IPv6 support is available for many features of Container Ingress Services (CIS) as well as F5 IPAM Controller to meet your needs for exposing your K8S workloads in an easy and secure way. BIG-IP admin IP address In versions prior to 2.6.0, you had to define a hostAliases block for your IPv6 BIG-IP URL (--bigip-url=bigip01). hostAliases: - hostnames: - bigip01 ip: aaa1:bbb2:ccc3:ddd4::100 Now you can use either of below formats: #IPv6 URL with non-standard port --bigip-url=https://[aaa1:bbb2:ccc3:ddd4::100]:8443 #IPv6 address --bigip-url=[aaa1:bbb2:ccc3:ddd4::100] VirtualServer, TransportServer CRD When deploying your custom resource, you simply specify an IPv6 address (virtualServerAddress) that will be the external IP exposing your workloads. apiVersion: "cis.f5.com/v1" kind: TransportServer metadata: name: simple-virtual-l4 labels: f5cr: "true" spec: virtualServerAddress: "aaa1:bbb2:ccc3:eee5::200" virtualServerPort: 80 mode: performance snat: auto pool: service: nginx servicePort: 8080 monitor: type: tcp interval: 10 timeout: 10 --- apiVersion: "cis.f5.com/v1" kind: VirtualServer metadata: name: l7-virtual-http labels: f5cr: "true" spec: # This is an insecure virtual, Please use TLSProfile to secure the virtual # check out tls examples to understand more. host: expo.example.com virtualServerAddress: "aaa1:bbb2:ccc3:eee5::201" virtualServerName: "l7-virtual-http" pools: - path: /expo service: svc-2 servicePort: 80 Service Type LoadBalancer (with F5 IPAM Controller) You can also simulate the one-click public cloud experience when exposing your workloads by simply specifying your service to be of type LoadBalancer. The listener IP allocation and configuration is automagically done for you with the help of F5 IPAM Controller. In F5 IPAM Controller deployment manifest, just specify your IPv6 range in the container args like below: - '{"Dev":"2001:db8:3::7-2001:db8:3::9","Test":"2001:db8:4::7-2001:db8:4::9"}' Then deploy your F5 IPAM Controller and if you see addresses in the logs like below, you're good to go. 2022/03/08 15:34:31 [DEBUG] Created New IPAM Client 2022/03/08 15:34:31 [DEBUG] [STORE] Using IPAM DB file from mount path 2022/03/08 15:34:31 [DEBUG] [STORE] [ipaddress status ipam_label reference] 2022/03/08 15:34:31 [DEBUG] [STORE] 2001:db8:4::7 1 Test ZpTSxCKs0gigByk5 2022/03/08 15:34:31 [DEBUG] [STORE] 2001:db8:4::8 1 Test 650YpEeEBF2H88Z8 2022/03/08 15:34:31 [DEBUG] [STORE] 2001:db8:4::9 1 Test la9aJTZ5Ubqi/2zU Note: For the CIS configuration, ensure that your CIS deployment manifest defines below parameters, which are particularly for service type LoadBalancer and IPv6 use case. "--custom-resource-mode=true", "--ipam=true", "--enable-ipv6=true", Then you can deploy a simple Service resource like below: apiVersion: v1 kind: Service metadata: annotations: cis.f5.com/health: '{"interval": 5, "timeout": 10}' cis.f5.com/ipamLabel: Test labels: app: svc-f5-demo-lb1 name: svc-f5-demo-lb1 namespace: default spec: ports: - name: svc-lb1-80 port: 80 protocol: TCP targetPort: 80 selector: app: f5-demo type: LoadBalancer At which point the magic happens and you see that your "EXTERNAL-IP" is populated with an IP address allocated by F5 IPAM Controller. # kubectl get svc svc-f5-demo-lb1 NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE svc-f5-demo-lb1 LoadBalancer 172.19.2.55 2001:db8:4::4 80:32410/TCP 43m On the BIG-IP you will see the virtual server created as defined in your Service object. Summary Moving to a single-stack IPv6 Kubernetes cluster can be difficult and requires a thorough review of all your application components as well as the parts that integrate with your cluster. F5 has ensured that even with a pure IPv6 environment you are covered with an enterprise grade ingress solution that not only simplifies exposing of your workloads, but also provides many security options as well.
Terence_Kam
Mar 17, 2022 Place Technical Articles
3.1KViews
2likes
0Comments

"}},"componentScriptGroups({\"componentId\":\"custom.widget.Beta_MetaNav\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"component({\"componentId\":\"custom.widget.Beta_Footer\"})":{"__typename":"Component","render({\"context\":{\"component\":{\"entities\":[],\"props\":{}},\"page\":{\"entities\":[],\"name\":\"TagPage\",\"props\":{},\"url\":\"https://community.f5.com/tag/kubernetes\"}}})":{"__typename":"ComponentRenderResult","html":"

Trademarks Policies Privacy California Privacy Do Not Sell My Personal Information

"}},"componentScriptGroups({\"componentId\":\"custom.widget.Beta_Footer\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"component({\"componentId\":\"custom.widget.Tag_Manager_Helper\"})":{"__typename":"Component","render({\"context\":{\"component\":{\"entities\":[],\"props\":{}},\"page\":{\"entities\":[],\"name\":\"TagPage\",\"props\":{},\"url\":\"https://community.f5.com/tag/kubernetes\"}}})":{"__typename":"ComponentRenderResult","html":" "}},"componentScriptGroups({\"componentId\":\"custom.widget.Tag_Manager_Helper\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"component({\"componentId\":\"custom.widget.Consent_Blackbar\"})":{"__typename":"Component","render({\"context\":{\"component\":{\"entities\":[],\"props\":{}},\"page\":{\"entities\":[],\"name\":\"TagPage\",\"props\":{},\"url\":\"https://community.f5.com/tag/kubernetes\"}}})":{"__typename":"ComponentRenderResult","html":"

"}},"componentScriptGroups({\"componentId\":\"custom.widget.Consent_Blackbar\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/community/NavbarDropdownToggle\"]})":[{"__ref":"CachedAsset:text:en_US-components/community/NavbarDropdownToggle-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageListTabs\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageListTabs-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageView/MessageViewInline\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageView/MessageViewInline-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/common/Pager/PagerLoadMore\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/common/Pager/PagerLoadMore-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/customComponent/CustomComponent\"]})":[{"__ref":"CachedAsset:text:en_US-components/customComponent/CustomComponent-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/common/OverflowNav\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/common/OverflowNav-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/users/UserLink\"]})":[{"__ref":"CachedAsset:text:en_US-components/users/UserLink-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageSubject\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageSubject-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageBody\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageBody-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageTime\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageTime-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/nodes/NodeIcon\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/nodes/NodeIcon-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageUnreadCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageUnreadCount-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageViewCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageViewCount-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/kudos/KudosCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/kudos/KudosCount-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageRepliesCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageRepliesCount-1744046271000"}],"cachedText({\"lastModified\":\"1744046271000\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/users/UserAvatar\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/users/UserAvatar-1744046271000"}]},"Theme:customTheme1":{"__typename":"Theme","id":"customTheme1"},"User:user:-1":{"__typename":"User","id":"user:-1","uid":-1,"login":"Former Member","email":"","avatar":null,"rank":null,"kudosWeight":1,"registrationData":{"__typename":"RegistrationData","status":"ANONYMOUS","registrationTime":null,"confirmEmailStatus":false,"registrationAccessLevel":"VIEW","ssoRegistrationFields":[]},"ssoId":null,"profileSettings":{"__typename":"ProfileSettings","dateDisplayStyle":{"__typename":"InheritableStringSettingWithPossibleValues","key":"layout.friendly_dates_enabled","value":"false","localValue":"true","possibleValues":["true","false"]},"dateDisplayFormat":{"__typename":"InheritableStringSetting","key":"layout.format_pattern_date","value":"dd-MMM-yyyy","localValue":"MM-dd-yyyy"},"language":{"__typename":"InheritableStringSettingWithPossibleValues","key":"profile.language","value":"en-US","localValue":null,"possibleValues":["en-US","de-DE"]},"repliesSortOrder":{"__typename":"InheritableStringSettingWithPossibleValues","key":"config.user_replies_sort_order","value":"DEFAULT","localValue":"DEFAULT","possibleValues":["DEFAULT","LIKES","PUBLISH_TIME","REVERSE_PUBLISH_TIME"]}},"deleted":false},"CachedAsset:pages-1746693049844":{"__typename":"CachedAsset","id":"pages-1746693049844","value":[{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetInvolved.MvpProgram","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved/mvp-program","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"BlogViewAllPostsPage","type":"BLOG","urlPath":"/category/:categoryId/blog/:boardId/all-posts/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"CasePortalPage","type":"CASE_PORTAL","urlPath":"/caseportal","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"CreateGroupHubPage","type":"GROUP_HUB","urlPath":"/groups/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"CaseViewPage","type":"CASE_DETAILS","urlPath":"/case/:caseId/:caseNumber","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"InboxPage","type":"COMMUNITY","urlPath":"/inbox","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetInvolved.AdvocacyProgram","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved/advocacy-program","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetHelp.NonCustomer","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/non-customer","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HelpFAQPage","type":"COMMUNITY","urlPath":"/help","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetHelp.F5Customer","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/f5-customer","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"IdeaMessagePage","type":"IDEA_POST","urlPath":"/idea/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"IdeaViewAllIdeasPage","type":"IDEA","urlPath":"/category/:categoryId/ideas/:boardId/all-ideas/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"LoginPage","type":"USER","urlPath":"/signin","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"BlogPostPage","type":"BLOG","urlPath":"/category/:categoryId/blogs/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetInvolved","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.Learn","type":"COMMUNITY","urlPath":"/c/how-do-i/learn","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1739501996000,"localOverride":null,"page":{"id":"Test","type":"CUSTOM","urlPath":"/custom-test-2","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ThemeEditorPage","type":"COMMUNITY","urlPath":"/designer/themes","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"TkbViewAllArticlesPage","type":"TKB","urlPath":"/category/:categoryId/kb/:boardId/all-articles/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"OccasionEditPage","type":"EVENT","urlPath":"/event/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"OAuthAuthorizationAllowPage","type":"USER","urlPath":"/auth/authorize/allow","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"PageEditorPage","type":"COMMUNITY","urlPath":"/designer/pages","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"PostPage","type":"COMMUNITY","urlPath":"/category/:categoryId/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ForumBoardPage","type":"FORUM","urlPath":"/category/:categoryId/discussions/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"TkbBoardPage","type":"TKB","urlPath":"/category/:categoryId/kb/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"EventPostPage","type":"EVENT","urlPath":"/category/:categoryId/events/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"UserBadgesPage","type":"COMMUNITY","urlPath":"/users/:login/:userId/badges","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"GroupHubMembershipAction","type":"GROUP_HUB","urlPath":"/membership/join/:nodeId/:membershipType","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"MaintenancePage","type":"COMMUNITY","urlPath":"/maintenance","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"IdeaReplyPage","type":"IDEA_REPLY","urlPath":"/idea/:boardId/:messageSubject/:messageId/comments/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"UserSettingsPage","type":"USER","urlPath":"/mysettings/:userSettingsTab","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"GroupHubsPage","type":"GROUP_HUB","urlPath":"/groups","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ForumPostPage","type":"FORUM","urlPath":"/category/:categoryId/discussions/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"OccasionRsvpActionPage","type":"OCCASION","urlPath":"/event/:boardId/:messageSubject/:messageId/rsvp/:responseType","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"VerifyUserEmailPage","type":"USER","urlPath":"/verifyemail/:userId/:verifyEmailToken","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"AllOccasionsPage","type":"OCCASION","urlPath":"/category/:categoryId/events/:boardId/all-events/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"EventBoardPage","type":"EVENT","urlPath":"/category/:categoryId/events/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"TkbReplyPage","type":"TKB_REPLY","urlPath":"/kb/:boardId/:messageSubject/:messageId/comments/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"IdeaBoardPage","type":"IDEA","urlPath":"/category/:categoryId/ideas/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"CommunityGuideLinesPage","type":"COMMUNITY","urlPath":"/communityguidelines","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"CaseCreatePage","type":"SALESFORCE_CASE_CREATION","urlPath":"/caseportal/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"TkbEditPage","type":"TKB","urlPath":"/kb/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ForgotPasswordPage","type":"USER","urlPath":"/forgotpassword","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"IdeaEditPage","type":"IDEA","urlPath":"/idea/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"TagPage","type":"COMMUNITY","urlPath":"/tag/:tagName","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"BlogBoardPage","type":"BLOG","urlPath":"/category/:categoryId/blog/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"OccasionMessagePage","type":"OCCASION_TOPIC","urlPath":"/event/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ManageContentPage","type":"COMMUNITY","urlPath":"/managecontent","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ClosedMembershipNodeNonMembersPage","type":"GROUP_HUB","urlPath":"/closedgroup/:groupHubId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetHelp.Community","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/community","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"CommunityPage","type":"COMMUNITY","urlPath":"/","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetInvolved.ContributeCode","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved/contribute-code","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ForumMessagePage","type":"FORUM_TOPIC","urlPath":"/discussions/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"IdeaPostPage","type":"IDEA","urlPath":"/category/:categoryId/ideas/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"BlogMessagePage","type":"BLOG_ARTICLE","urlPath":"/blog/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"RegistrationPage","type":"USER","urlPath":"/register","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"EditGroupHubPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ForumEditPage","type":"FORUM","urlPath":"/discussions/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ResetPasswordPage","type":"USER","urlPath":"/resetpassword/:userId/:resetPasswordToken","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"TkbMessagePage","type":"TKB_ARTICLE","urlPath":"/kb/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.Learn.AboutIrules","type":"COMMUNITY","urlPath":"/c/how-do-i/learn/about-irules","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"BlogEditPage","type":"BLOG","urlPath":"/blog/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetHelp.F5Support","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/f5-support","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ManageUsersPage","type":"USER","urlPath":"/users/manage/:tab?/:manageUsersTab?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ForumReplyPage","type":"FORUM_REPLY","urlPath":"/discussions/:boardId/:messageSubject/:messageId/replies/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"PrivacyPolicyPage","type":"COMMUNITY","urlPath":"/privacypolicy","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"NotificationPage","type":"COMMUNITY","urlPath":"/notifications","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"UserPage","type":"USER","urlPath":"/users/:login/:userId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HealthCheckPage","type":"COMMUNITY","urlPath":"/health","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"OccasionReplyPage","type":"OCCASION_REPLY","urlPath":"/event/:boardId/:messageSubject/:messageId/comments/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ManageMembersPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId/manage/:tab?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"SearchResultsPage","type":"COMMUNITY","urlPath":"/search","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"BlogReplyPage","type":"BLOG_REPLY","urlPath":"/blog/:boardId/:messageSubject/:messageId/replies/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"GroupHubPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"TermsOfServicePage","type":"COMMUNITY","urlPath":"/termsofservice","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetHelp","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI.GetHelp.SecurityIncident","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/security-incident","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"CategoryPage","type":"CATEGORY","urlPath":"/category/:categoryId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"ForumViewAllTopicsPage","type":"FORUM","urlPath":"/category/:categoryId/discussions/:boardId/all-topics/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"TkbPostPage","type":"TKB","urlPath":"/category/:categoryId/kbs/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"GroupHubPostPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1746693049844,"localOverride":null,"page":{"id":"HowDoI","type":"COMMUNITY","urlPath":"/c/how-do-i","__typename":"PageDescriptor"},"__typename":"PageResource"}],"localOverride":false},"CachedAsset:text:en_US-components/context/AppContext/AppContextProvider-0":{"__typename":"CachedAsset","id":"text:en_US-components/context/AppContext/AppContextProvider-0","value":{"noCommunity":"Cannot find community","noUser":"Cannot find current user","noNode":"Cannot find node with id {nodeId}","noMessage":"Cannot find message with id {messageId}","userBanned":"We're sorry, but you have been banned from using this site.","userBannedReason":"You have been banned for the following reason: {reason}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/Loading/LoadingDot-0":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/Loading/LoadingDot-0","value":{"title":"Loading..."},"localOverride":false},"CachedAsset:theme:customTheme1-1746693049182":{"__typename":"CachedAsset","id":"theme:customTheme1-1746693049182","value":{"id":"customTheme1","animation":{"fast":"150ms","normal":"250ms","slow":"500ms","slowest":"750ms","function":"cubic-bezier(0.07, 0.91, 0.51, 1)","__typename":"AnimationThemeSettings"},"avatar":{"borderRadius":"50%","collections":["custom"],"__typename":"AvatarThemeSettings"},"basics":{"browserIcon":{"imageAssetName":"JimmyPackets-512-1702592938213.png","imageLastModified":"1702592945815","__typename":"ThemeAsset"},"customerLogo":{"imageAssetName":"f5_logo_fix-1704824537976.svg","imageLastModified":"1704824540697","__typename":"ThemeAsset"},"maximumWidthOfPageContent":"1600px","oneColumnNarrowWidth":"800px","gridGutterWidthMd":"30px","gridGutterWidthXs":"10px","pageWidthStyle":"WIDTH_OF_PAGE_CONTENT","__typename":"BasicsThemeSettings"},"buttons":{"borderRadiusSm":"5px","borderRadius":"5px","borderRadiusLg":"5px","paddingY":"5px","paddingYLg":"7px","paddingYHero":"var(--lia-bs-btn-padding-y-lg)","paddingX":"12px","paddingXLg":"14px","paddingXHero":"42px","fontStyle":"NORMAL","fontWeight":"400","textTransform":"NONE","disabledOpacity":0.5,"primaryTextColor":"var(--lia-bs-white)","primaryTextHoverColor":"var(--lia-bs-white)","primaryTextActiveColor":"var(--lia-bs-white)","primaryBgColor":"var(--lia-bs-primary)","primaryBgHoverColor":"hsl(var(--lia-bs-primary-h), var(--lia-bs-primary-s), calc(var(--lia-bs-primary-l) * 0.85))","primaryBgActiveColor":"hsl(var(--lia-bs-primary-h), var(--lia-bs-primary-s), calc(var(--lia-bs-primary-l) * 0.7))","primaryBorder":"1px solid transparent","primaryBorderHover":"1px solid transparent","primaryBorderActive":"1px solid transparent","primaryBorderFocus":"1px solid var(--lia-bs-white)","primaryBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","secondaryTextColor":"var(--lia-bs-gray-900)","secondaryTextHoverColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.95))","secondaryTextActiveColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.9))","secondaryBgColor":"var(--lia-bs-gray-400)","secondaryBgHoverColor":"hsl(var(--lia-bs-gray-400-h), var(--lia-bs-gray-400-s), calc(var(--lia-bs-gray-400-l) * 0.96))","secondaryBgActiveColor":"hsl(var(--lia-bs-gray-400-h), var(--lia-bs-gray-400-s), calc(var(--lia-bs-gray-400-l) * 0.92))","secondaryBorder":"1px solid transparent","secondaryBorderHover":"1px solid transparent","secondaryBorderActive":"1px solid transparent","secondaryBorderFocus":"1px solid transparent","secondaryBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","tertiaryTextColor":"var(--lia-bs-gray-900)","tertiaryTextHoverColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.95))","tertiaryTextActiveColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.9))","tertiaryBgColor":"transparent","tertiaryBgHoverColor":"transparent","tertiaryBgActiveColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.04)","tertiaryBorder":"1px solid transparent","tertiaryBorderHover":"1px solid hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","tertiaryBorderActive":"1px solid transparent","tertiaryBorderFocus":"1px solid transparent","tertiaryBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","destructiveTextColor":"var(--lia-bs-danger)","destructiveTextHoverColor":"hsl(var(--lia-bs-danger-h), var(--lia-bs-danger-s), calc(var(--lia-bs-danger-l) * 0.95))","destructiveTextActiveColor":"hsl(var(--lia-bs-danger-h), var(--lia-bs-danger-s), calc(var(--lia-bs-danger-l) * 0.9))","destructiveBgColor":"var(--lia-bs-gray-300)","destructiveBgHoverColor":"hsl(var(--lia-bs-gray-300-h), var(--lia-bs-gray-300-s), calc(var(--lia-bs-gray-300-l) * 0.96))","destructiveBgActiveColor":"hsl(var(--lia-bs-gray-300-h), var(--lia-bs-gray-300-s), calc(var(--lia-bs-gray-300-l) * 0.92))","destructiveBorder":"1px solid transparent","destructiveBorderHover":"1px solid transparent","destructiveBorderActive":"1px solid transparent","destructiveBorderFocus":"1px solid transparent","destructiveBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","__typename":"ButtonsThemeSettings"},"border":{"color":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","mainContent":"NONE","sideContent":"NONE","radiusSm":"3px","radius":"5px","radiusLg":"9px","radius50":"100vw","__typename":"BorderThemeSettings"},"boxShadow":{"xs":"0 0 0 1px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.08), 0 3px 0 -1px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.08)","sm":"0 2px 4px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.06)","md":"0 5px 15px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.15)","lg":"0 10px 30px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.15)","__typename":"BoxShadowThemeSettings"},"cards":{"bgColor":"var(--lia-panel-bg-color)","borderRadius":"var(--lia-panel-border-radius)","boxShadow":"var(--lia-box-shadow-xs)","__typename":"CardsThemeSettings"},"chip":{"maxWidth":"300px","height":"30px","__typename":"ChipThemeSettings"},"coreTypes":{"defaultMessageLinkColor":"var(--lia-bs-primary)","defaultMessageLinkDecoration":"none","defaultMessageLinkFontStyle":"NORMAL","defaultMessageLinkFontWeight":"400","defaultMessageFontStyle":"NORMAL","defaultMessageFontWeight":"400","defaultMessageFontFamily":"var(--lia-bs-font-family-base)","forumColor":"#0C5C8D","forumFontFamily":"var(--lia-bs-font-family-base)","forumFontWeight":"var(--lia-default-message-font-weight)","forumLineHeight":"var(--lia-bs-line-height-base)","forumFontStyle":"var(--lia-default-message-font-style)","forumMessageLinkColor":"var(--lia-default-message-link-color)","forumMessageLinkDecoration":"var(--lia-default-message-link-decoration)","forumMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","forumMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","forumSolvedColor":"#62C026","blogColor":"#730015","blogFontFamily":"var(--lia-bs-font-family-base)","blogFontWeight":"var(--lia-default-message-font-weight)","blogLineHeight":"1.75","blogFontStyle":"var(--lia-default-message-font-style)","blogMessageLinkColor":"var(--lia-default-message-link-color)","blogMessageLinkDecoration":"var(--lia-default-message-link-decoration)","blogMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","blogMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","tkbColor":"#C20025","tkbFontFamily":"var(--lia-bs-font-family-base)","tkbFontWeight":"var(--lia-default-message-font-weight)","tkbLineHeight":"1.75","tkbFontStyle":"var(--lia-default-message-font-style)","tkbMessageLinkColor":"var(--lia-default-message-link-color)","tkbMessageLinkDecoration":"var(--lia-default-message-link-decoration)","tkbMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","tkbMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","qandaColor":"#4099E2","qandaFontFamily":"var(--lia-bs-font-family-base)","qandaFontWeight":"var(--lia-default-message-font-weight)","qandaLineHeight":"var(--lia-bs-line-height-base)","qandaFontStyle":"var(--lia-default-message-link-font-style)","qandaMessageLinkColor":"var(--lia-default-message-link-color)","qandaMessageLinkDecoration":"var(--lia-default-message-link-decoration)","qandaMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","qandaMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","qandaSolvedColor":"#3FA023","ideaColor":"#F3704B","ideaFontFamily":"var(--lia-bs-font-family-base)","ideaFontWeight":"var(--lia-default-message-font-weight)","ideaLineHeight":"var(--lia-bs-line-height-base)","ideaFontStyle":"var(--lia-default-message-font-style)","ideaMessageLinkColor":"var(--lia-default-message-link-color)","ideaMessageLinkDecoration":"var(--lia-default-message-link-decoration)","ideaMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","ideaMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","contestColor":"#FCC845","contestFontFamily":"var(--lia-bs-font-family-base)","contestFontWeight":"var(--lia-default-message-font-weight)","contestLineHeight":"var(--lia-bs-line-height-base)","contestFontStyle":"var(--lia-default-message-link-font-style)","contestMessageLinkColor":"var(--lia-default-message-link-color)","contestMessageLinkDecoration":"var(--lia-default-message-link-decoration)","contestMessageLinkFontStyle":"ITALIC","contestMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","occasionColor":"#EE4B5B","occasionFontFamily":"var(--lia-bs-font-family-base)","occasionFontWeight":"var(--lia-default-message-font-weight)","occasionLineHeight":"var(--lia-bs-line-height-base)","occasionFontStyle":"var(--lia-default-message-font-style)","occasionMessageLinkColor":"var(--lia-default-message-link-color)","occasionMessageLinkDecoration":"var(--lia-default-message-link-decoration)","occasionMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","occasionMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","grouphubColor":"#491B62","categoryColor":"#949494","communityColor":"#FFFFFF","productColor":"#949494","__typename":"CoreTypesThemeSettings"},"colors":{"black":"#000000","white":"#FFFFFF","gray100":"#F7F7F7","gray200":"#F7F7F7","gray300":"#E8E8E8","gray400":"#D9D9D9","gray500":"#CCCCCC","gray600":"#949494","gray700":"#707070","gray800":"#545454","gray900":"#333333","dark":"#545454","light":"#F7F7F7","primary":"#0C5C8D","secondary":"#333333","bodyText":"#222222","bodyBg":"#F5F5F5","info":"#1D9CD3","success":"#62C026","warning":"#FFD651","danger":"#C20025","alertSystem":"#FF6600","textMuted":"#707070","highlight":"#FFFCAD","outline":"var(--lia-bs-primary)","custom":["#C20025","#081B85","#009639","#B3C6D7","#7CC0EB","#F29A36"],"__typename":"ColorsThemeSettings"},"divider":{"size":"3px","marginLeft":"4px","marginRight":"4px","borderRadius":"50%","bgColor":"var(--lia-bs-gray-600)","bgColorActive":"var(--lia-bs-gray-600)","__typename":"DividerThemeSettings"},"dropdown":{"fontSize":"var(--lia-bs-font-size-sm)","borderColor":"var(--lia-bs-border-color)","borderRadius":"var(--lia-bs-border-radius-sm)","dividerBg":"var(--lia-bs-gray-300)","itemPaddingY":"5px","itemPaddingX":"20px","headerColor":"var(--lia-bs-gray-700)","__typename":"DropdownThemeSettings"},"email":{"link":{"color":"#0069D4","hoverColor":"#0061c2","decoration":"none","hoverDecoration":"underline","__typename":"EmailLinkSettings"},"border":{"color":"#e4e4e4","__typename":"EmailBorderSettings"},"buttons":{"borderRadiusLg":"5px","paddingXLg":"16px","paddingYLg":"7px","fontWeight":"700","primaryTextColor":"#ffffff","primaryTextHoverColor":"#ffffff","primaryBgColor":"#0069D4","primaryBgHoverColor":"#005cb8","primaryBorder":"1px solid transparent","primaryBorderHover":"1px solid transparent","__typename":"EmailButtonsSettings"},"panel":{"borderRadius":"5px","borderColor":"#e4e4e4","__typename":"EmailPanelSettings"},"__typename":"EmailThemeSettings"},"emoji":{"skinToneDefault":"#ffcd43","skinToneLight":"#fae3c5","skinToneMediumLight":"#e2cfa5","skinToneMedium":"#daa478","skinToneMediumDark":"#a78058","skinToneDark":"#5e4d43","__typename":"EmojiThemeSettings"},"heading":{"color":"var(--lia-bs-body-color)","fontFamily":"Inter","fontStyle":"NORMAL","fontWeight":"600","h1FontSize":"30px","h2FontSize":"25px","h3FontSize":"20px","h4FontSize":"18px","h5FontSize":"16px","h6FontSize":"16px","lineHeight":"1.2","subHeaderFontSize":"11px","subHeaderFontWeight":"500","h1LetterSpacing":"normal","h2LetterSpacing":"normal","h3LetterSpacing":"normal","h4LetterSpacing":"normal","h5LetterSpacing":"normal","h6LetterSpacing":"normal","subHeaderLetterSpacing":"2px","h1FontWeight":"var(--lia-bs-headings-font-weight)","h2FontWeight":"var(--lia-bs-headings-font-weight)","h3FontWeight":"var(--lia-bs-headings-font-weight)","h4FontWeight":"var(--lia-bs-headings-font-weight)","h5FontWeight":"var(--lia-bs-headings-font-weight)","h6FontWeight":"var(--lia-bs-headings-font-weight)","__typename":"HeadingThemeSettings"},"icons":{"size10":"10px","size12":"12px","size14":"14px","size16":"16px","size20":"20px","size24":"24px","size30":"30px","size40":"40px","size50":"50px","size60":"60px","size80":"80px","size120":"120px","size160":"160px","__typename":"IconsThemeSettings"},"imagePreview":{"bgColor":"var(--lia-bs-gray-900)","titleColor":"var(--lia-bs-white)","controlColor":"var(--lia-bs-white)","controlBgColor":"var(--lia-bs-gray-800)","__typename":"ImagePreviewThemeSettings"},"input":{"borderColor":"var(--lia-bs-gray-600)","disabledColor":"var(--lia-bs-gray-600)","focusBorderColor":"var(--lia-bs-primary)","labelMarginBottom":"10px","btnFontSize":"var(--lia-bs-font-size-sm)","focusBoxShadow":"0 0 0 3px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","checkLabelMarginBottom":"2px","checkboxBorderRadius":"3px","borderRadiusSm":"var(--lia-bs-border-radius-sm)","borderRadius":"var(--lia-bs-border-radius)","borderRadiusLg":"var(--lia-bs-border-radius-lg)","formTextMarginTop":"4px","textAreaBorderRadius":"var(--lia-bs-border-radius)","activeFillColor":"var(--lia-bs-primary)","__typename":"InputThemeSettings"},"loading":{"dotDarkColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.2)","dotLightColor":"hsla(var(--lia-bs-white-h), var(--lia-bs-white-s), var(--lia-bs-white-l), 0.5)","barDarkColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.06)","barLightColor":"hsla(var(--lia-bs-white-h), var(--lia-bs-white-s), var(--lia-bs-white-l), 0.4)","__typename":"LoadingThemeSettings"},"link":{"color":"var(--lia-bs-primary)","hoverColor":"hsl(var(--lia-bs-primary-h), var(--lia-bs-primary-s), calc(var(--lia-bs-primary-l) - 10%))","decoration":"none","hoverDecoration":"underline","__typename":"LinkThemeSettings"},"listGroup":{"itemPaddingY":"15px","itemPaddingX":"15px","borderColor":"var(--lia-bs-gray-300)","__typename":"ListGroupThemeSettings"},"modal":{"contentTextColor":"var(--lia-bs-body-color)","contentBg":"var(--lia-bs-white)","backgroundBg":"var(--lia-bs-black)","smSize":"440px","mdSize":"760px","lgSize":"1080px","backdropOpacity":0.3,"contentBoxShadowXs":"var(--lia-bs-box-shadow-sm)","contentBoxShadow":"var(--lia-bs-box-shadow)","headerFontWeight":"700","__typename":"ModalThemeSettings"},"navbar":{"position":"FIXED","background":{"attachment":null,"clip":null,"color":"var(--lia-bs-white)","imageAssetName":null,"imageLastModified":"0","origin":null,"position":"CENTER_CENTER","repeat":"NO_REPEAT","size":"COVER","__typename":"BackgroundProps"},"backgroundOpacity":0.8,"paddingTop":"15px","paddingBottom":"15px","borderBottom":"1px solid var(--lia-bs-border-color)","boxShadow":"var(--lia-bs-box-shadow-sm)","brandMarginRight":"30px","brandMarginRightSm":"10px","brandLogoHeight":"30px","linkGap":"10px","linkJustifyContent":"flex-start","linkPaddingY":"5px","linkPaddingX":"10px","linkDropdownPaddingY":"9px","linkDropdownPaddingX":"var(--lia-nav-link-px)","linkColor":"var(--lia-bs-body-color)","linkHoverColor":"var(--lia-bs-primary)","linkFontSize":"var(--lia-bs-font-size-sm)","linkFontStyle":"NORMAL","linkFontWeight":"400","linkTextTransform":"NONE","linkLetterSpacing":"normal","linkBorderRadius":"var(--lia-bs-border-radius-sm)","linkBgColor":"transparent","linkBgHoverColor":"transparent","linkBorder":"none","linkBorderHover":"none","linkBoxShadow":"none","linkBoxShadowHover":"none","linkTextBorderBottom":"none","linkTextBorderBottomHover":"none","dropdownPaddingTop":"10px","dropdownPaddingBottom":"15px","dropdownPaddingX":"10px","dropdownMenuOffset":"2px","dropdownDividerMarginTop":"10px","dropdownDividerMarginBottom":"10px","dropdownBorderColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","controllerBgHoverColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.1)","controllerIconColor":"var(--lia-bs-body-color)","controllerIconHoverColor":"var(--lia-bs-body-color)","controllerTextColor":"var(--lia-nav-controller-icon-color)","controllerTextHoverColor":"var(--lia-nav-controller-icon-hover-color)","controllerHighlightColor":"hsla(30, 100%, 50%)","controllerHighlightTextColor":"var(--lia-yiq-light)","controllerBorderRadius":"var(--lia-border-radius-50)","hamburgerColor":"var(--lia-nav-controller-icon-color)","hamburgerHoverColor":"var(--lia-nav-controller-icon-color)","hamburgerBgColor":"transparent","hamburgerBgHoverColor":"transparent","hamburgerBorder":"none","hamburgerBorderHover":"none","collapseMenuMarginLeft":"20px","collapseMenuDividerBg":"var(--lia-nav-link-color)","collapseMenuDividerOpacity":0.16,"__typename":"NavbarThemeSettings"},"pager":{"textColor":"var(--lia-bs-link-color)","textFontWeight":"var(--lia-font-weight-md)","textFontSize":"var(--lia-bs-font-size-sm)","__typename":"PagerThemeSettings"},"panel":{"bgColor":"var(--lia-bs-white)","borderRadius":"var(--lia-bs-border-radius)","borderColor":"var(--lia-bs-border-color)","boxShadow":"none","__typename":"PanelThemeSettings"},"popover":{"arrowHeight":"8px","arrowWidth":"16px","maxWidth":"300px","minWidth":"100px","headerBg":"var(--lia-bs-white)","borderColor":"var(--lia-bs-border-color)","borderRadius":"var(--lia-bs-border-radius)","boxShadow":"0 0.5rem 1rem hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.15)","__typename":"PopoverThemeSettings"},"prism":{"color":"#000000","bgColor":"#f5f2f0","fontFamily":"var(--font-family-monospace)","fontSize":"var(--lia-bs-font-size-base)","fontWeightBold":"var(--lia-bs-font-weight-bold)","fontStyleItalic":"italic","tabSize":2,"highlightColor":"#b3d4fc","commentColor":"#62707e","punctuationColor":"#6f6f6f","namespaceOpacity":"0.7","propColor":"#990055","selectorColor":"#517a00","operatorColor":"#906736","operatorBgColor":"hsla(0, 0%, 100%, 0.5)","keywordColor":"#0076a9","functionColor":"#d3284b","variableColor":"#c14700","__typename":"PrismThemeSettings"},"rte":{"bgColor":"var(--lia-bs-white)","borderRadius":"var(--lia-panel-border-radius)","boxShadow":" var(--lia-panel-box-shadow)","customColor1":"#bfedd2","customColor2":"#fbeeb8","customColor3":"#f8cac6","customColor4":"#eccafa","customColor5":"#c2e0f4","customColor6":"#2dc26b","customColor7":"#f1c40f","customColor8":"#e03e2d","customColor9":"#b96ad9","customColor10":"#3598db","customColor11":"#169179","customColor12":"#e67e23","customColor13":"#ba372a","customColor14":"#843fa1","customColor15":"#236fa1","customColor16":"#ecf0f1","customColor17":"#ced4d9","customColor18":"#95a5a6","customColor19":"#7e8c8d","customColor20":"#34495e","customColor21":"#000000","customColor22":"#ffffff","defaultMessageHeaderMarginTop":"14px","defaultMessageHeaderMarginBottom":"10px","defaultMessageItemMarginTop":"0","defaultMessageItemMarginBottom":"10px","diffAddedColor":"hsla(170, 53%, 51%, 0.4)","diffChangedColor":"hsla(43, 97%, 63%, 0.4)","diffNoneColor":"hsla(0, 0%, 80%, 0.4)","diffRemovedColor":"hsla(9, 74%, 47%, 0.4)","specialMessageHeaderMarginTop":"14px","specialMessageHeaderMarginBottom":"10px","specialMessageItemMarginTop":"0","specialMessageItemMarginBottom":"10px","__typename":"RteThemeSettings"},"tags":{"bgColor":"var(--lia-bs-gray-200)","bgHoverColor":"var(--lia-bs-gray-400)","borderRadius":"var(--lia-bs-border-radius-sm)","color":"var(--lia-bs-body-color)","hoverColor":"var(--lia-bs-body-color)","fontWeight":"var(--lia-font-weight-md)","fontSize":"var(--lia-font-size-xxs)","textTransform":"UPPERCASE","letterSpacing":"0.5px","__typename":"TagsThemeSettings"},"toasts":{"borderRadius":"var(--lia-bs-border-radius)","paddingX":"12px","__typename":"ToastsThemeSettings"},"typography":{"fontFamilyBase":"Atkinson Hyperlegible","fontStyleBase":"NORMAL","fontWeightBase":"400","fontWeightLight":"300","fontWeightNormal":"400","fontWeightMd":"500","fontWeightBold":"700","letterSpacingSm":"normal","letterSpacingXs":"normal","lineHeightBase":"1.3","fontSizeBase":"15px","fontSizeXxs":"11px","fontSizeXs":"12px","fontSizeSm":"13px","fontSizeLg":"20px","fontSizeXl":"24px","smallFontSize":"14px","customFonts":[],"__typename":"TypographyThemeSettings"},"unstyledListItem":{"marginBottomSm":"5px","marginBottomMd":"10px","marginBottomLg":"15px","marginBottomXl":"20px","marginBottomXxl":"25px","__typename":"UnstyledListItemThemeSettings"},"yiq":{"light":"#ffffff","dark":"#000000","__typename":"YiqThemeSettings"},"colorLightness":{"primaryDark":0.36,"primaryLight":0.74,"primaryLighter":0.89,"primaryLightest":0.95,"infoDark":0.39,"infoLight":0.72,"infoLighter":0.85,"infoLightest":0.93,"successDark":0.24,"successLight":0.62,"successLighter":0.8,"successLightest":0.91,"warningDark":0.39,"warningLight":0.68,"warningLighter":0.84,"warningLightest":0.93,"dangerDark":0.41,"dangerLight":0.72,"dangerLighter":0.89,"dangerLightest":0.95,"__typename":"ColorLightnessThemeSettings"},"localOverride":false,"__typename":"Theme"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/Loading/LoadingDot-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/Loading/LoadingDot-1744046271000","value":{"title":"Loading..."},"localOverride":false},"CachedAsset:text:en_US-components/common/EmailVerification-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/common/EmailVerification-1744046271000","value":{"email.verification.title":"Email Verification Required","email.verification.message.update.email":"To participate in the community, you must first verify your email address. The verification email was sent to {email}. To change your email, visit My Settings.","email.verification.message.resend.email":"To participate in the community, you must first verify your email address. The verification email was sent to {email}. Resend email."},"localOverride":false},"CachedAsset:text:en_US-pages/tags/TagPage-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-pages/tags/TagPage-1744046271000","value":{"tagPageTitle":"Tag:\"{tagName}\" | {communityTitle}","tagPageForNodeTitle":"Tag:\"{tagName}\" in \"{title}\" | {communityTitle}","name":"Tags Page","tag":"Tag: {tagName}"},"localOverride":false},"CachedAsset:quilt:f5.prod:pages/tags/TagPage:community:zihoc95639-1746693046204":{"__typename":"CachedAsset","id":"quilt:f5.prod:pages/tags/TagPage:community:zihoc95639-1746693046204","value":{"id":"TagPage","container":{"id":"Common","headerProps":{"removeComponents":["community.widget.bannerWidget"],"__typename":"QuiltContainerSectionProps"},"items":[{"id":"tag-header-widget","layout":"ONE_COLUMN","bgColor":"var(--lia-bs-white)","showBorder":"BOTTOM","sectionEditLevel":"LOCKED","columnMap":{"main":[{"id":"tags.widget.TagsHeaderWidget","__typename":"QuiltComponent"}],"__typename":"OneSectionColumns"},"__typename":"OneColumnQuiltSection"},{"id":"messages-list-for-tag-widget","layout":"ONE_COLUMN","columnMap":{"main":[{"id":"messages.widget.messageListForNodeByRecentActivityWidget","props":{"viewVariant":{"type":"inline","props":{"useUnreadCount":true,"useViewCount":true,"useAuthorLogin":true,"clampBodyLines":3,"useAvatar":true,"useBoardIcon":false,"useKudosCount":true,"usePreviewMedia":true,"useTags":false,"useNode":true,"useNodeLink":true,"useTextBody":true,"truncateBodyLength":-1,"useBody":true,"useRepliesCount":true,"useSolvedBadge":true,"timeStampType":"conversation.lastPostingActivityTime","useMessageTimeLink":true,"clampSubjectLines":2}},"panelType":"divider","useTitle":false,"hideIfEmpty":false,"pagerVariant":{"type":"loadMore"},"style":"list","showTabs":true,"tabItemMap":{"default":{"mostRecent":true,"mostRecentUserContent":false,"newest":false},"additional":{"mostKudoed":true,"mostViewed":true,"mostReplies":false,"noReplies":false,"noSolutions":false,"solutions":false}}},"__typename":"QuiltComponent"}],"__typename":"OneSectionColumns"},"__typename":"OneColumnQuiltSection"}],"__typename":"QuiltContainer"},"__typename":"Quilt"},"localOverride":false},"CachedAsset:quiltWrapper:f5.prod:Common:1746691379349":{"__typename":"CachedAsset","id":"quiltWrapper:f5.prod:Common:1746691379349","value":{"id":"Common","header":{"backgroundImageProps":{"assetName":"header.jpg","backgroundSize":"COVER","backgroundRepeat":"NO_REPEAT","backgroundPosition":"LEFT_CENTER","lastModified":"1702932449000","__typename":"BackgroundImageProps"},"backgroundColor":"transparent","items":[{"id":"custom.widget.GainsightShared","props":{"widgetVisibility":"signedInOnly","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"custom.widget.Beta_MetaNav","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"community.widget.navbarWidget","props":{"showUserName":false,"showRegisterLink":true,"style":{"boxShadow":"var(--lia-bs-box-shadow-sm)","linkFontWeight":"700","controllerHighlightColor":"hsla(30, 100%, 50%)","dropdownDividerMarginBottom":"10px","hamburgerBorderHover":"none","linkFontSize":"15px","linkBoxShadowHover":"none","backgroundOpacity":0.4,"controllerBorderRadius":"var(--lia-border-radius-50)","hamburgerBgColor":"transparent","linkTextBorderBottom":"none","hamburgerColor":"var(--lia-nav-controller-icon-color)","brandLogoHeight":"48px","linkLetterSpacing":"normal","linkBgHoverColor":"transparent","collapseMenuDividerOpacity":0.16,"paddingBottom":"10px","dropdownPaddingBottom":"15px","dropdownMenuOffset":"2px","hamburgerBgHoverColor":"transparent","borderBottom":"0","hamburgerBorder":"none","dropdownPaddingX":"10px","brandMarginRightSm":"10px","linkBoxShadow":"none","linkJustifyContent":"center","linkColor":"var(--lia-bs-primary)","collapseMenuDividerBg":"var(--lia-nav-link-color)","dropdownPaddingTop":"10px","controllerHighlightTextColor":"var(--lia-yiq-dark)","background":{"imageAssetName":"","color":"var(--lia-bs-white)","size":"COVER","repeat":"NO_REPEAT","position":"CENTER_CENTER","imageLastModified":""},"linkBorderRadius":"var(--lia-bs-border-radius-sm)","linkHoverColor":"var(--lia-bs-primary)","position":"FIXED","linkBorder":"none","linkTextBorderBottomHover":"2px solid #0C5C8D","brandMarginRight":"30px","hamburgerHoverColor":"var(--lia-nav-controller-icon-color)","linkBorderHover":"none","collapseMenuMarginLeft":"20px","linkFontStyle":"NORMAL","linkPaddingX":"10px","paddingTop":"10px","linkPaddingY":"5px","linkTextTransform":"NONE","dropdownBorderColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","controllerBgHoverColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.1)","linkDropdownPaddingX":"var(--lia-nav-link-px)","linkBgColor":"transparent","linkDropdownPaddingY":"9px","controllerIconColor":"#0C5C8D","dropdownDividerMarginTop":"10px","linkGap":"10px","controllerIconHoverColor":"#0C5C8D"},"links":{"sideLinks":[],"mainLinks":[{"children":[{"linkType":"INTERNAL","id":"migrated-link-1","params":{"boardId":"TechnicalForum","categoryId":"Forums"},"routeName":"ForumBoardPage"},{"linkType":"INTERNAL","id":"migrated-link-2","params":{"boardId":"WaterCooler","categoryId":"Forums"},"routeName":"ForumBoardPage"}],"linkType":"INTERNAL","id":"migrated-link-0","params":{"categoryId":"Forums"},"routeName":"CategoryPage"},{"children":[{"linkType":"INTERNAL","id":"migrated-link-4","params":{"boardId":"codeshare","categoryId":"CrowdSRC"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"migrated-link-5","params":{"boardId":"communityarticles","categoryId":"CrowdSRC"},"routeName":"TkbBoardPage"}],"linkType":"INTERNAL","id":"migrated-link-3","params":{"categoryId":"CrowdSRC"},"routeName":"CategoryPage"},{"children":[{"linkType":"INTERNAL","id":"migrated-link-7","params":{"boardId":"TechnicalArticles","categoryId":"Articles"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"article-series","params":{"boardId":"article-series","categoryId":"Articles"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"security-insights","params":{"boardId":"security-insights","categoryId":"Articles"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"migrated-link-8","params":{"boardId":"DevCentralNews","categoryId":"Articles"},"routeName":"TkbBoardPage"}],"linkType":"INTERNAL","id":"migrated-link-6","params":{"categoryId":"Articles"},"routeName":"CategoryPage"},{"children":[{"linkType":"INTERNAL","id":"migrated-link-10","params":{"categoryId":"CommunityGroups"},"routeName":"CategoryPage"},{"linkType":"INTERNAL","id":"migrated-link-11","params":{"categoryId":"F5-Groups"},"routeName":"CategoryPage"}],"linkType":"INTERNAL","id":"migrated-link-9","params":{"categoryId":"GroupsCategory"},"routeName":"CategoryPage"},{"children":[],"linkType":"INTERNAL","id":"migrated-link-12","params":{"boardId":"Events","categoryId":"top"},"routeName":"EventBoardPage"},{"children":[],"linkType":"INTERNAL","id":"migrated-link-13","params":{"boardId":"Suggestions","categoryId":"top"},"routeName":"IdeaBoardPage"},{"children":[],"linkType":"EXTERNAL","id":"Common-external-link","url":"https://community.f5.com/c/how-do-i","target":"SELF"}]},"className":"QuiltComponent_lia-component-edit-mode__lQ9Z6","showSearchIcon":false},"__typename":"QuiltComponent"},{"id":"community.widget.bannerWidget","props":{"backgroundColor":"transparent","visualEffects":{"showBottomBorder":false},"backgroundImageProps":{"backgroundSize":"COVER","backgroundPosition":"CENTER_CENTER","backgroundRepeat":"NO_REPEAT"},"fontColor":"#222222"},"__typename":"QuiltComponent"},{"id":"community.widget.breadcrumbWidget","props":{"backgroundColor":"var(--lia-bs-primary)","linkHighlightColor":"#FFFFFF","visualEffects":{"showBottomBorder":false},"backgroundOpacity":60,"linkTextColor":"#FFFFFF"},"__typename":"QuiltComponent"}],"__typename":"QuiltWrapperSection"},"footer":{"backgroundImageProps":{"assetName":null,"backgroundSize":"COVER","backgroundRepeat":"NO_REPEAT","backgroundPosition":"CENTER_CENTER","lastModified":null,"__typename":"BackgroundImageProps"},"backgroundColor":"var(--lia-bs-body-color)","items":[{"id":"custom.widget.Beta_Footer","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"custom.widget.Tag_Manager_Helper","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"custom.widget.Consent_Blackbar","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"}],"__typename":"QuiltWrapperSection"},"__typename":"QuiltWrapper","localOverride":false},"localOverride":false},"CachedAsset:text:en_US-components/common/ActionFeedback-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/common/ActionFeedback-1744046271000","value":{"joinedGroupHub.title":"Welcome","joinedGroupHub.message":"You are now a member of this group and are subscribed to updates.","groupHubInviteNotFound.title":"Invitation Not Found","groupHubInviteNotFound.message":"Sorry, we could not find your invitation to the group. The owner may have canceled the invite.","groupHubNotFound.title":"Group Not Found","groupHubNotFound.message":"The grouphub you tried to join does not exist. It may have been deleted.","existingGroupHubMember.title":"Already Joined","existingGroupHubMember.message":"You are already a member of this group.","accountLocked.title":"Account Locked","accountLocked.message":"Your account has been locked due to multiple failed attempts. Try again in {lockoutTime} minutes.","editedGroupHub.title":"Changes Saved","editedGroupHub.message":"Your group has been updated.","leftGroupHub.title":"Goodbye","leftGroupHub.message":"You are no longer a member of this group and will not receive future updates.","deletedGroupHub.title":"Deleted","deletedGroupHub.message":"The group has been deleted.","groupHubCreated.title":"Group Created","groupHubCreated.message":"{groupHubName} is ready to use","accountClosed.title":"Account Closed","accountClosed.message":"The account has been closed and you will now be redirected to the homepage","resetTokenExpired.title":"Reset Password Link has Expired","resetTokenExpired.message":"Try resetting your password again","invalidUrl.title":"Invalid URL","invalidUrl.message":"The URL you're using is not recognized. Verify your URL and try again.","accountClosedForUser.title":"Account Closed","accountClosedForUser.message":"{userName}'s account is closed","inviteTokenInvalid.title":"Invitation Invalid","inviteTokenInvalid.message":"Your invitation to the community has been canceled or expired.","inviteTokenError.title":"Invitation Verification Failed","inviteTokenError.message":"The url you are utilizing is not recognized. Verify your URL and try again","pageNotFound.title":"Access Denied","pageNotFound.message":"You do not have access to this area of the community or it doesn't exist","eventAttending.title":"Responded as Attending","eventAttending.message":"You'll be notified when there's new activity and reminded as the event approaches","eventInterested.title":"Responded as Interested","eventInterested.message":"You'll be notified when there's new activity and reminded as the event approaches","eventNotFound.title":"Event Not Found","eventNotFound.message":"The event you tried to respond to does not exist.","redirectToRelatedPage.title":"Showing Related Content","redirectToRelatedPageForBaseUsers.title":"Showing Related Content","redirectToRelatedPageForBaseUsers.message":"The content you are trying to access is archived","redirectToRelatedPage.message":"The content you are trying to access is archived","relatedUrl.archivalLink.flyoutMessage":"The content you are trying to access is archived View Archived Content"},"localOverride":false},"CachedAsset:component:custom.widget.GainsightShared-en-us-1746693083847":{"__typename":"CachedAsset","id":"component:custom.widget.GainsightShared-en-us-1746693083847","value":{"component":{"id":"custom.widget.GainsightShared","template":{"id":"GainsightShared","markupLanguage":"HTML","style":null,"texts":{},"defaults":{"config":{"applicablePages":[],"description":"Shared functions for Gainsight integration","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.GainsightShared","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"TEXTHTML","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"Shared functions for Gainsight integration","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Beta_MetaNav-en-us-1746693083847":{"__typename":"CachedAsset","id":"component:custom.widget.Beta_MetaNav-en-us-1746693083847","value":{"component":{"id":"custom.widget.Beta_MetaNav","template":{"id":"Beta_MetaNav","markupLanguage":"HANDLEBARS","style":null,"texts":{},"defaults":{"config":{"applicablePages":[],"description":"MetaNav menu at the top of every page.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Beta_MetaNav","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"CUSTOM","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"MetaNav menu at the top of every page.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Beta_Footer-en-us-1746693083847":{"__typename":"CachedAsset","id":"component:custom.widget.Beta_Footer-en-us-1746693083847","value":{"component":{"id":"custom.widget.Beta_Footer","template":{"id":"Beta_Footer","markupLanguage":"HANDLEBARS","style":null,"texts":{},"defaults":{"config":{"applicablePages":[],"description":"DevCentral´s custom footer.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Beta_Footer","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"CUSTOM","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"DevCentral´s custom footer.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Tag_Manager_Helper-en-us-1746693083847":{"__typename":"CachedAsset","id":"component:custom.widget.Tag_Manager_Helper-en-us-1746693083847","value":{"component":{"id":"custom.widget.Tag_Manager_Helper","template":{"id":"Tag_Manager_Helper","markupLanguage":"HANDLEBARS","style":null,"texts":{},"defaults":{"config":{"applicablePages":[],"description":"Helper widget to inject Tag Manager scripts into head element","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Tag_Manager_Helper","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"CUSTOM","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"Helper widget to inject Tag Manager scripts into head element","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Consent_Blackbar-en-us-1746693083847":{"__typename":"CachedAsset","id":"component:custom.widget.Consent_Blackbar-en-us-1746693083847","value":{"component":{"id":"custom.widget.Consent_Blackbar","template":{"id":"Consent_Blackbar","markupLanguage":"HTML","style":null,"texts":{},"defaults":{"config":{"applicablePages":[],"description":"","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Consent_Blackbar","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"TEXTHTML","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:text:en_US-components/community/Breadcrumb-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/community/Breadcrumb-1744046271000","value":{"navLabel":"Breadcrumbs","dropdown":"Additional parent page navigation"},"localOverride":false},"CachedAsset:text:en_US-components/tags/TagsHeaderWidget-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/tags/TagsHeaderWidget-1744046271000","value":{"tag":"{tagName}","topicsCount":"{count} {count, plural, one {Topic} other {Topics}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageListForNodeByRecentActivityWidget-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageListForNodeByRecentActivityWidget-1744046271000","value":{"title@userScope:other":"Recent Content","title@userScope:self":"Contributions","title@board:FORUM@userScope:other":"Recent Discussions","title@board:BLOG@userScope:other":"Recent Blogs","emptyDescription":"No content to show","MessageListForNodeByRecentActivityWidgetEditor.nodeScope.label":"Scope","title@instance:1706288370055":"Content Feed","title@instance:1743095186784":"Most Recent Updates","title@instance:1704317906837":"Content Feed","title@instance:1743095018194":"Most Recent Updates","title@instance:1702668293472":"Community Feed","title@instance:1743095117047":"Most Recent Updates","title@instance:1704319314827":"Blog Feed","title@instance:1743095235555":"Most Recent Updates","title@instance:1704320290851":"My Contributions","title@instance:1703720491809":"Forum Feed","title@instance:1743095311723":"Most Recent Updates","title@instance:1703028709746":"Group Content Feed","title@instance:VTsglH":"Content Feed"},"localOverride":false},"Category:category:Forums":{"__typename":"Category","id":"category:Forums","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Forum:board:TechnicalForum":{"__typename":"Forum","id":"board:TechnicalForum","forumPolicies":{"__typename":"ForumPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Forum:board:WaterCooler":{"__typename":"Forum","id":"board:WaterCooler","forumPolicies":{"__typename":"ForumPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:Articles":{"__typename":"Category","id":"category:Articles","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"displayId":"Articles"},"Tkb:board:TechnicalArticles":{"__typename":"Tkb","id":"board:TechnicalArticles","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"displayId":"TechnicalArticles","nodeType":"board","conversationStyle":"TKB","title":"Technical Articles","shortTitle":"Technical Articles","parent":{"__ref":"Category:category:Articles"}},"Tkb:board:DevCentralNews":{"__typename":"Tkb","id":"board:DevCentralNews","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:GroupsCategory":{"__typename":"Category","id":"category:GroupsCategory","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:F5-Groups":{"__typename":"Category","id":"category:F5-Groups","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:CommunityGroups":{"__typename":"Category","id":"category:CommunityGroups","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Occasion:board:Events":{"__typename":"Occasion","id":"board:Events","boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"occasionPolicies":{"__typename":"OccasionPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Idea:board:Suggestions":{"__typename":"Idea","id":"board:Suggestions","boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"ideaPolicies":{"__typename":"IdeaPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:CrowdSRC":{"__typename":"Category","id":"category:CrowdSRC","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:codeshare":{"__typename":"Tkb","id":"board:codeshare","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:communityarticles":{"__typename":"Tkb","id":"board:communityarticles","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:security-insights":{"__typename":"Tkb","id":"board:security-insights","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:article-series":{"__typename":"Tkb","id":"board:article-series","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Conversation:conversation:306550":{"__typename":"Conversation","id":"conversation:306550","topic":{"__typename":"TkbTopicMessage","uid":306550},"lastPostingActivityTime":"2025-01-15T10:02:24.982-08:00","solved":false},"User:user:242856":{"__typename":"User","uid":242856,"login":"MichaelOLeary","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0yNDI4NTYtMjA2NzVpMjAwQzU1OUQzMEFFMDM2RQ"},"id":"user:242856"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE5MzdpMDUxQjQzRTRDMTIyRTZFNQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE5MzdpMDUxQjQzRTRDMTIyRTZFNQ?revision=68","title":"k8s.jpg","associationType":"COVER","width":500,"height":500,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1MDZpNjFFNDAwMzE1RkI2MTAwQQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1MDZpNjFFNDAwMzE1RkI2MTAwQQ?revision=68","title":"k8s-arch-options.png","associationType":"BODY","width":699,"height":263,"altText":"k8s-arch-options.png"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzNpQjkyNDZCQjVGMzc1RUQ0MQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzNpQjkyNDZCQjVGMzc1RUQ0MQ?revision=68","title":"secure-k8s-gw.PNG","associationType":"BODY","width":2508,"height":1201,"altText":"secure-k8s-gw.PNG"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzZpNUNEOUUzOUMxNEFCOTI2NQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzZpNUNEOUUzOUMxNEFCOTI2NQ?revision=68","title":"k8s-site.PNG","associationType":"BODY","width":2518,"height":1183,"altText":"k8s-site.PNG"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1NzFpQUQ1NUIyM0U2MDcyMEMyOQ?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1NzFpQUQ1NUIyM0U2MDcyMEMyOQ?revision=68","title":"vk8s.PNG","associationType":"BODY","width":2165,"height":1284,"altText":"vk8s.PNG"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1OTBpMkMwRDFFNTcxNjI1NUQ4Qg?revision=68\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1OTBpMkMwRDFFNTcxNjI1NUQ4Qg?revision=68","title":"mk8s.PNG","associationType":"BODY","width":2359,"height":1339,"altText":"mk8s.PNG"},"TkbTopicMessage:message:306550":{"__typename":"TkbTopicMessage","subject":"Kubernetes architecture options with F5 Distributed Cloud Services","conversation":{"__ref":"Conversation:conversation:306550"},"id":"message:306550","revisionNum":68,"uid":306550,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:242856"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":9723},"postTime":"2023-01-11T09:22:19.033-08:00","lastPublishTime":"2025-01-15T10:02:24.982-08:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" \n Summary \n F5 Distributed Cloud Services (F5 XC) can both integrate with your existing Kubernetes (K8s) clusters and/or host a K8s workload itself. Within these distinctions, we have multiple architecture options. This article explores four major architectures in ascending order of sophistication and advantages. \n \n Architecture #1: External Load Balancer (Secure K8s Gateway) \n Architecture #2: CE as a pod (K8s site) \n Architecture #3: Managed Namespace (vK8s) \n Architecture #4: Managed K8s (mK8s) \n \n Kubernetes Architecture Options \n As K8s continues to grow, options for how we run K8s and integrate with existing K8s platforms continue to grow. F5 XC can both integrate with your existing K8s clusters and/or run a managed K8s platform itself. Multiple architectures exist within these offerings too, so I was thoroughly confused when I first heard about these possibilities. \n A colleague recently laid it out for me in a conversation: \n \"Michael, listen up: XC can either integrate with your K8s platform, run inside your K8s platform, host virtual K8s (Namespace-aaS), or run a K8s platform in your environment.\" I replied, \"That's great. Now I have a mental model for differentiating between architecture options.\" \n This article will overview these architectures and provide 101-level context: when, how, and why would you implement these options? \n Side note 1: F5 XC concepts and terms \n F5 XC is a global platform that can provide networking and app delivery services, as well as compute (K8s workloads). We call each of our global PoP's a Regional Edge (RE). RE's are highly meshed to form the backbone of the global platform. They connect your sites, they can expose your services to the Internet, and they can run workloads. This platform is extensible into your data center by running one or more XC Nodes in your network, also called a Customer Edge (CE). A CE is a compute node in your network that registers to our global control plane and is then managed by a customer as SaaS. \n The registration of one or more CE's creates a customer site in F5 XC. A CE can run on a hypervisor (VMWare/KVM/Etc), a Hyperscaler (AWS, Azure, GCP, etc), baremetal, or even as a k8s pod, and can be deployed in HA clusters. \n XC Mesh functionality provides connectivity between sites, security services, and observability. Optionally, in addition, XC App Stack functionality allows a large and arbitrary number of managed clusters to be logically grouped into a virtual site with a single K8s mgmt interface. So where Mesh services provide the networking, App Stack services provide the Kubernetes compute mgmt. Our first 2 architectures require Mesh services only, and our last two require App Stack. \n Side note 2: Service-to-service communication \n I'm often asked how to allow services between clusters to communicate with each other. This is possible and easy with XC. Each site can publish services to every other site, including K8s sites. This means that any K8s service can be reachable from other sites you choose. And this can be true in any of the architectures below, although more granular controls are possible with the more sophisticated architectures. I'll explore this common question more in a separate article. \n Architecture 1: External Load Balancer (Secure K8s Gateway) \n In a Secure Kubernetes Gateway architecture, you have integration with your existing K8s platform, using the XC node as the external load balancer for your K8s cluster. In this scenario, you create a ServiceAccount and kubeconfig file to configure XC. The XC node then performs service discovery against your K8s API server. I've covered this process in a previous article, but the advantage is that you can integrate with existing K8s platforms. This allows exposing both NodePort and ClusterIP services via the XC node. \n \n XC is not hosting any workloads in this architecture, but it is exposing your services to your local network, or remote sites, or the Internet. In the diagram above, I show a web application being accesssed from a remote site (and/or the Internet) where the origin pool is a NodePort service discovered in a K8s cluster. \n Architecture 2: Run a site within a K8s cluster (K8s site type) \n Creating a K8s site is easy - just deploy a single manifest found here. This file deploys multiple resources in your cluster, and together these resources work to provide the services of a CE, and create a customer site. I've heard this referred to as \"running a CE inside of K8s\" or \"running your CE as a pod\". However, when I say \"CE node\" I'm usually referring to a discreet compute node like a VM or piece of hardware; this architecture is actually a group of pods and related resources that run within K8s to create a XC customer site. \n With XC running inside your existing cluster, you can expose services within the cluster by DNS name because the site will resolve these from within the cluster. Your service can then be exposed anywhere by the F5 XC platform. This is similar to Architecture 1 above, but with this model, your site is simply a group of pods within K8s. An advantage here is the ability to expose services of other types (e.g. ClusterIP). \n A site deployed into a K8s cluster will only support Mesh functionality and does not support AppStack functionality (i.e., you cannot run a cluster within your cluster). In this architecture, XC acts as a K8s ingress controller with built-in application security. It also enables Mesh features, such as publishing of other sites' services on this site, and publishing of this site's discovered services on other sites. \n Architecture 3: vK8s (Namespace-as-a-Service) \n If the services you use include AppStack capabilities, then architectures #3 and #4 are possible for you. In these scenarios, our XC node actually runs your K8s on your workloads. We are no longer integrating XC with your existing K8s platform. XC is the platform. \n A simple way to run K8s workloads is to use a virtual k8s (vK8s) architecture. This could be referred to as a \"managed Namespace\" because by creating a vK8s object in XC you get a single namespace in a virtual cluster. \n Your Namespace can be fully hosted (deployed to RE's) or run on your VM's (CE's), or both. Your kubeconfig file will allow access to your Namespace via the hosted API server. Via your regular kubectl CLI (or via the web console) you can create/delete/manage K8s resources (Deployments, Services, Secrets, ServiceAccounts, etc) and view application resource metrics. \n This is great if you have workloads that you want to deploy to remote regions where you do not have infrastructure and would prefer to run in F5's RE's, or if you have disparate clusters across multiple sites and you'd like to manage multiple K8s clusters via a single centralized, virtual cluster. \n Best practice guard rails for vK8s \n With a vK8s architecture, you don't have your own cluster, but rather a managed Namespace. So there are some restrictions (for example, you cannot run a container as root, bind to a privileged port, or to the Host network). You cannot create CRD's, ClusterRoles, PodSecurityPolicies, or Namespaces, so K8s operators are not supported. In short, you don't have a managed cluster, but a managed Namespace on a virtual cluster. \n Architecture 4: mK8s (Managed K8s) \n In managed k8s (mk8s, also known as physical K8s or pk8s) deployment, we have an enterprise-level K8s distribution that is run at your site. This means you can use XC to deploy/manage/upgrade K8s infrastructure, but you manage the Kubernetes resources. The benefits include what is typical for 3rd-party K8s mgmt solutions, but also some key differentiators: \n \n multi-cloud, with automation for Azure, AWS, and GCP environments \n consumed by you as SaaS \n enterprise-level traffic control natively \n allows a large and arbitrary number of managed clusters to be logically managed with a single K8s mgmt interface \n \n You can enable kubectl access against your local cluster and disable the hosted API server, so your kubeconfig file can point to a global URL or a local endpoint on-prem. \n Another benefit of mK8s is that you are running a full K8s cluster at your site, not just a Namespace in a virtual cluster. The restrictions that apply to vK8s (see above) do not apply to mK8s, so you could run privileged pods if required, use Operators that make use of ClusterRoles and CRDs, and perform other tasks that require cluster-wide access. \n Traffic management controls with mK8s \n Because your workloads run in a cluster managed by XC, we can apply more sophisticated and native policies to K8s traffic than non-managed clusters in earlier architectures: \n \n Service isolation can be enforced within the cluster, so that pods in a given namespace cannot communicate with services outside of that namespace, by default. \n More service-to-service controls exist so that you can decide which services can reach with other services with more granularity. \n Egress control can be natively enforced for outbound traffic from the cluster, by namespace, labels, IP ranges, or other methods. E.g.: Svc A can reach myapi.example.com but no other Internet service. \n WAF policies, bot defense, L3/4 policies, etc—all of these policies that you have typically applied with network firewalls, WAF's, etc—can be applied natively within the platform. \n \n This architecture took me a long time to understand, and longer to fully appreciate. But once you have run your workloads natively on a managed K8s platform that is connected to a global backbone and capable of performing network and application delivery within the platform, the security and traffic mgmt benefits become very compelling. \n Conclusion: \n As K8s continues to expand, management solutions of your clusters make it possible to secure your K8s services, whether they are managed by XC or exist in disparate clusters. With F5 XC as a global platform consumed as a service—not a discreet installation managed by you—the available architectures here are unique and therefore can accommodate the diverse (and changing!) ways we see K8s run today. \n Related Articles \n \n Securely connecting Kubernetes Microservices with F5 Distributed Cloud \n Multi-cluster Multi-cloud Networking for K8s with F5 Distributed Cloud - Architecture Pattern \n Multiple Kubernetes Clusters and Path-Based Routing with F5 Distributed Cloud \n ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"10582","kudosSumWeight":29,"repliesCount":5,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE5MzdpMDUxQjQzRTRDMTIyRTZFNQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1MDZpNjFFNDAwMzE1RkI2MTAwQQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzNpQjkyNDZCQjVGMzc1RUQ0MQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjEzMzZpNUNEOUUzOUMxNEFCOTI2NQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1NzFpQUQ1NUIyM0U2MDcyMEMyOQ?revision=68\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDY1NTAtMjE1OTBpMkMwRDFFNTcxNjI1NUQ4Qg?revision=68\"}"}}],"totalCount":6,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:281666":{"__typename":"Conversation","id":"conversation:281666","topic":{"__typename":"TkbTopicMessage","uid":281666},"lastPostingActivityTime":"2022-02-18T08:22:42.920-08:00","solved":false},"User:user:150953":{"__typename":"User","uid":150953,"login":"Eric_Chen","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0xNTA5NTMtZXB1akpu?image-coordinates=267%2C0%2C1348%2C1080"},"id":"user:150953"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE2NjYtMzc0MGk5NUIyRDdCNERCMTQyODM4?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE2NjYtMzc0MGk5NUIyRDdCNERCMTQyODM4?revision=1","title":"0151T000002dnl3QAA.png","associationType":"BODY","width":600,"height":338,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE2NjYtNTA2NGlFMjkzNTM5OUU1RTFFMzBG?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE2NjYtNTA2NGlFMjkzNTM5OUU1RTFFMzBG?revision=1","title":"0151T000002dnlNQAQ.png","associationType":"BODY","width":600,"height":338,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE2NjYtNjQ0M2k5QTk4NTA0OTU1NkVCMzQy?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE2NjYtNjQ0M2k5QTk4NTA0OTU1NkVCMzQy?revision=1","title":"0151T000002dnl4QAA.png","associationType":"BODY","width":600,"height":389,"altText":null},"TkbTopicMessage:message:281666":{"__typename":"TkbTopicMessage","subject":"3 Ways to use F5 BIG-IP with OpenShift 4","conversation":{"__ref":"Conversation:conversation:281666"},"id":"message:281666","revisionNum":1,"uid":281666,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:150953"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":8699},"postTime":"2020-06-29T11:25:52.000-07:00","lastPublishTime":"2020-06-29T11:25:52.000-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" F5 BIG-IP can provide key infrastructure and application services in a RedHat OpenShift 4 environment. Examples include providing core load balancing for the OpenShift API and Router, DNS services for the cluster, a supplement or replacement for the OpenShift Router, and security protection for the OpenShift management and application services. #1. Core Services OpenShift 4 requires a method to provide high availability to the OpenShift API (port 6443), MachineConfig (22623), and Router services (80/443). BIG-IP Local Traffic Manager (LTM) can provide these trusted services easily. OpenShift also requires several DNS records that the BIG-IP can provide accelerated responses as a DNS cache and/or providing Global Server Load Balancing of cluster DNS records. Additional documentation about OpenShift 4 Network Requirements (RedHat) Networking Requirements for user-provisioned infrastructure #2 OpenShift Router RedHat provides their own OpenShift Router for L7 load balancing, but the F5 BIG-IP can also provide these services using Container Ingress Services. Instead of deploying load balancing resources on the same nodes that are hosting OpenShift workloads; F5 BIG-IP provides these services outside of the cluster on either hardware or Virtual Edition platforms. Container Ingress Services can run either as an auxiliary router to the included router or a replacement. Additional articles that are related to Container Ingress Services •\tUsing F5 BIG-IP Controller for OpenShift #3 Security F5 can help filter, authenticate, and validate requests that are going into or out of an OpenShift cluster. LTM can be used to host sensitive SSL resources outside of the cluster (including on a hardware HSM if necessary) as well as filtering of requests (i.e. disallow requests to internal resources like the management console). Advanced Web Application Firewall (AWAF) policies can be deployed to stymie bad actors from reaching sensitive applications. Access Policy Manager can provide OpenID Connect services for the OpenShift management console and help with providing identity services for applications and microservices that are running on OpenShift (i.e. converting BasicAuth request into a JWT token for a microservice). Additional documentation related to attaching a security policy to an OpenShift Route •\tAS3 Override Where Can I Try This? The environment that was used to write this article and create the companion video can be found at: https://github.com/f5devcentral/f5-k8s-demo/tree/ocp4/ocp4. For folks that are part of F5 you can access this in our Unified Demo Framework and can schedule labs with customers/partners (search for \"OpenShift 4.3 with CIS\"). I plan on publishing a version of this demo environment that can run natively in AWS. Check back to this article for any updates. Thanks! ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"2927","kudosSumWeight":6,"repliesCount":3,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE2NjYtMzc0MGk5NUIyRDdCNERCMTQyODM4?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE2NjYtNTA2NGlFMjkzNTM5OUU1RTFFMzBG?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE2NjYtNjQ0M2k5QTk4NTA0OTU1NkVCMzQy?revision=1\"}"}}],"totalCount":3,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:307273":{"__typename":"Conversation","id":"conversation:307273","topic":{"__typename":"TkbTopicMessage","uid":307273},"lastPostingActivityTime":"2023-10-09T13:15:57.350-07:00","solved":false},"User:user:194353":{"__typename":"User","uid":194353,"login":"MattHarmon","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0xOTQzNTMtMjA4OTRpMUNEQUQzODNBOURDMjA0OA"},"id":"user:194353"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NDdpNzk0MTk5QTFFRjI3N0Q0OQ?revision=33\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NDdpNzk0MTk5QTFFRjI3N0Q0OQ?revision=33","title":"RE Network - Virtual Sites.jpg","associationType":"COVER","width":860,"height":558,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0MzJpMTM2RkQ5MTM5MDUwNDZCNQ?revision=33\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0MzJpMTM2RkQ5MTM5MDUwNDZCNQ?revision=33","title":"BGP_Peer_Report_12302022.jpg","associationType":"BODY","width":752,"height":556,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjlpRkQ1Qzc4NEI4Rjg0NTIwNA?revision=33\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjlpRkQ1Qzc4NEI4Rjg0NTIwNA?revision=33","title":"decryption-circle-virtualsites.jpg","associationType":"BODY","width":837,"height":547,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE1MjRpQ0JCN0QyRjNEQTBCRjQyMA?revision=33\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE1MjRpQ0JCN0QyRjNEQTBCRjQyMA?revision=33","title":"Important Note.jpg","associationType":"BODY","width":727,"height":132,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjFpRDc2NDBCNDNCQjFBQTNDNw?revision=33\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjFpRDc2NDBCNDNCQjFBQTNDNw?revision=33","title":"Virtual_Sites_-_Manage___Shared_Configuration.jpg","associationType":"BODY","width":565,"height":322,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjJpMzA3OUQ4MzhENDUyNDVGNw?revision=33\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjJpMzA3OUQ4MzhENDUyNDVGNw?revision=33","title":"Public_IP_Addresses_-_Manage___Shared_Configuration.jpg","associationType":"BODY","width":933,"height":393,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjNpOTFFRDYxMEI4NTIxRjJCMw?revision=33\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjNpOTFFRDYxMEI4NTIxRjJCMw?revision=33","title":"HTTP_Load_Balancers_-_Load_Balancers___Load_Balancers.jpg","associationType":"BODY","width":730,"height":220,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NzFpNjcxQ0NBRUEzNTczQ0MyQw?revision=33\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NzFpNjcxQ0NBRUEzNTczQ0MyQw?revision=33","title":"verification.jpg","associationType":"BODY","width":1036,"height":433,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NzJpMkJCMUE3RUI0RjgwQTE4Qw?revision=33\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NzJpMkJCMUE3RUI0RjgwQTE4Qw?revision=33","title":"Traffic Routing - Virtual Site Example.jpg","associationType":"BODY","width":781,"height":437,"altText":null},"TkbTopicMessage:message:307273":{"__typename":"TkbTopicMessage","subject":"F5 Distributed Cloud - Regional Decryption with Virtual Sites","conversation":{"__ref":"Conversation:conversation:307273"},"id":"message:307273","revisionNum":33,"uid":307273,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:194353"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":" In this article we discuss how the F5 Distributed Cloud can be configured to support regulatory demands for TLS termination of traffic to specific regions around the world. The article provides insight into the F5 Distributed Cloud global backbone and application delivery network (ADN). The article goes on to inspect how the F5 Distriubted Cloud is able to achieve these custom topologies in a multi-tenant architecture while adhearing to the \"rules of the internet\" for route summarization. Read on to learn about the flexibility of F5's SaaS platform providing application delivery and security solutions for your applications. ","introduction":"","metrics":{"__typename":"MessageMetrics","views":6398},"postTime":"2023-01-17T05:00:00.021-08:00","lastPublishTime":"2023-10-09T13:15:57.350-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" \n Introduction: \n With the continued growth of multi-cloud infrastructures, enterprises are looking for a \"cloud above the clouds\" as their new strategic point of delivery and control. This is exactly what the F5 Distributed Cloud (F5 XC) provides with its SaaS-delivered data planes that we call Regional Edges (RE). The F5 Distributed Cloud enables enterprises to use our SaaS platform for Application Proxying, Load Balancing, Web Application Firewall, API Discovery, and Security, Bot and Fraud Protection, Multi-Cloud Networking, and much more. To achieve these multi-cloud delivery and security solutions, we must terminate TLS within our Regional Edges. However, what if your organization has rules and regulations around encryption, such that inspection points and key storage must reside within specific countries or continents in which the enterprise operates? Let's look at how you can resolve this challenge with F5 Distributed Cloud's Virtual Site configuration. \n F5 Distributed Cloud - Networking Overview \n By default, our F5 Distributed Cloud's SaaS data \n planes of Regional Edges are connected to two networks, the Internet, and the F5 Global Backbone. Our Global Backbone is a multi-terrabit, low-latency physical network with a software-defined, distributed overlay network. We call this the F5 Distributed Cloud Application Delivery Network (ADN). The ADN provides interconnectivity between our Regional Edges, Customer's Data Centers, Customer's Clouds, and our Customer Edge software. If you review Hurricane Electric Internet Services BGP Peering Report, you'll see F5 is one of the most peered global networks in the world with almost 5000 IPv4 adjacencies and almost 4000 IPv6 adjacencies. \n These adjacencies are critical to the success of our Anycast route advertisement, which uses the underlay routing of the internet for BGP multi-path to route clients to the nearest Regional Edge. Internet standards require organizations advertising network prefixes to the internet to be a /24 or greater. F5 Distributed Cloud advertises multiple /24 or greater prefixes globally from all Regional Edges. If you're a networking nerd like me, you can dig deeper into F5's advertisements via Hurricane Electric, linked above. If you're curious about how internet advertisements have changed over the years, you might refer to APNIC's article on BGP in 2021 - The BGP Table, where they review how the internet routing tables have changed over the years. \n \n \n By now you're probably asking, why all this networking background... well, because I like it and it is my article, so ha! Well, in reality, this relates to how the F5 Distributed Cloud routes \"Virtual Sites\" within the platform. By definition, a Virtual Site is a \"tool for indirection. Instead of doing configuration on each Site, it allows for performing a given configuration on set (or group) of Sites. Virtual Site is a configuration object that defines the Sites that are members of the set.\" By default, when you sign up for F5 Distributed Cloud, you are provided a public IP address to which you can attach Application Delivery and Security services to. This IP address is part of a pre-defined Virtual Site, which is configured for ALL Regional Edges to process traffic for any services attached to your tenant's IP address. This virtual site is used when the \"VIP Advertisement\" is set to \"Internet\". \n \n The Solution \n \n As you could probably guess by now, we will use Virtual Sites to build logical topologies within the F5 Distributed Cloud. This configuration allows you to configure which Regional Edges are allowed to store keys, decrypt traffic, proxy, and apply other application delivery and security features of your choosing on a per-application basis. Within the F5 Distributed Cloud you can mix and match Virtual Site configurations for your applications, where some may be using the default of all Regional Edges, and some may be using only specific Regional Edges due to the application and security requirements. \n \n \n Let's step through this diagram and discuss how this solution works. You can see All Region Edges are advertising a /24 of 185.56.152.0 to the internet. Now let's imagine we want to have two custom topologies, one for North America's Regional Edges, and one for Regional Edges within Europe. We would configure two new Virtual Sites, which I will show you how to do the steps in the next section. There is a pre-requisite to setting up decryption topologies via Virtual Sites; we need an additional Virtual IP (VIP) address per topology within the tenant. Remember, a tenant comes with a single IP by default, but you can add additional IPs. These IPs can come from the F5 Distributed Cloud blocks, or you can Bring Your Own IPs (BYOIP), but this requires that you bring a full /24 due to the rules of the internet for advertisement. \n \n \n Once you have your additional VIPs and your Virtual Sites configured, you're now able to apply these to individual application configurations. We use the Advertise VIP section for the application and select the IP address to advertise associated to the virtual site topology desired. The IP address that is associated with this virtual site will now be advertised into the F5 Global Backbone as a /32. Traffic from the internet will still follow its closest path to the /24 prefix, but once traffic reaches the F5 Regional Edge's routing edge, it'll use the F5 Global Backbone to the nearest Regional Edge configured as a Virtual Site for this application. Simply, the backbone follows its more specific route. Once traffic reaches the Regional Edge associated with the Virtual Site, the Regional Edge software will decrypt, proxy, and apply any additional configurations to the application's traffic flow. \n \n Configuration Steps: \n **Reminder — You must have additional public IP addres(es) within your tenant as a pre-requisite** \n \n Create a Virtual Site - From \"Select Service\" - Choose \"Shared Configuration\" ==> Manage -> Virtual Site\n \n Use existing labels to choose sites or regions of sites that you would like to be included in your Regional Edge Virtual Site \n IMPORTANT - Be sure to include more than one Regional Edge for your Virtual Site \n Graphic below shows London and Paris' Regional Edges configured as part of my EU Virtual Site \n \n \n \n \n Update additional Tenant IP Address - From \"Select Service\" - Choose \"Shared Configuration\" ==> Manage -> Public IP Addresses\n \n Click the three dots under \"Actions\" and choose \"Manage Configuration\" \n In the top right corner, click \"Edit Configuration\" \n Under \"Virtual Site\" - choose the Virtual Site you created in Step 1 \n \n \n \n Update VIP Advertisement on Load Balancer—From \"Select Service\" - Choose \"Load Balancers\" ==> Manage -> Load Balancer -> HTTP Load Balancer \n \n Since our Virtual Site and Public IP address are configured within the shared namespace, we can apply this config to any Load Balancer in any Namespace. \n Either create a new HTTP Load Balancer, or edit an existing Load Balancer's configuration \n Navigate to \"Other Settings\" and change the VIP Advertisement to \"Internet (Specific VIP)\" \n Choose the public IP Address that you modified for the Virtual Site in step 2. \n \n \n \n \n Verification: \n Be sure that DNS has been updated to reflect the FQDN(s) associated with the HTTP Load Balancer and now resolve to the new public IP address. Verify by running a nslookup or dig from your local machine. The reason I chose EU Regional Edges for this article was to simplify the verification that this solution is working, as I am based in the United States. My traffic will typically traverse the NY Regional Edge. \n Navigate: \"Select Service\" - Choose \"Load Balancers\" ==> Virtual Host -> Load Balancer -> Click \"Performance Monitoring\" under your Load Balancer -> Click \"Requests\" tab and find your request. \n You'll see for the client request below, the client is in the United States, specifically Mentor, Ohio. However, the L5-7 services, TLS Termination, Proxy, Load Balancing, and additional delivery and security features occurred in the Paris Regional Edge (pa4-par). When reviewing your request, you can click on the JSON tab, which provides upwards of 100 attributes of each request. \n \n For this specific traffic flow, the client traversed the Internet to our NY Regional Edge, then utilized the F5 Global Backbone to the nearest Regional Edge configured as part of the Virtual Site, which in this client's case the BGP multi-path of Paris. Once the traffic arrives in the Virtual Site Regional Edge (Paris), the F5 XC Software then provides L3-L7 services, most importantly, TLS termination and proxying of the traffic. \n In the below diagram, we see this traffic flow as red line traversing the internet, yellow line using F5 Global Backbone, and green line within the Virtual Site Regional Edge. The green line would be the termination point for the client-side traffic flow. This client-side flow is to the Regional Edge software where L3-4 connection is established, the TLS session setup begins, and ultimately where application layer services would be proxied to the origin service. \n \n Summary: \n I hope you found this article useful and learned how F5 Distributed Cloud allows you to be specific in where your keys are stored and utilized for TLS termination within the F5 Distributed Cloud SaaS architecture. By using Virtual Sites and additional Public IP Addresses, you can build custom topologies of our Regional Edges over the F5 Global Backbone. Please leave comments or questions, and I'd be happy to update the article or provide answers directly in the comments. \n Additional Links to Distributed Cloud Content: \n Kubernetes Architecture Options with F5 Distributed Cloud \n App Stack, the \"iRule\" of F5 Distributed Cloud \n Multi-Cluster, Multi-Cloud Networking for K8S with F5 Distributed Cloud – Architecture Pattern ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"10410","kudosSumWeight":17,"repliesCount":2,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NDdpNzk0MTk5QTFFRjI3N0Q0OQ?revision=33\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0MzJpMTM2RkQ5MTM5MDUwNDZCNQ?revision=33\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjlpRkQ1Qzc4NEI4Rjg0NTIwNA?revision=33\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE1MjRpQ0JCN0QyRjNEQTBCRjQyMA?revision=33\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjFpRDc2NDBCNDNCQjFBQTNDNw?revision=33\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjJpMzA3OUQ4MzhENDUyNDVGNw?revision=33\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NjNpOTFFRDYxMEI4NTIxRjJCMw?revision=33\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDg","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NzFpNjcxQ0NBRUEzNTczQ0MyQw?revision=33\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDk","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMDcyNzMtMjE0NzJpMkJCMUE3RUI0RjgwQTE4Qw?revision=33\"}"}}],"totalCount":9,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:280471":{"__typename":"Conversation","id":"conversation:280471","topic":{"__typename":"TkbTopicMessage","uid":280471},"lastPostingActivityTime":"2021-09-30T07:33:15.000-07:00","solved":false},"User:user:49440":{"__typename":"User","uid":49440,"login":"Chris_Zhang","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/m_assets/avatars/default/avatar-1.svg?time=0"},"id":"user:49440"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtMTI5NTNpOUEyMTc4MEVDREU4Qjg2OQ?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtMTI5NTNpOUEyMTc4MEVDREU4Qjg2OQ?revision=1","title":"0151T000003lgmfQAA.png","associationType":"BODY","width":1611,"height":880,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtMzE0N2kzMDUwRTlEODBGQzRCRjQw?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtMzE0N2kzMDUwRTlEODBGQzRCRjQw?revision=1","title":"0151T000003lgn4QAA.png","associationType":"BODY","width":749,"height":704,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtNjk4MmlDODY1Njg1MzEzQ0E5Q0Y0?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtNjk4MmlDODY1Njg1MzEzQ0E5Q0Y0?revision=1","title":"0151T000003lgnEQAQ.png","associationType":"BODY","width":385,"height":406,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtNTA2NWkzRjc2NzA5NUEwOUMwOUY0?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtNTA2NWkzRjc2NzA5NUEwOUMwOUY0?revision=1","title":"0151T000003lgnFQAQ.png","associationType":"BODY","width":906,"height":483,"altText":null},"TkbTopicMessage:message:280471":{"__typename":"TkbTopicMessage","subject":"Better together - F5 Container Ingress Services and NGINX Plus Ingress Controller Integration","conversation":{"__ref":"Conversation:conversation:280471"},"id":"message:280471","revisionNum":1,"uid":280471,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:49440"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":5470},"postTime":"2020-03-05T14:52:16.000-08:00","lastPublishTime":"2020-03-05T14:52:16.000-08:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Introduction The F5 Container Ingress Services (CIS) can be integrated with the NGINX Plus Ingress Controllers (NIC) within a Kubernetes (k8s) environment. The benefits are getting the best of both worlds, with the BIG-IP providing comprehensive L4 ~ L7 security services, while leveraging NGINX Plus as the de facto standard for micro services solution. This architecture is depicted below. The integration is made fluid via the CIS, a k8s pod that listens to events in the cluster and dynamically populates the BIG-IP pool pointing to the NIC's as they scale. There are a few components need to be stitched together to support this integration, each of which is discussed in detail over the proceeding sections. NGINX Plus Ingress Controller Follow this (https://docs.nginx.com/nginx-ingress-controller/installation/building-ingress-controller-image/) to build the NIC image. The NIC can be deployed using the Manifests either as a Daemon-Set or a Service. See this ( https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/ ). A sample Deployment file deploying NIC as a Service is shown below, apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: nginx-ingress\n namespace: nginx-ingress\nspec:\n replicas: 3\n selector:\n matchLabels:\n app: nginx-ingress\n template:\n metadata:\n labels:\n app: nginx-ingress\n #annotations:\n #prometheus.io/scrape: \"true\"\n #prometheus.io/port: \"9113\"\n spec:\n serviceAccountName: nginx-ingress\n imagePullSecrets:\n - name: abgmbh.azurecr.io\n containers:\n - image: abgmbh.azurecr.io/nginx-plus-ingress:edge\n name: nginx-plus-ingress\n ports:\n - name: http\n containerPort: 80\n - name: https\n containerPort: 443\n #- name: prometheus\n #containerPort: 9113\n securityContext:\n allowPrivilegeEscalation: true\n runAsUser: 101 #nginx\n capabilities:\n drop:\n - ALL\n add:\n - NET_BIND_SERVICE\n env:\n - name: POD_NAMESPACE\n valueFrom:\n fieldRef:\n fieldPath: metadata.namespace\n - name: POD_NAME\n valueFrom:\n fieldRef:\n fieldPath: metadata.name\n args:\n - -nginx-plus\n - -nginx-configmaps=$(POD_NAMESPACE)/nginx-config\n - -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret\n - -ingress-class=sock-shop\n #- -v=3 # Enables extensive logging. Useful for troubleshooting.\n #- -report-ingress-status\n #- -external-service=nginx-ingress\n #- -enable-leader-election\n #- -enable-prometheus-metrics\n Notice the ‘- -ingress-class=sock-shop’ argument, it means that the NIC will only work with an Ingress that is annotated with ‘sock-shop’. The absence of this annotation makes NIC the default for all Ingress created. Below shows the counterpart Ingress with the ‘sock-shop’ annotation. apiVersion: extensions/v1beta1\nkind: Ingress\nmetadata:\n name: sock-shop-ingress\n annotations:\n kubernetes.io/ingress.class: \"sock-shop\"\nspec:\n tls:\n - hosts:\n - socks.ab.gmbh\n secretName: wildcard.ab.gmbh\n rules:\n - host: socks.ab.gmbh\n http:\n paths:\n - path: /\n backend:\n serviceName: front-end\n servicePort: 80\n This Ingress says if hostname is socks.ab.gmbh and path is ‘/’, send traffic to a service named ‘front-end’, which is part of the socks application itself. The above concludes Ingress configuration with the NIC. F5 Container Ingress Services The next step is to leverage the CIS to dynamically populate the BIG-IP pool with the NIC addresses. Follow this ( https://clouddocs.f5.com/containers/v2/kubernetes/kctlr-app-install.html ) to deploy the CIS. A sample Deployment file is shown below, apiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n name: k8s-bigip-ctlr-deployment\n namespace: kube-system\nspec:\n # DO NOT INCREASE REPLICA COUNT\n replicas: 1\n template:\n metadata:\n name: k8s-bigip-ctlr\n labels:\n app: k8s-bigip-ctlr\n spec:\n # Name of the Service Account bound to a Cluster Role with the required\n # permissions\n serviceAccountName: bigip-ctlr\n containers:\n - name: k8s-bigip-ctlr\n image: \"f5networks/k8s-bigip-ctlr\"\n env:\n - name: BIGIP_USERNAME\n valueFrom:\n secretKeyRef:\n # Replace with the name of the Secret containing your login\n # credentials\n name: bigip-login\n key: username\n - name: BIGIP_PASSWORD\n valueFrom:\n secretKeyRef:\n # Replace with the name of the Secret containing your login\n # credentials\n name: bigip-login\n key: password\n command: [\"/app/bin/k8s-bigip-ctlr\"]\n args: [\n # See the k8s-bigip-ctlr documentation for information about\n # all config options\n # https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest\n \"--bigip-username=$(BIGIP_USERNAME)\",\n \"--bigip-password=$(BIGIP_PASSWORD)\",\n \"--bigip-url=https://x.x.x.x:8443\",\n \"--bigip-partition=k8s\",\n \"--pool-member-type=cluster\",\n \"--agent=as3\",\n \"--manage-ingress=false\",\n \"--insecure=true\",\n \"--as3-validation=true\",\n \"--node-poll-interval=30\",\n \"--verify-interval=30\",\n \"--log-level=INFO\"\n ]\n imagePullSecrets:\n # Secret that gives access to a private docker registry\n - name: f5-docker-images\n # Secret containing the BIG-IP system login credentials\n - name: bigip-login\n Notice the following arguments below. They tell the CIS to consume AS3 declaration to configure the BIG-IP. According to PM, CCCL (Common Controller Core Library) – used to orchestrate F5 BIG-IP, is getting removed this sprint for the CIS 2.0 release. '--manage-ingress=false' means CIS is not doing anything for Ingress resources defined within the k8s, this is because that CIS is not the Ingress Controller, NGINX Plus is, as far as k8s is concerned. The CIS will create a partition named k8s_AS3 on the BIG-IP, this is used to hold L4~7 configuration relating to the AS3 declaration. The best practice is also to manually create a partition named 'k8s' (in our example), where networking info will be stored (e.g., ARP, FDB). \"--bigip-url=https://x.x.x.x:8443\",\n \"--bigip-partition=k8s\",\n \"--pool-member-type=cluster\",\n \"--agent=as3\",\n \"--manage-ingress=false\",\n \"--insecure=true\",\n \"--as3-validation=true\",\n To apply AS3, the declaration is embedded within a ConfigMap applied to the CIS pod. kind: ConfigMap\napiVersion: v1\nmetadata:\n name: as3-template\n namespace: kube-system\n labels:\n f5type: virtual-server\n as3: \"true\"\ndata:\n template: |\n {\n \"class\": \"AS3\",\n \"action\": \"deploy\",\n \"persist\": true,\n \"declaration\": {\n \"class\": \"ADC\",\n \"id\":\"1847a369-5a25-4d1b-8cad-5740988d4423\",\n \"schemaVersion\": \"3.16.0\",\n \"Nginx_IC\": {\n \"class\": \"Tenant\",\n \"Nginx_IC_vs\": {\n \"class\": \"Application\",\n \"template\": \"https\",\n \"serviceMain\": {\n \"class\": \"Service_HTTPS\",\n \"virtualAddresses\": [\n \"10.1.0.14\"\n ],\n \"virtualPort\": 443,\n \"redirect80\": false,\n \"serverTLS\": {\n \"bigip\": \"/Common/clientssl\"\n },\n \"clientTLS\": {\n \"bigip\": \"/Common/serverssl\"\n },\n \"pool\": \"Nginx_IC_pool\"\n },\n \"Nginx_IC_pool\": {\n \"class\": \"Pool\",\n \"monitors\": [\n \"https\"\n ],\n \"members\": [\n {\n \"servicePort\": 443,\n \"shareNodes\": true,\n \"serverAddresses\": []\n }\n ]\n }\n }\n }\n } \n }\n They are telling the BIG-IP to create a tenant called ‘Nginx_IC’, a virtual named ‘Nginx_IC_vs’ and a pool named ‘Nginx_IC_pool’. The CIS will update the serverAddresses with the NIC addresses dynamically. Now, create a Service to expose the NIC’s. apiVersion: v1\nkind: Service\nmetadata:\n name: nginx-ingress\n namespace: nginx-ingress\n labels:\n cis.f5.com/as3-tenant: Nginx_IC\n cis.f5.com/as3-app: Nginx_IC_vs\n cis.f5.com/as3-pool: Nginx_IC_pool\nspec:\n type: ClusterIP\n ports:\n - port: 443\n targetPort: 443\n protocol: TCP\n name: https\n selector:\n app: nginx-ingress\n Notice the labels, they match with the AS3 declaration and this allows the CIS to populate the NIC’s addresses to the correct pool. Also notice the kind of the manifest ‘Service’, this means only a Service is created, not an Ingress, as far as k8s is concerned. On the BIG-IP, the following should be created. The end product is below. Please note that this article is focused solely on control plane, that is, how to get the CIS to populate the BIG-IP with NIC's addresses. The specific mechanisms to deliver packets from the BIG-IP to the NIC's on the data plane is not discussed, as it is decoupled from control plane. For data plane specifics, please take a look here ( https://clouddocs.f5.com/containers/v2/ ). Hope this article helps to lift the veil on some integration mysteries. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"19512","kudosSumWeight":11,"repliesCount":27,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtMTI5NTNpOUEyMTc4MEVDREU4Qjg2OQ?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtMzE0N2kzMDUwRTlEODBGQzRCRjQw?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtNjk4MmlDODY1Njg1MzEzQ0E5Q0Y0?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODA0NzEtNTA2NWkzRjc2NzA5NUEwOUMwOUY0?revision=1\"}"}}],"totalCount":4,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:281416":{"__typename":"Conversation","id":"conversation:281416","topic":{"__typename":"TkbTopicMessage","uid":281416},"lastPostingActivityTime":"2019-06-05T08:50:48.000-07:00","solved":false},"User:user:115030":{"__typename":"User","uid":115030,"login":"Rodrigo_Albuque","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/m_assets/avatars/default/avatar-6.svg?time=0"},"id":"user:115030"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNTY3aTUyQTU0NjZGRDg0QTBERjA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNTY3aTUyQTU0NjZGRDg0QTBERjA?revision=1","title":"0151T000003dF59QAE.png","associationType":"BODY","width":614,"height":188,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMzQ0NGlBMjlGRUI4OTk5RDBBM0My?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMzQ0NGlBMjlGRUI4OTk5RDBBM0My?revision=1","title":"0151T000003dF5OQAU.png","associationType":"BODY","width":1232,"height":508,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtODUyM2lGQjU4ODQ5Njc2RDk3M0Q4?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtODUyM2lGQjU4ODQ5Njc2RDk3M0Q4?revision=1","title":"0151T000003dF5TQAU.png","associationType":"BODY","width":1464,"height":148,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNTQ5N2lCRTdFRjhEQTdDRjM1NUFD?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNTQ5N2lCRTdFRjhEQTdDRjM1NUFD?revision=1","title":"0151T000003dF5YQAU.png","associationType":"BODY","width":2006,"height":126,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTI5NTBpRTE5N0U2N0I1NDE1RUZDMQ?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTI5NTBpRTE5N0U2N0I1NDE1RUZDMQ?revision=1","title":"0151T000003dF5dQAE.png","associationType":"BODY","width":1214,"height":720,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTU3NTJpNDY5RTIyOENDRkU2RjdFOA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTU3NTJpNDY5RTIyOENDRkU2RjdFOA?revision=1","title":"0151T000003dF5iQAE.png","associationType":"BODY","width":1135,"height":690,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTA1MjBpRDA4ODk5QUJEN0E2MUI2RA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTA1MjBpRDA4ODk5QUJEN0E2MUI2RA?revision=1","title":"0151T000003dF5nQAE.png","associationType":"BODY","width":1196,"height":188,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNTY3OWk1NTg2RDJGN0RENjI3RTVD?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNTY3OWk1NTg2RDJGN0RENjI3RTVD?revision=1","title":"0151T000003dF5sQAE.png","associationType":"BODY","width":1678,"height":126,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNzU5aTI4NkY3NUE1NDhBNzc0NTk?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNzU5aTI4NkY3NUE1NDhBNzc0NTk?revision=1","title":"0151T000003dF5xQAE.png","associationType":"BODY","width":1218,"height":404,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMjMzMGlFQjNERjVGRkIwQTcyRUFE?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMjMzMGlFQjNERjVGRkIwQTcyRUFE?revision=1","title":"0151T000003dF6RQAU.png","associationType":"BODY","width":1134,"height":1322,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTQ3NzlpNDcwRDFEN0I1RTQ0QkNCMw?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTQ3NzlpNDcwRDFEN0I1RTQ0QkNCMw?revision=1","title":"0151T000003dF6WQAU.png","associationType":"BODY","width":1334,"height":152,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTM5ODhpQTQ4MkJENEFGMTYxN0QwRA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTM5ODhpQTQ4MkJENEFGMTYxN0QwRA?revision=1","title":"0151T000003dF6bQAE.png","associationType":"BODY","width":1820,"height":124,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtODYxOGk4OUIyNDUyQTZGNjExM0FD?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtODYxOGk4OUIyNDUyQTZGNjExM0FD?revision=1","title":"0151T000003dF6gQAE.png","associationType":"BODY","width":1312,"height":540,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtODAzOWk4MEVGMkJDOUE4RDA1QzY3?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtODAzOWk4MEVGMkJDOUE4RDA1QzY3?revision=1","title":"0151T000003dF6lQAE.png","associationType":"BODY","width":1330,"height":1290,"altText":null},"TkbTopicMessage:message:281416":{"__typename":"TkbTopicMessage","subject":"Exploring Kubernetes API using Wireshark part 1: Creating, Listing and Deleting Pods","conversation":{"__ref":"Conversation:conversation:281416"},"id":"message:281416","revisionNum":1,"uid":281416,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:115030"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":4599},"postTime":"2019-06-05T08:50:48.000-07:00","lastPublishTime":"2019-06-05T08:50:48.000-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Related Articles: Exploring Kubernetes API using Wireshark part 2: Namespaces Exploring Kubernetes API using Wireshark part 3: Python Client API Quick Intro This article answers the following question: What happens when we create, list and delete pods under the hood? More specifically on the wire. I used these 3 commands: I'll show you on Wireshark the communication between kubectl client and master node (API) for each of the above commands. I used a proxy so we don't have to worry about TLS layer and focus on HTTP only. Creating NGINX pod pcap: creating_pod.pcap (use http filter on Wireshark) Here's our YAML file: Here's how we create this pod: Here's what we see on Wireshark: Behind the scenes, kubectl command sent an HTTP POST with our YAML file converted to JSON but notice the same thing was sent (kind, apiVersion, metadata, spec): You can even expand it if you want to but I didn't to keep it short. Then, Kubernetes master (API) responds with HTTP 201 Created to confirm our pod has been created: Notice that master node replies with similar data with the additional status column because after pod is created it's supposed to have a status too. Listing Pods pcap: listing_pods.pcap (use http filter on Wireshark) When we list pods, kubectl just sends a HTTP GET request instead of POST because we don't need to submit any data apart from headers: This is the full GET request: And here's the HTTP 200 OK with JSON file that contains all information about all pods from default's namespace: I just wanted to emphasise that when you list a pod the resource type that comes back is PodList and when we created our pod it was just Pod. Remember? The other thing I'd like to point out is that all of your pods' information should be listed under items. All kubectl does is to display some of the API's info in a humanly readable way. Deleting NGINX pod pcap: deleting_pod.pcap (use http filter on Wireshark) Behind the scenes, we're just sending an HTTP DELETE to Kubernetes master: Also notice that the pod's name is also included in the URI: /api/v1/namespaces/default/pods/nginx ← this is pods' name HTTP DELETE just like HTTP GET is pretty straightforward: Our master node replies with HTTP 200 OK as well as some json file with all the info about the pod, including about it's termination: It's also good to emphasise here that when our pod is deleted, master node returns JSON file with all information available about the pod. I highlighted some interesting info. For example, resource type is now just Pod (not PodList when we're just listing our pods). ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"2780","kudosSumWeight":3,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNTY3aTUyQTU0NjZGRDg0QTBERjA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMzQ0NGlBMjlGRUI4OTk5RDBBM0My?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtODUyM2lGQjU4ODQ5Njc2RDk3M0Q4?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNTQ5N2lCRTdFRjhEQTdDRjM1NUFD?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTI5NTBpRTE5N0U2N0I1NDE1RUZDMQ?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTU3NTJpNDY5RTIyOENDRkU2RjdFOA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTA1MjBpRDA4ODk5QUJEN0E2MUI2RA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDg","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNTY3OWk1NTg2RDJGN0RENjI3RTVD?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDk","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtNzU5aTI4NkY3NUE1NDhBNzc0NTk?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEw","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMjMzMGlFQjNERjVGRkIwQTcyRUFE?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEx","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTQ3NzlpNDcwRDFEN0I1RTQ0QkNCMw?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEy","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtMTM5ODhpQTQ4MkJENEFGMTYxN0QwRA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEz","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtODYxOGk4OUIyNDUyQTZGNjExM0FD?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE0","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MTYtODAzOWk4MEVGMkJDOUE4RDA1QzY3?revision=1\"}"}}],"totalCount":14,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:297576":{"__typename":"Conversation","id":"conversation:297576","topic":{"__typename":"TkbTopicMessage","uid":297576},"lastPostingActivityTime":"2023-06-05T23:00:59.117-07:00","solved":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTkyNzhpQkM4NDM2MEI5N0FCQjA1Mg?revision=29\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTkyNzhpQkM4NDM2MEI5N0FCQjA1Mg?revision=29","title":"MichaelOLeary_0-1661959216870.png","associationType":"BODY","width":814,"height":314,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTkyNzlpMjJGNDMyNTA0ODVGRkY5Rg?revision=29\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTkyNzlpMjJGNDMyNTA0ODVGRkY5Rg?revision=29","title":"MichaelOLeary_0-1661960484397.png","associationType":"BODY","width":838,"height":660,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTgzMzhpQjM4QjA2MEZGNDQ3MUQ4RA?revision=29\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTgzMzhpQjM4QjA2MEZGNDQ3MUQ4RA?revision=29","title":"azure-rbac-k8s-authz-flow.png","associationType":"BODY","width":895,"height":290,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTgzMzdpQjIxNTUyQUMxMkRDNzFBMQ?revision=29\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTgzMzdpQjIxNTUyQUMxMkRDNzFBMQ?revision=29","title":"eks-iam.png","associationType":"BODY","width":500,"height":308,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTkyODFpRDg3OERGRTQ1OTJGNEZBNg?revision=29\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTkyODFpRDg3OERGRTQ1OTJGNEZBNg?revision=29","title":"MichaelOLeary_0-1661961836381.png","associationType":"BODY","width":856,"height":506,"altText":null},"TkbTopicMessage:message:297576":{"__typename":"TkbTopicMessage","subject":"Service Discovery and authentication options for Kubernetes providers (EKS, AKS, GCP)","conversation":{"__ref":"Conversation:conversation:297576"},"id":"message:297576","revisionNum":29,"uid":297576,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:242856"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":4504},"postTime":"2022-08-31T05:00:00.031-07:00","lastPublishTime":"2023-06-05T23:00:59.117-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Summary \n Hosted Kubernetes (K8s) providers use different services for authenticating to your hosted K8s cluster. A kubeconfig file will define your cluster's API server address and your authentication to this server. Uploading this kubeconfig file is how you enable service discovery from another platform like F5 Distributed Cloud (F5 XC). \n What is Service Discovery in F5 XC? \n Service Discovery in this case means something outside the cluster is querying the K8s API and learning the services and pods running inside the cluster. In our case, we want to send traffic into K8s via a platform that can publish our k8s services internally or publicly, and also apply security to the applications exposed. \n \n How do I configure k8s Service Discovery? \n Follow these instructions to configure K8s Service Discovery in XC. You will notice that a kubeconfig file is typically required to be uploaded to XC. \n User authentication to AKS, EKS, and GKE \n In K8s, User authentication happens outside of the cluster. There is no \"User\" resource in K8s - you cannot create a User with kubectl. Unlike ServiceAccounts (SAs), which are created inside K8s and whose authentication secrets exist inside K8s, a User is authenticated with a system outside of K8s. There are multiple authentication schemes, and certificates and OAuth are very common to see. \n \n For this article, I'll review the typical authentication providers major providers: Azure's AKS, AWS's EKS, and Google's GKE. \n Azure AKS \n AKS authentication (authn) starts at Azure Active Directory, and authorization (authz) can be applied at AAD or k8s RBAC. For this article I am focused on authn (not authz). \n \n If you follow official instructions to create an AKS cluster via CLI, you will run the az aks get-credentials command to get the access credentials for an AKS cluster and merge them into your kubeconfig file. \n You will see that a client certificate and key is included in the User section of your kubeconfig file. This kubeconfig file can be uploaded to F5 XC for successful service discovery because it does not rely on any additional software to be on the kubectl client. \n (However, be cautious. Don't share the kubeconfig file further, since it contains credentials to AKS. Microsoft provides instructions to use Azure role-based access control (Azure RBAC) to control access to these credentials. These Azure roles let you define who can retrieve the kubeconfig file, and what permissions they then have within the cluster.) \n AWS EKS \n \"Typical\" EKS kubeconfig \n The AWS EKS cluster authentication process requires the use of an AWS IAM identity, and typically relies on an additional software being installed on the kubectl client machine. Instead of credentials being configured directly in the kubeconfig file, either the aws cli or aws-iam-authenticator are used locally on the kubectl client to generate a token. This token is sent to the k8s master API server, which is verified against AWS IAM (see steps 1-2 in the image below). \n\n \n Use a service account for EKS access \n The AWS user that created the cluster will have system:masters permissions to your cluster, but if you want to add another user to your cluster, you must edit the aws-auth ConfigMap within Kubernetes. AWS has instructions to do this so I won't repeat them here, but the implications should be clear for what we are trying to do: \n \n You don't want a 3rd party authenticating to EKS as your primary user account. If you created the EKS cluster with your primary account, you should create a different user for EKS access. Create an AWS IAM user for this purpose, and then add this user to the aws-auth ConfigMap within Kubernetes. \n EKS requires authentication from the kubectl client. Instead of hard-coding credentials into the kubeconfig file, the kubeconfig file tells the client to run one of the two following commands that rely on the existing of additional software: aws-iam-authenticator token -i [clustername] or aws eks get-token --region [region] --cluster-name [cluster] \n \n We can update our kubeconfig file so that it can use credentials directly in the kubeconfig file, so that we can upload this file to F5 XC for service discovery. \n Creating a kubeconfig file that uses aws cli or aws-iam-authenticator with provided credentials \n In the case of F5 XC, we can see from the instructions that \"...,you must add AWS credentials in the kubeconfig file for successful service discovery\". This means that to upload your kubeconfig to XC you will need to configure it to use the aws-iam-authenticator option, and provide your IAM aws_access_key_id and aws_secret_access_key as env vars in your kubeconfig. Example kubeconfig: \n\n\n\n .... <everything else in kubeconfig file> ...\nusers:\n- name: arn:aws:eks:us-east-1:<my_aws_acct_number>:cluster/<my_eks_cluster_name>\n user:\n exec:\n apiVersion: client.authentication.k8s.io/v1alpha1\n args:\n - token\n - -i\n - <my_eks_cluster_name>\n command: aws-iam-authenticator\n env:\n - name: AWS_ACCESS_KEY_ID\n value: \"<my_aws_access_key_id>\"\n - name: AWS_SECRET_ACCESS_KEY\n value: \"<my_aws_secret_access_key>\"\n \n\n\n\n I can use the above kubeconfig file for successful service discovery in XC. I did make some notes: \n \n In my testing, the aws cli method did not work in XC. It appears that the XC node has aws-iam-authenticator installed, but not aws cli. \n Notice the apiVersion above is v1alpha1. This works when uploaded to F5 XC, but when testing this file on my local workstation, I had to ensure my kubectl client was downgraded to kubectl v 1.23.6 or lower. v1alpha1 has been deprecated in newer kubectl versions, but was required for me to authenticate with the aws-iam-authenticator method. This is not ideal. It appears that the XC node has a version of kubectl that allows for v1alpha1 auth, but we know this will be deprecated eventually. \n \n These notes led me to asking: can I get a kubeconfig file to authenticate to AWS without 3rd party tools? Read on for how to do this, but first I'll cover Google's GKE. \n Google GKE \n Similar to EKS, GKE uses gcloud cli as a \"credential helper\" in the typical kubeconfig file for GKE cluster administration. This excellent article covers GKE authentication very well. \n This means we need to take a similar approach as we did for EKS, if we want to authenticate to the GKE API server from a location without gcloud installed. For GKE, I followed an approach I learnd here this blog post, but I'll summarize for you now: \n \n Create a Kubeconfig file that contains your cluster API server address and CA certificate. These are safe to commit in git because they do not need to be private. \n Create a Service Account in Google, generate a key for this service account and save it in JSON format. Export the variable GOOGLE_APPLICATION_CREDENTIALS so the value is the location of your .json file. \n Use kubectl commands. These should work because kubectl as a client is aware of the env var above. \n \n This article from Google explains the community's requirement that provider-specific code is removed from the OSS codebase in v1.25 (as of my writing, v1.24 is the most recent release). This requires an update to kubectl that will remove existing functionality that allows kubectl to authenticate to GKE. Soon, you will need to use a new credential helper from Google called gke-gcloud-auth-plugin. Soon, you'll be relying on a different 3rd party plugin for GKE authentication. So our question will come up again, as it did earlier: Can't I just have a kubeconfig file that doesn't rely on a 3rd party library? \n Creating a kubeconfig file that does not rely on other software \n What if we need to upload our kubeconfig to a machine that does not have the aws cli, aws-iam-authenticator, or any other 3rd party credential helpers installed? Sometimes you just need a kubeconfig that doesn't rely on 3rd party software or their IAM service, and this is what ServiceAccounts can achieve. \n \n I wrote an article with a script that you can use, focused on K8s SA's for service discovery with F5 XC. Since it's all documented in another article I'll summarize the steps here. \n \n Create a Service Account (SA) with permissions to discover services. \n Export the secret for that SA to use as an auth token. \n Create a kubeconfig file that targets your cluster and authenticates with your SA. \n This file can be uploaded to XC. \n \n Summary \n A kubeconfig file is relatively straightforward for most k8s admins. However, uploading a kubeconfig file into a hosted service, for the purpose of k8s service discovery, will sometimes require further analysis. Remember: \n \n Your kubeconfig file defines your authentication to the k8s API server. So use a service account (not your primary user account) \n If your kubeconfig file relies on a helper like aws cli, aws-iam-authenticator, or gcloud, you can probably work around this by editing your kubeconfig file to use a SA and token instead. This may be required if you upload the kubeconfig file to a hosted platform like F5 XC. \n \n All this might seem complex at first, but after service discovery is configured the ability to expose your services anywhere - publicly or privately - is incredibly easy and powerful. Good luck and reach out with feedback! ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"9527","kudosSumWeight":3,"repliesCount":3,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTkyNzhpQkM4NDM2MEI5N0FCQjA1Mg?revision=29\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTkyNzlpMjJGNDMyNTA0ODVGRkY5Rg?revision=29\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTgzMzhpQjM4QjA2MEZGNDQ3MUQ4RA?revision=29\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTgzMzdpQjIxNTUyQUMxMkRDNzFBMQ?revision=29\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTc1NzYtMTkyODFpRDg3OERGRTQ1OTJGNEZBNg?revision=29\"}"}}],"totalCount":5,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:291352":{"__typename":"Conversation","id":"conversation:291352","topic":{"__typename":"TkbTopicMessage","uid":291352},"lastPostingActivityTime":"2022-04-21T05:00:29.200-07:00","solved":false},"User:user:276871":{"__typename":"User","uid":276871,"login":"Nicolas_Menant","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/m_assets/avatars/default/avatar-5.svg?time=0"},"id":"user:276871"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTI5MTBpODE1OTdCQUZCMkE5QTJBQQ?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTI5MTBpODE1OTdCQUZCMkE5QTJBQQ?revision=1","title":"0EM1T000001N3VW.png","associationType":"BODY","width":606,"height":597,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTIzNDBpRkU3NDJGMTYyMEFEN0M5Qw?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTIzNDBpRkU3NDJGMTYyMEFEN0M5Qw?revision=1","title":"0EM1T000001N3Vb.png","associationType":"BODY","width":565,"height":491,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTM5OTlpRTM2MzUzNDU2REUwREM4Mg?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTM5OTlpRTM2MzUzNDU2REUwREM4Mg?revision=1","title":"0EM1T000001N3Vg.png","associationType":"BODY","width":835,"height":168,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTU0MDNpM0E0QUI0M0NGMjIwRkYzOA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTU0MDNpM0E0QUI0M0NGMjIwRkYzOA?revision=1","title":"0EM1T000001N3Vl.png","associationType":"BODY","width":1103,"height":327,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItODQ5MWk0QUQ1RDU2MDNGNDlBQkZD?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItODQ5MWk0QUQ1RDU2MDNGNDlBQkZD?revision=1","title":"0EM1T000001N3Vq.png","associationType":"BODY","width":713,"height":188,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItOTYyOWkyOTQ2NTEyNTI5RENGNURF?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItOTYyOWkyOTQ2NTEyNTI5RENGNURF?revision=1","title":"0EM1T000001N3Vv.png","associationType":"BODY","width":642,"height":188,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNDMwM2kxODMzNTQxODdEMjBGRTA4?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNDMwM2kxODMzNTQxODdEMjBGRTA4?revision=1","title":"0EM1T000001N3W0.png","associationType":"BODY","width":1105,"height":407,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNjAyN2lGQ0VBMjE5ODk4Q0ZBQzYw?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNjAyN2lGQ0VBMjE5ODk4Q0ZBQzYw?revision=1","title":"0EM1T000001N3W5.png","associationType":"BODY","width":510,"height":660,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTE4NTRpMTU0MDJCRDQ5N0MzRjQ3OA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTE4NTRpMTU0MDJCRDQ5N0MzRjQ3OA?revision=1","title":"0EM1T000001N3WF.png","associationType":"BODY","width":697,"height":257,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNDcyMWkxQTc1M0NCRDc3REIxNEFD?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNDcyMWkxQTc1M0NCRDc3REIxNEFD?revision=1","title":"0EM1T000001N3WK.png","associationType":"BODY","width":545,"height":881,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNzc4MGk5NTY1RkUyMDU4MUVGRTYx?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNzc4MGk5NTY1RkUyMDU4MUVGRTYx?revision=1","title":"0EM1T000001N3WU.png","associationType":"BODY","width":294,"height":72,"altText":null},"TkbTopicMessage:message:291352":{"__typename":"TkbTopicMessage","subject":"CIS and Kubernetes - Part 1: Install Kubernetes and Calico","conversation":{"__ref":"Conversation:conversation:291352"},"id":"message:291352","revisionNum":1,"uid":291352,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:276871"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":4399},"postTime":"2019-11-04T01:13:47.000-08:00","lastPublishTime":"2019-11-04T01:13:47.000-08:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Welcome to this series to see how to: Install Kubernetes and Calico (Part 1) Deploy F5 Container Ingress Services (F5 CIS) to tie applications lifecycle to our application services (Part 2) Here is the setup of our lab environment: BIG-IP Version: 15.0.1 Kubernetes component: Ubuntu 18.04 LTM We consider that your BIG-IPs are already setup and running: Licensed and setup as a cluster The networking setup is already done Part 1: Install Kubernetes and Calico Setup our systems before installing kubernetes Step1: Update our systems and install docker To run containers in Pods, Kubernetes uses a container runtime. We will use docker and follow the recommendation provided here As root on ALL Kubernetes components (Master and Node): # Install packages to allow apt to use a repository over HTTPS\napt-get -y update && apt-get install -y apt-transport-https ca-certificates curl software-properties-common\n\n# Add Docker’s official GPG key\ncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -\n\n# Add Docker apt repository.\nadd-apt-repository \\\n \"deb [arch=amd64] https://download.docker.com/linux/ubuntu \\\n $(lsb_release -cs) \\\n stable\"\n\n# Install Docker CE.\napt-get -y update && apt-get install -y docker-ce=18.06.2~ce~3-0~ubuntu\n\n# Setup daemon.\ncat > /etc/docker/daemon.json <<EOF\n{\n \"exec-opts\": [\"native.cgroupdriver=systemd\"],\n \"log-driver\": \"json-file\",\n \"log-opts\": {\n \"max-size\": \"100m\"\n },\n \"storage-driver\": \"overlay2\"\n}\nEOF\n\nmkdir -p /etc/systemd/system/docker.service.d\n\n# Restart docker.\nsystemctl daemon-reload\nsystemctl restart docker\n\n We may do a quick test to ensure docker run as expected: docker run hello-world\n Step2: Setup Kubernetes tools (kubeadm, kubelet and kubectl) To setup Kubernetes, we will leverage the following tools: kubeadm: the command to bootstrap the cluster. kubelet: the component that runs on all of the machines in your cluster and does things like starting pods and containers. kubectl: the command line util to talk to your cluster. As root on ALL Kubernetes components (Master and Node): curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -\ncat <<EOF | tee /etc/apt/sources.list.d/kubernetes.list\ndeb https://apt.kubernetes.io/ kubernetes-xenial main\nEOF\napt-get -y update\n We can review which version of kubernetes is supported with F5 Container Ingress Services here At the time of this article, the latest supported version is v1.13.4. We'll make sure to install this specific version with our following step apt-get install -qy kubelet=1.13.4-00 kubeadm=1.13.4-00 kubectl=1.13.4-00 kubernetes-cni=0.6.0-00\napt-mark hold kubelet kubeadm kubectl\n Install Kubernetes Step1: Setup Kubernetes with kubeadm We will follow the steps provided in the documentation here As root on the MASTER node (make sure to update the api server address to reflect your master node IP): kubeadm init --apiserver-advertise-address=10.1.20.20 --pod-network-cidr=192.168.0.0/16\n Note: SAVE somewhere the kubeadm join command. It is needed to \"assimilate\" the node later. In my example, it looks like the following (YOURS WILL BE DIFFERENT): kubeadm join 10.1.20.20:6443 --token rlbc20.va65z7eauz89mmuv --discovery-token-ca-cert-hash sha256:42eca5bf49c645ff143f972f6bc88a59468a30276f907bf40da3bcf5127c0375\n Now you should NOT be ROOT anymore. Go back to your non root user. Since i use Ubuntu, i'll use the default \"ubuntu\" user Run the following commands as highlighted in the screenshot above: mkdir -p $HOME/.kube\nsudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config\nsudo chown $(id -u):$(id -g) $HOME/.kube/config\n Step2: Install the networking component of Kubernetes The last step is to setup the network related to our k8s infrastructure. In our kubeadm init command, we used --pod-network-cidr=192.168.0.0/16 in order to be able to setup next on network leveraging Calico as documented here kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml\n You may monitor the deployment by running the command: kubectl get pods --all-namespaces\n After some time (<1 min), everything should have a \"Running\" status. Make sure that CoreDNS started also properly. If everything is up and running, we have our master setup properly and can go to the node to setup k8s on it. Step3: Add the Node to our Kubernetes Cluster Now that the master is setup properly, we can assimilate the node. You need to retrieve the \"kubeadmin join …\" command that you received at the end of the \"kubeadm init …\" cmd. You must run the following command as ROOT on the Kubernetes NODE (remember that you got a different hash and token, the command below is an example): kubeadm join 10.1.20.20:6443 --token rlbc20.va65z7eauz89mmuv --discovery-token-ca-cert-hash sha256:42eca5bf49c645ff143f972f6bc88a59468a30276f907bf40da3bcf5127c0375\n We can check the status of our node by running the following command on our MASTER (ubuntu user) kubectl get nodes\n Both component should have a \"Ready\" status. Last step is to setup Calico between our BIG-IPs and our Kubernetes cluster Setup Calico We need to setup Calico on our BIG-IPs and k8S components. We will setup our environment with the following AS Number: 64512 Step1: BIG-IPs Calico setup F5 has documented this procedure here We will use our self IPs on the internal network. Therefore we need to make sure of the following: The self IP has a portlock down setup to \"Allow All\" Or add a TCP custom port to the self IP: TCP port 179 You need to allow BGP on the default route domain 0 on your BIG-IPs. Connect to the BIG-IP GUI on go into Network > Route domain. Click on Route Domain \"0\" and allow BGP Click on \"Update\" Once this is done, connect via SSH and get into a bash shell on both BIG-IPs Run the following commands: #access the IMI Shell\nimish\n\n#Switch to enable mode\nenable\n\n#Enter configuration mode\nconfig terminal\n#Setup route bgp with AS Number 64512\nrouter bgp 64512\n\n#Create BGP Peer group\nneighbor calico-k8s peer-group\n\n#assign peer group as BGP neighbors\nneighbor calico-k8s remote-as 64512\n\n#we need to add all the peers: the other BIG-IP, our k8s components\nneighbor 10.1.20.20 peer-group calico-k8s\nneighbor 10.1.20.21 peer-group calico-k8s\n\n#on BIG-IP1, run\nneighbor 10.1.20.12 peer-group calico-k8s\n\n#on BIG-IP2, run\nneighbor 10.1.20.11 peer-group calico-k8s\n\n#save configuration\nwrite\n\n#exit\nend\n You can review your setup with the command show ip bgp neighbors\n Note: your other BIG-IP should be identified with a router ID and have a BGP state of \"Active\". The k8s node won't have a router ID since BGP hasn't already been setup on those nodes. Keep your BIG-IP SSH sessions open. We'll re-use the imish terminal once our k8s components have Calico setup Step2: Kubernetes Calico setup On the MASTER node (not as root), we need to retrieve the calicoctl binary curl -O -L https://github.com/projectcalico/calicoctl/releases/download/v3.10.0/calicoctl\nchmod +x calicoctl\nsudo mv calicoctl /usr/local/bin\n We need to setup calicoctl as explained here sudo mkdir /etc/calico\n Create a file /etc/calico/calicoctl.cfg with your preferred editor (you'll need sudo privilegies). This file should contain the following apiVersion: projectcalico.org/v3\nkind: CalicoAPIConfig\nmetadata:\nspec:\n datastoreType: \"kubernetes\"\n kubeconfig: \"/home/ubuntu/config\"\n Note: you may have to change the path specified by the kubeconfig parameter based on the user you use to do kubectl command To make sure that calicoctl is properly setup, run the command calicoctl get nodes\n You should get a list of your Kubernetes nodes Now we can work on our Calico/BGP configuration as documented here On the MASTER node: cat << EOF | calicoctl create -f -\n apiVersion: projectcalico.org/v3\n kind: BGPConfiguration\n metadata:\n name: default\n spec:\n logSeverityScreen: Info\n nodeToNodeMeshEnabled: true\n asNumber: 64512\nEOF\n Note: Because we setup nodeToNodeMeshEnabled to True, the k8s node will receive the same config We may now setup our BIG-IP BGP peers. Replace the peerIP Value with the IP of your BIG-IPs cat << EOF | calicoctl create -f -\napiVersion: projectcalico.org/v3\nkind: BGPPeer\nmetadata:\n name: bgppeer-global-bigip1\nspec:\n peerIP: 10.1.20.11\n asNumber: 64512\nEOF\n cat << EOF | calicoctl create -f -\napiVersion: projectcalico.org/v3\nkind: BGPPeer\nmetadata:\n name: bgppeer-global-bigip2\nspec:\n peerIP: 10.1.20.12\n asNumber: 64512\nEOF\n Review your setup with the command: calicoctl get bgpPeer\n If you go back to your BIG-IP SSH connections, you may check that your Kubernetes nodes have a router ID now in your BGP configuration: imish\nshow ip bgp neighbors\n Summary So far we have: Setup Kubernetes Setup Calico between our BIG-IPs and our Kubernetes cluster In the next article, we will setup F5 container Ingress Services (F5 CIS) ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"9588","kudosSumWeight":1,"repliesCount":1,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTI5MTBpODE1OTdCQUZCMkE5QTJBQQ?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTIzNDBpRkU3NDJGMTYyMEFEN0M5Qw?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTM5OTlpRTM2MzUzNDU2REUwREM4Mg?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTU0MDNpM0E0QUI0M0NGMjIwRkYzOA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItODQ5MWk0QUQ1RDU2MDNGNDlBQkZD?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItOTYyOWkyOTQ2NTEyNTI5RENGNURF?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNDMwM2kxODMzNTQxODdEMjBGRTA4?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDg","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNjAyN2lGQ0VBMjE5ODk4Q0ZBQzYw?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDk","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItMTE4NTRpMTU0MDJCRDQ5N0MzRjQ3OA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEw","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNDcyMWkxQTc1M0NCRDc3REIxNEFD?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEx","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTEzNTItNzc4MGk5NTY1RkUyMDU4MUVGRTYx?revision=1\"}"}}],"totalCount":11,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:291538":{"__typename":"Conversation","id":"conversation:291538","topic":{"__typename":"TkbTopicMessage","uid":291538},"lastPostingActivityTime":"2023-12-15T10:13:15.793-08:00","solved":false},"User:user:180940":{"__typename":"User","uid":180940,"login":"buulam","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0xODA5NDAtMTYzMjJpRTQxNTQzRDk2N0MyQkU3Ng"},"id":"user:180940"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTE1MzgtMTY0ODVpMThDNzU1MDQxQzJBNDcwQQ?revision=9\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTE1MzgtMTY0ODVpMThDNzU1MDQxQzJBNDcwQQ?revision=9","title":"DC-Cover-Stock_0005_luca-bravo-XJXWbfSo2f0-unsplash.jpg","associationType":"COVER","width":1000,"height":1000,"altText":""},"TkbTopicMessage:message:291538":{"__typename":"TkbTopicMessage","subject":"Understanding Modern Application Architecture - Part 1","conversation":{"__ref":"Conversation:conversation:291538"},"id":"message:291538","revisionNum":9,"uid":291538,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:180940"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":4031},"postTime":"2022-01-25T11:00:00.039-08:00","lastPublishTime":"2023-12-15T10:13:15.793-08:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" \n This is part 1 of a series. Here are the other parts: \n Understanding Modern Application Architecture - Part 2 \n Understanding Modern Application Architecture - Part 3 \n \n Over the past decade, there has been a change taking place in how applications are built. As applications become more expansive in capabilities and more critical to how a business operates, (or in many cases, the application is the business itself) a new style of architecture has allowed for increased scalability, portability, resiliency, and agility. To support the goals of a modern application, the surrounding infrastructure has had to evolve as well. Platforms like Kubernetes have played a big role in unlocking the potential of modern applications and is a new paradigm in itself for how infrastructure is managed and served. \n \n To help our community transition the skillset they've built to deal with monolithic applications, we've put together a series of videos to drive home concepts around modern applications. This article highlights some of the details found within the video series. \n \n In these first three videos, we breakdown the definition of a Modern Application. \n \n \n \n One might think that by name only, a modern application is simply an application that is current. But we're actually speaking in comparison to a monolithic application. Monolithic applications are made up of a single, or a just few pieces. They are rigid in how they are deployed and fragile in their dependencies. Modern applications will instead incorporate microservices. Where a monolithic application might have all functions built into one broad encompassing service, microservices will break down the service into smaller functions that can be worked on separately. \n \n \n \n A modern application will also incorporate 4 main pillars. \n \n Scalability ensures that the application can handle the needs of a growing user base, both for surges as well as long term growth. \n Portability ensures that the application can be transportable from its underlying environment while still maintaining all of its functionality and management plane capabilities. \n Resiliency ensures that failures within the system go unnoticed or pose minimal disruption to users of the application. \n Agility ensures that the application can accommodate for rapid changes whether that be to code or to infrastructure. \n \n \n There are also 6 design principles of a modern application. \n \n Being agnostic will allow the application to have freedom to run on any platform. \n Leveraging open source software where it makes sense can often allow you to move quickly with an application but later be able to adopt commercial versions of that software when full support is needed. \n Defining by code allows for more uniformity of configuration and move away rigid interfaces that require specialized knowledge. \n Automated CI/CD processes ensures the quick integration and deployment of code so that improvements are constantly happening while any failures are minimized and contained. \n Secure development ensures that application security is integrated into the development process and code is tested thoroughly before being deployed into production. \n Distributed Storage and Infrastructure ensures that applications are not bound by any physical limitations and components can be located where they make the most sense. \n \n \n \n \n These videos should help set the foundation for what a modern application is. The next videos in the series will start to define the fundamental technical components for the platforms that bring together a modern application. \n \n Continued in Part 2 \n ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"3731","kudosSumWeight":8,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTE1MzgtMTY0ODVpMThDNzU1MDQxQzJBNDcwQQ?revision=9\"}"}}],"totalCount":1,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[{"__typename":"VideoEdge","cursor":"MHxodHRwczovL3lvdXR1LmJlL1phSklrZzQ4VVdVfDB8MjU7MjV8fA","node":{"__typename":"AssociatedVideo","videoTag":{"__typename":"VideoTag","vid":"https://youtu.be/ZaJIkg48UWU","thumbnail":"https://i.ytimg.com/vi/ZaJIkg48UWU/hqdefault.jpg","uploading":false,"height":113,"width":200,"title":null},"videoAssociationType":"INLINE_BODY"}},{"__typename":"VideoEdge","cursor":"MHxodHRwczovL3lvdXR1LmJlL3FtRmN6dEFMdU13fDF8MjU7MjV8fA","node":{"__typename":"AssociatedVideo","videoTag":{"__typename":"VideoTag","vid":"https://youtu.be/qmFcztALuMw","thumbnail":"https://i.ytimg.com/vi/qmFcztALuMw/hqdefault.jpg","uploading":false,"height":113,"width":200,"title":null},"videoAssociationType":"INLINE_BODY"}},{"__typename":"VideoEdge","cursor":"MHxodHRwczovL3lvdXR1LmJlL21oMk52cml0TGpNfDJ8MjU7MjV8fA","node":{"__typename":"AssociatedVideo","videoTag":{"__typename":"VideoTag","vid":"https://youtu.be/mh2NvritLjM","thumbnail":"https://i.ytimg.com/vi/mh2NvritLjM/hqdefault.jpg","uploading":false,"height":113,"width":200,"title":null},"videoAssociationType":"INLINE_BODY"}}],"totalCount":3,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:281409":{"__typename":"Conversation","id":"conversation:281409","topic":{"__typename":"TkbTopicMessage","uid":281409},"lastPostingActivityTime":"2019-07-08T01:31:58.000-07:00","solved":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMjQ0M2lBQjA2MUJFNzIwNDFENDE3?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMjQ0M2lBQjA2MUJFNzIwNDFENDE3?revision=1","title":"0151T000003kTv8QAE.png","associationType":"BODY","width":417,"height":112,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktNDAzMGk5QjdDQkFCQjI4RUY2RUNG?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktNDAzMGk5QjdDQkFCQjI4RUY2RUNG?revision=1","title":"0151T000003kTvDQAU.png","associationType":"BODY","width":360,"height":72,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTIxMmlCQTNCN0YyNzBDRTI1NzdB?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTIxMmlCQTNCN0YyNzBDRTI1NzdB?revision=1","title":"0151T000003kTvIQAU.png","associationType":"BODY","width":245,"height":53,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTAyOTlpMUM2NDg5MDBCNUIyRjhBNg?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTAyOTlpMUM2NDg5MDBCNUIyRjhBNg?revision=1","title":"0151T000003kTvNQAU.png","associationType":"BODY","width":487,"height":53,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTQ3MjhpRjU4RDMxQ0EzOTdGRDM3Mw?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTQ3MjhpRjU4RDMxQ0EzOTdGRDM3Mw?revision=1","title":"0151T000003kTvSQAU.png","associationType":"BODY","width":801,"height":475,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTU0M2kwRkYyMzQ2RDVDQzA5OTM0?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTU0M2kwRkYyMzQ2RDVDQzA5OTM0?revision=1","title":"0151T000003kTvXQAU.png","associationType":"BODY","width":943,"height":73,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktOTkzaUU5RkYzNzI5OUFGOEI1RDA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktOTkzaUU5RkYzNzI5OUFGOEI1RDA?revision=1","title":"0151T000003kTvcQAE.png","associationType":"BODY","width":741,"height":250,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktODAyMmk2ODAzM0ZGQzczMkM0N0M4?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktODAyMmk2ODAzM0ZGQzczMkM0N0M4?revision=1","title":"0151T000003kTvhQAE.png","associationType":"BODY","width":187,"height":54,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTA4MzhpNDAxOEZERTIzNTdCNjY4OQ?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTA4MzhpNDAxOEZERTIzNTdCNjY4OQ?revision=1","title":"0151T000003kTvmQAE.png","associationType":"BODY","width":350,"height":601,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTg1MmlERThGNTZDQjlERkNCNjM4?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTg1MmlERThGNTZDQjlERkNCNjM4?revision=1","title":"0151T000003kTvrQAE.png","associationType":"BODY","width":456,"height":113,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktOTg0MmlEMjdDM0RGRjc0QTU0QTA3?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktOTg0MmlEMjdDM0RGRjc0QTU0QTA3?revision=1","title":"0151T000003kTv9QAE.png","associationType":"BODY","width":352,"height":607,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktOTE2Mmk3MDUxQkE2MjEwODBBOURE?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktOTE2Mmk3MDUxQkE2MjEwODBBOURE?revision=1","title":"0151T000003kTvdQAE.png","associationType":"BODY","width":456,"height":90,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktNzUxMmlGNTgyNTZDMDEzQzg1NTlC?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktNzUxMmlGNTgyNTZDMDEzQzg1NTlC?revision=1","title":"0151T000003kTvnQAE.png","associationType":"BODY","width":382,"height":74,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMjAyMWlBQjVBOTM2MzlEMkE3RTY4?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMjAyMWlBQjVBOTM2MzlEMkE3RTY4?revision=1","title":"0151T000003kTvTQAU.png","associationType":"BODY","width":490,"height":283,"altText":null},"TkbTopicMessage:message:281409":{"__typename":"TkbTopicMessage","subject":"Exploring Kubernetes API using Wireshark part 3: Python Client API","conversation":{"__ref":"Conversation:conversation:281409"},"id":"message:281409","revisionNum":1,"uid":281409,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:115030"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":3334},"postTime":"2019-07-08T01:31:58.000-07:00","lastPublishTime":"2019-07-08T01:31:58.000-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Quick Intro In this article, we continue our exploration of Kubernetes API but this time we're going to use Python along with Wireshark. I strongly advise you to go through Exploring Kubernetes API using Wireshark part 1: Creating, Listing and Deleting Pods and Exploring Kubernetes API using Wireshark part 2: Namespaces first. We're not creating any app here. I'll just show you how you can explore Kubernetes API using Python's client API and Wireshark output retrieved from kubectl get pods command: In case you want to follow along, I've set up gcloud tool as I'm using Google Cloud and GOOGLE_APPLICATION_CREDENTIALS environment variable on my Mac so I had no problem authenticating to Kubernetes API. I hope you understand that when I say Kubernetes API, I'm talking about the API on Kubernetes Master node on my Google Cloud's Kubernetes cluster. What I'm going to do here In this article, we're going to do the following (using Python): Authenticate to kube API Connect to /api/v1 and then move to /api/v1/namespaces/default/pods Add pod information to a variable named pods from variable pods, create a list of: pod names pod status At the end, we display the above info Authenticating/Authorising your code to contact Kube API Here we're loading auth and cluster info from kube-config file and storing into into our client API config: Connecting to /api/v1 We now store into v1 variable the API object. For the sake of simplicity, think of it as we were pointing to /api/v1/ folder and we haven't decided where to go yet: Connecting to /api/v1/namespaces/default/pods We now move to /api/v1/namespaces/default/pods which where we find information about pods that belong to default namespace (watch=False just means we will not be monitoring for changes, we're just retrieving it as one-off thing): Now, what we've stored in the above variable (ret) is the equivalent of moving to the root directory in our JSON tree (Object) in the output below: The output above was from kubectl get pods command which lists all pods from default namespace and that's equivalent to what we're doing here using Python's Kubernetes Client API. Python shows us similar options about where to go when I type ret. and hit tab: We've got kind, apiVersion, metadata and items as options to move to but items is what we're looking for because it's supposed to store pod's information. So let's move to items and store it in another variable called pods to make it easier to remember that it actually holds all pods' information from default namespace: Listing pods' names We're now in a comfortable place and ready to play! Let's keep pods variable as our placeholder! On Wireshark, pod's name is located in metadata.name: \\ So let's create another variable called pods_names to store all pods' names: Listing Pods' status (phase) What we're looking for is in status.phase: Let's store it in status variable in Python: Displaying the output We can now display the output using built-in zip() method: If you want something pretty you can use prettytable: ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"3324","kudosSumWeight":1,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMjQ0M2lBQjA2MUJFNzIwNDFENDE3?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktNDAzMGk5QjdDQkFCQjI4RUY2RUNG?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTIxMmlCQTNCN0YyNzBDRTI1NzdB?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTAyOTlpMUM2NDg5MDBCNUIyRjhBNg?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTQ3MjhpRjU4RDMxQ0EzOTdGRDM3Mw?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTU0M2kwRkYyMzQ2RDVDQzA5OTM0?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktOTkzaUU5RkYzNzI5OUFGOEI1RDA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDg","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktODAyMmk2ODAzM0ZGQzczMkM0N0M4?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDk","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTA4MzhpNDAxOEZERTIzNTdCNjY4OQ?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEw","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMTg1MmlERThGNTZDQjlERkNCNjM4?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEx","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktOTg0MmlEMjdDM0RGRjc0QTU0QTA3?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEy","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktOTE2Mmk3MDUxQkE2MjEwODBBOURE?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDEz","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktNzUxMmlGNTgyNTZDMDEzQzg1NTlC?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE0","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODE0MDktMjAyMWlBQjVBOTM2MzlEMkE3RTY4?revision=1\"}"}}],"totalCount":14,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:293315":{"__typename":"Conversation","id":"conversation:293315","topic":{"__typename":"TkbTopicMessage","uid":293315},"lastPostingActivityTime":"2022-03-17T15:31:43.597-07:00","solved":false},"User:user:275866":{"__typename":"User","uid":275866,"login":"Terence_Kam","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0yNzU4NjYtMTY5MjFpQkYxOUNFNUQ0OENGREMwNw"},"id":"user:275866"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTMzMTUtMTcwMTdpNEEwOTA4MDkzRTUyOERDQw?revision=5\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTMzMTUtMTcwMTdpNEEwOTA4MDkzRTUyOERDQw?revision=5","title":"fabio-oyXis2kALVg-unsplash.jpg","associationType":"COVER","width":2048,"height":1536,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTMzMTUtMTY5MjRpQTc0NkZBMzhDMDI5ODY2Qg?revision=5\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTMzMTUtMTY5MjRpQTc0NkZBMzhDMDI5ODY2Qg?revision=5","title":"Terence_Kam_0-1646757343460.png","associationType":"BODY","width":997,"height":190,"altText":null},"TkbTopicMessage:message:293315":{"__typename":"TkbTopicMessage","subject":"Integrating with your IPv6 Kubernetes Cluster","conversation":{"__ref":"Conversation:conversation:293315"},"id":"message:293315","revisionNum":5,"uid":293315,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:275866"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":3100},"postTime":"2022-03-17T15:25:30.465-07:00","lastPublishTime":"2022-03-17T15:31:43.597-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" \n Why important to focus on IPv6 \n IPv6 was first introduced as a standard in 1995, yet it is only in the last five years that adoption has accelerated due to the growing need to address the limitations of IPv4. In even some countries like Japan, the pool of IPv4 addresses has long been considered exhausted and many utilize NAT architectures to buy time. \n With the explosion of cloud-based applications and IoT, not even Kubernetes can escape the call for IPv6 support and all the caveats that come with it. For network engineers, the IPv6 is well known. However, one can argue that the container world is still very new to the protocol. Fortunately, this is changing and arguably with more speed now that dual-stack IPv4/IPv6 networking has reached general availability (GA) in Kubernetes v1.23 and single-stack IPv6 alpha support in Istio v1.9. \n IPv6 changes everything in Kubernetes \n When deploying IPv6-only clusters, your pod and service subnets will talk to each other on 128-bit addresses from a block of IPs you defined (default /122) and will therefore need NAT64 and DNS64 in place when calling to IPv4 only services such as DockerHub, GitHub, and other package libraries sitting internally or on the Internet. This also applies to libraries required when installing pre-requisites for K8S. Your CNI configuration, for example Calico, will also need specific variables set to enable IPv6. https://projectcalico.docs.tigera.io/networking/ipv6 \n Once you have your cluster set up, you can then focus on all the hard parts of IPv6 integration. For example, your BGP routers will need to ensure proper configuration for IPv6 peers, and everything will need to allow neighbor discovery (or NDP) to succeed or else you will fail to achieve the simplest of tests — the ICMP ping. All your containerized applications will also need to support communicating on IPv6 sockets. This seems to be the biggest challenge with most applications that are still migrating from the IPv4 world. \n Fortunately, F5 has made the ingress part easier. \n How can BIG-IP integrate with your IPv6-only cluster for reverse proxy and security services? \n IPv6 support is available for many features of Container Ingress Services (CIS) as well as F5 IPAM Controller to meet your needs for exposing your K8S workloads in an easy and secure way. \n BIG-IP admin IP address \n In versions prior to 2.6.0, you had to define a hostAliases block for your IPv6 BIG-IP URL (--bigip-url=bigip01). \n hostAliases:\n - hostnames:\n - bigip01\n ip: aaa1:bbb2:ccc3:ddd4::100 \n Now you can use either of below formats: \n #IPv6 URL with non-standard port\n--bigip-url=https://[aaa1:bbb2:ccc3:ddd4::100]:8443\n \n#IPv6 address\n--bigip-url=[aaa1:bbb2:ccc3:ddd4::100] \n VirtualServer, TransportServer CRD \n When deploying your custom resource, you simply specify an IPv6 address (virtualServerAddress) that will be the external IP exposing your workloads. \n apiVersion: \"cis.f5.com/v1\"\nkind: TransportServer\nmetadata:\n name: simple-virtual-l4\n labels:\n f5cr: \"true\"\nspec:\n virtualServerAddress: \"aaa1:bbb2:ccc3:eee5::200\"\n virtualServerPort: 80\n mode: performance\n snat: auto\n pool:\n service: nginx\n servicePort: 8080\n monitor:\n type: tcp\n interval: 10\n timeout: 10\n---\n \napiVersion: \"cis.f5.com/v1\"\nkind: VirtualServer\nmetadata:\n name: l7-virtual-http\n labels:\n f5cr: \"true\"\nspec:\n # This is an insecure virtual, Please use TLSProfile to secure the virtual\n # check out tls examples to understand more.\n host: expo.example.com\n virtualServerAddress: \"aaa1:bbb2:ccc3:eee5::201\"\n virtualServerName: \"l7-virtual-http\"\n pools:\n - path: /expo\n service: svc-2\n servicePort: 80 \n Service Type LoadBalancer (with F5 IPAM Controller) \n You can also simulate the one-click public cloud experience when exposing your workloads by simply specifying your service to be of type LoadBalancer. The listener IP allocation and configuration is automagically done for you with the help of F5 IPAM Controller. \n In F5 IPAM Controller deployment manifest, just specify your IPv6 range in the container args like below: \n - '{\"Dev\":\"2001:db8:3::7-2001:db8:3::9\",\"Test\":\"2001:db8:4::7-2001:db8:4::9\"}' \n Then deploy your F5 IPAM Controller and if you see addresses in the logs like below, you're good to go. \n 2022/03/08 15:34:31 [DEBUG] Created New IPAM Client\n2022/03/08 15:34:31 [DEBUG] [STORE] Using IPAM DB file from mount path\n2022/03/08 15:34:31 [DEBUG] [STORE] [ipaddress status ipam_label reference]\n2022/03/08 15:34:31 [DEBUG] [STORE] 2001:db8:4::7 1 Test ZpTSxCKs0gigByk5\n2022/03/08 15:34:31 [DEBUG] [STORE] 2001:db8:4::8 1 Test 650YpEeEBF2H88Z8\n2022/03/08 15:34:31 [DEBUG] [STORE] 2001:db8:4::9 1 Test la9aJTZ5Ubqi/2zU \n Note: For the CIS configuration, ensure that your CIS deployment manifest defines below parameters, which are particularly for service type LoadBalancer and IPv6 use case. \n \"--custom-resource-mode=true\",\n\"--ipam=true\",\n\"--enable-ipv6=true\", \n Then you can deploy a simple Service resource like below: \n apiVersion: v1\nkind: Service\nmetadata:\n annotations:\n cis.f5.com/health: '{\"interval\": 5, \"timeout\": 10}'\n cis.f5.com/ipamLabel: Test\n labels:\n app: svc-f5-demo-lb1\n name: svc-f5-demo-lb1\n namespace: default\nspec:\n ports:\n - name: svc-lb1-80\n port: 80\n protocol: TCP\n targetPort: 80\n selector:\n app: f5-demo\n type: LoadBalancer \n At which point the magic happens and you see that your \"EXTERNAL-IP\" is populated with an IP address allocated by F5 IPAM Controller. \n # kubectl get svc svc-f5-demo-lb1\nNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE\nsvc-f5-demo-lb1 LoadBalancer 172.19.2.55 2001:db8:4::4 80:32410/TCP 43m \n On the BIG-IP you will see the virtual server created as defined in your Service object. \n \n Summary \n Moving to a single-stack IPv6 Kubernetes cluster can be difficult and requires a thorough review of all your application components as well as the parts that integrate with your cluster. \n F5 has ensured that even with a pure IPv6 environment you are covered with an enterprise grade ingress solution that not only simplifies exposing of your workloads, but also provides many security options as well. \n ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"6310","kudosSumWeight":2,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTMzMTUtMTcwMTdpNEEwOTA4MDkzRTUyOERDQw?revision=5\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuM3wyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTMzMTUtMTY5MjRpQTc0NkZBMzhDMDI5ODY2Qg?revision=5\"}"}}],"totalCount":2,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"CachedAsset:text:en_US-components/community/Navbar-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/community/Navbar-1744046271000","value":{"community":"Community Home","inbox":"Inbox","manageContent":"Manage Content","tos":"Terms of Service","forgotPassword":"Forgot Password","themeEditor":"Theme Editor","edit":"Edit Navigation Bar","skipContent":"Skip to content","migrated-link-9":"Groups","migrated-link-7":"Technical Articles","migrated-link-8":"DevCentral News","migrated-link-1":"Technical Forum","migrated-link-10":"Community Groups","migrated-link-2":"Water Cooler","migrated-link-11":"F5 Groups","Common-external-link":"How Do I...?","migrated-link-0":"Forums","article-series":"Article Series","migrated-link-5":"Community Articles","migrated-link-6":"Articles","security-insights":"Security Insights","migrated-link-3":"CrowdSRC","migrated-link-4":"CodeShare","migrated-link-12":"Events","migrated-link-13":"Suggestions"},"localOverride":false},"CachedAsset:text:en_US-components/community/NavbarHamburgerDropdown-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/community/NavbarHamburgerDropdown-1744046271000","value":{"hamburgerLabel":"Side Menu"},"localOverride":false},"CachedAsset:text:en_US-components/community/BrandLogo-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/community/BrandLogo-1744046271000","value":{"logoAlt":"Khoros","themeLogoAlt":"Brand Logo"},"localOverride":false},"CachedAsset:text:en_US-components/community/NavbarTextLinks-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/community/NavbarTextLinks-1744046271000","value":{"more":"More"},"localOverride":false},"CachedAsset:text:en_US-components/authentication/AuthenticationLink-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/authentication/AuthenticationLink-1744046271000","value":{"title.login":"Sign In","title.registration":"Register","title.forgotPassword":"Forgot Password","title.multiAuthLogin":"Sign In"},"localOverride":false},"CachedAsset:text:en_US-components/nodes/NodeLink-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/nodes/NodeLink-1744046271000","value":{"place":"Place {name}"},"localOverride":false},"CachedAsset:text:en_US-components/tags/TagSubscriptionAction-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/tags/TagSubscriptionAction-1744046271000","value":{"success.follow.title":"Following Tag","success.unfollow.title":"Unfollowed Tag","success.follow.message.followAcrossCommunity":"You will be notified when this tag is used anywhere across the community","success.unfollowtag.message":"You will no longer be notified when this tag is used anywhere in this place","success.unfollowtagAcrossCommunity.message":"You will no longer be notified when this tag is used anywhere across the community","unexpected.error.title":"Error - Action Failed","unexpected.error.message":"An unidentified problem occurred during the action you took. Please try again later.","buttonTitle":"{isSubscribed, select, true {Unfollow} false {Follow} other{}}","unfollow":"Unfollow"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/QueryHandler-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/QueryHandler-1744046271000","value":{"title":"Query Handler"},"localOverride":false},"Category:category:top":{"__typename":"Category","id":"category:top","nodeType":"category"},"CachedAsset:text:en_US-components/community/NavbarDropdownToggle-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/community/NavbarDropdownToggle-1744046271000","value":{"ariaLabelClosed":"Press the down arrow to open the menu"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageListTabs-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageListTabs-1744046271000","value":{"mostKudoed":"{value, select, IDEA {Most Votes} other {Most Likes}}","mostReplies":"Most Replies","mostViewed":"Most Viewed","newest":"{value, select, IDEA {Newest Ideas} OCCASION {Newest Events} other {Newest Topics}}","newestOccasions":"Newest Events","mostRecent":"Most Recent","noReplies":"No Replies Yet","noSolutions":"No Solutions Yet","solutions":"Solutions","mostRecentUserContent":"Most Recent","trending":"Trending","draft":"Drafts","spam":"Spam","abuse":"Abuse","moderation":"Moderation","tags":"Tags","PAST":"Past","UPCOMING":"Upcoming","sortBymostRecent":"Sort By Most Recent","sortBymostRecentUserContent":"Sort By Most Recent","sortBymostKudoed":"Sort By Most Likes","sortBymostReplies":"Sort By Most Replies","sortBymostViewed":"Sort By Most Viewed","sortBynewest":"Sort By Newest Topics","sortBynewestOccasions":"Sort By Newest Events","otherTabs":" Messages list in the {tab} for {conversationStyle}","guides":"Guides","archives":"Archives"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageView/MessageViewInline-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageView/MessageViewInline-1744046271000","value":{"bylineAuthor":"{bylineAuthor}","bylineBoard":"{bylineBoard}","anonymous":"Anonymous","place":"Place {bylineBoard}","gotoParent":"Go to parent {name}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/Pager/PagerLoadMore-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/Pager/PagerLoadMore-1744046271000","value":{"loadMore":"Show More"},"localOverride":false},"CachedAsset:text:en_US-components/customComponent/CustomComponent-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/customComponent/CustomComponent-1744046271000","value":{"errorMessage":"Error rendering component id: {customComponentId}","bannerTitle":"Video provider requires cookies to play the video. Accept to continue or {url} it directly on the provider's site.","buttonTitle":"Accept","urlText":"watch"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/OverflowNav-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/OverflowNav-1744046271000","value":{"toggleText":"More"},"localOverride":false},"CachedAsset:text:en_US-components/users/UserLink-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/users/UserLink-1744046271000","value":{"authorName":"View Profile: {author}","anonymous":"Anonymous"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageSubject-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageSubject-1744046271000","value":{"noSubject":"(no subject)"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageBody-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageBody-1744046271000","value":{"showMessageBody":"Show More","mentionsErrorTitle":"{mentionsType, select, board {Board} user {User} message {Message} other {}} No Longer Available","mentionsErrorMessage":"The {mentionsType} you are trying to view has been removed from the community.","videoProcessing":"Video is being processed. Please try again in a few minutes.","bannerTitle":"Video provider requires cookies to play the video. Accept to continue or {url} it directly on the provider's site.","buttonTitle":"Accept","urlText":"watch"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageTime-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageTime-1744046271000","value":{"postTime":"Published: {time}","lastPublishTime":"Last Update: {time}","conversation.lastPostingActivityTime":"Last posting activity time: {time}","conversation.lastPostTime":"Last post time: {time}","moderationData.rejectTime":"Rejected time: {time}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/nodes/NodeIcon-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/nodes/NodeIcon-1744046271000","value":{"contentType":"Content Type {style, select, FORUM {Forum} BLOG {Blog} TKB {Knowledge Base} IDEA {Ideas} OCCASION {Events} other {}} icon"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageUnreadCount-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageUnreadCount-1744046271000","value":{"unread":"{count} unread","comments":"{count, plural, one { unread comment} other{ unread comments}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageViewCount-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageViewCount-1744046271000","value":{"textTitle":"{count, plural,one {View} other{Views}}","views":"{count, plural, one{View} other{Views}}"},"localOverride":false},"CachedAsset:text:en_US-components/kudos/KudosCount-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/kudos/KudosCount-1744046271000","value":{"textTitle":"{count, plural,one {{messageType, select, IDEA{Vote} other{Like}}} other{{messageType, select, IDEA{Votes} other{Likes}}}}","likes":"{count, plural, one{like} other{likes}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageRepliesCount-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageRepliesCount-1744046271000","value":{"textTitle":"{count, plural,one {{conversationStyle, select, IDEA{Comment} OCCASION{Comment} other{Reply}}} other{{conversationStyle, select, IDEA{Comments} OCCASION{Comments} other{Replies}}}}","comments":"{count, plural, one{Comment} other{Comments}}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/users/UserAvatar-1744046271000":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/users/UserAvatar-1744046271000","value":{"altText":"{login}'s avatar","altTextGeneric":"User's avatar"},"localOverride":false}}}},"page":"/tags/TagPage/TagPage","query":{"messages.widget.messagelistfornodebyrecentactivitywidget-tab-main-messages-list-for-tag-widget-0":"mostViewed","tagName":"kubernetes"},"buildId":"-gVUpXaWnPcjlrLJZ92B7","runtimeConfig":{"buildInformationVisible":false,"logLevelApp":"info","logLevelMetrics":"info","openTelemetryClientEnabled":false,"openTelemetryConfigName":"f5","openTelemetryServiceVersion":"25.3.0","openTelemetryUniverse":"prod","openTelemetryCollector":"http://localhost:4318","openTelemetryRouteChangeAllowedTime":"5000","apolloDevToolsEnabled":false,"inboxMuteWipFeatureEnabled":false},"isFallback":false,"isExperimentalCompile":false,"dynamicIds":["./components/customComponent/CustomComponent/CustomComponent.tsx","./components/community/Navbar/NavbarWidget.tsx","./components/community/Breadcrumb/BreadcrumbWidget.tsx","./components/tags/TagsHeaderWidget/TagsHeaderWidget.tsx","./components/messages/MessageListForNodeByRecentActivityWidget/MessageListForNodeByRecentActivityWidget.tsx","./components/tags/TagSubscriptionAction/TagSubscriptionAction.tsx","./components/customComponent/CustomComponentContent/TemplateContent.tsx","../shared/client/components/common/List/ListGroup/ListGroup.tsx","./components/messages/MessageView/MessageView.tsx","./components/messages/MessageView/MessageViewInline/MessageViewInline.tsx","../shared/client/components/common/Pager/PagerLoadMore/PagerLoadMore.tsx","./components/customComponent/CustomComponentContent/HtmlContent.tsx","./components/customComponent/CustomComponentContent/CustomComponentScripts.tsx"],"appGip":true,"scriptLoader":[]}