Knowledge sharing: Containers, Kubernetes, Openshift, F5 Container Connector, NGINX Ingress
For anyone interested about the free traning for "F5 Container Connector for Kubernetes" or "F5 OpenShift Container Integration" at "LearnF5". For NGINX being installed in Kubernetes there is enough info but for F5 Contaner Connector/Container Ingress Services there is not so much: https://docs.nginx.com/nginx-ingress-controller/f5-ingresslink/ https://www.nginx.com/products/nginx-ingress-controller/ https://community.f5.com/t5/technical-articles/better-together-f5-container-ingress-services-and-nginx-plus/ta-p/280471 F5 Devcentral also has youtube channel with usefull info: https://www.youtube.com/c/devcentral If you don't have good knowledge about containers and kubernetes then first check the links below. For Docker containers in youtube you will find a lot of good training for example: you need to learn Kubernetes RIGHT NOW!! - YouTube Docker Tutorial for Beginners [FULL COURSE in 3 Hours] - YouTube Docker overview | Docker Documentation The same is true for Kubernetes and they have a free test lab on their site: Learn Kubernetes Basics | Kubernetes you need to learn Docker RIGHT NOW!! // Docker Containers 101 - YouTube Red Hat has some free training and IBM provides some free labs for Containers, Kubernetes, Openshift etc.: Training and Certification (redhat.com) IBM CloudLabs: Free, Interactive Kubernetes Tutorials | IBM Red Hat OpenShift Tutorials | IBM
Exploring Kubernetes API using Wireshark part 1: Creating, Listing and Deleting Pods
Related Articles: Exploring Kubernetes API using Wireshark part 2: Namespaces Exploring Kubernetes API using Wireshark part 3: Python Client API Quick Intro This article answers the following question: What happens when we create, list and delete pods under the hood? More specifically on the wire. I used these 3 commands: I'll show you on Wireshark the communication between kubectl client and master node (API) for each of the above commands. I used a proxy so we don't have to worry about TLS layer and focus on HTTP only. Creating NGINX pod pcap: creating_pod.pcap (use http filter on Wireshark) Here's our YAML file: Here's how we create this pod: Here's what we see on Wireshark: Behind the scenes, kubectl command sent an HTTP POST with our YAML file converted to JSON but notice the same thing was sent (kind, apiVersion, metadata, spec): You can even expand it if you want to but I didn't to keep it short. Then, Kubernetes master (API) responds with HTTP 201 Created to confirm our pod has been created: Notice that master node replies with similar data with the additional status column because after pod is created it's supposed to have a status too. Listing Pods pcap: listing_pods.pcap (use http filter on Wireshark) When we list pods, kubectl just sends a HTTP GET request instead of POST because we don't need to submit any data apart from headers: This is the full GET request: And here's the HTTP 200 OK with JSON file that contains all information about all pods from default's namespace: I just wanted to emphasise that when you list a pod the resource type that comes back is PodList and when we created our pod it was just Pod. Remember? The other thing I'd like to point out is that all of your pods' information should be listed under items. All kubectl does is to display some of the API's info in a humanly readable way. Deleting NGINX pod pcap: deleting_pod.pcap (use http filter on Wireshark) Behind the scenes, we're just sending an HTTP DELETE to Kubernetes master: Also notice that the pod's name is also included in the URI: /api/v1/namespaces/default/pods/nginx ← this is pods' name HTTP DELETE just like HTTP GET is pretty straightforward: Our master node replies with HTTP 200 OK as well as some json file with all the info about the pod, including about it's termination: It's also good to emphasise here that when our pod is deleted, master node returns JSON file with all information available about the pod. I highlighted some interesting info. For example, resource type is now just Pod (not PodList when we're just listing our pods).What is Kubernetes?
Kubernetes is a container-orchestration platform. Its goal is to abstract away the complexity to run containerised applications in terms of network, storage, scaling and more. It also provides a declarative REST API (which is extensible) in order to automate the process of application hosting and exposure. If that sounds confusing, think of it as the thing that abstracts your infrastructure. We no longer have to worry about servers but only how to deploy our application to Kubernetes. This is how a Kubernetes cluster may look like: It is comprised of a cluster of physical servers or virtual machines known as nodes in Kubernetes world. We can add or remove nodes at will and Kubernetes can scale down or up to a staggering amount of up to 5,000 nodes! Master nodes vs Worker Nodes There are 2 kinds of nodes you should initially know about: master and worker nodes¹. ¹ OpenShift (an enterprise fork of Kubernetes) adds the notion of infrastructure node. Infrastructure nodes are meant to host shared services (e.g. router nodes, monitoring, etc). Master nodes manage the Kubernetes cluster using 4 main components: Scheduler schedules pods to worker nodes. Controller manager makes sure cluster's actual state = desired state. ETCD is where Kubernetes store its objects and metadata. API Server validates objects before they're stored in ETCD and of course is the central point of contact for object creation, retrieval and to watch the state of objects and cluster in general. A popular tool to "talk" to API Server is kubectl. If you install Kubernetes, you would definitely use kubectl². ² Openshift has a similar tool called "oc" Worker nodes communicate with Master node's API Server in the following manner: Kubelet runs on each worker node and watches API SERVER to continuously monitor for Pods that should be created, deleted or changed. When we first add a Node to Kubernetes cluster, Kubelet is the daemon that registers the Node resource to API Server. Kube-proxy makes sure client traffic is redirected to the correct pod networking-wise in an efficient manner. Redirection is accomplished by using either iptables rules or IPVS virtual servers. Container runtime is usually Docker. Pods and Containers: Where does a Kubernetes application reside? Not every application is compatible with Kubernetes environment. Developers have to create their application in a specific way with small replicable components (also known as micro-services) that are independent from other components. Such components are hosted inside of a Pod. Pods run in Worker nodes: Within pods we can find one or more containers and that's where our application (or small chunk of our application) resides. In Appendix section, I will explain why using pods instead of containers directly. Understanding Pod's scalability component Pods are supposed to be replicable so application is designed in such a way to enable horizontal (auto) scalability. That's one of the powers of Kubernetes! We have a cluster of nodes where chunks of our application (pods) can easily increase or decrease in numbers. This is also the reason why our pods should be coded in a way that allows them to be replicable. Imagine our application has a component called shopping-trolley and another one called check-out: Our shopping-trolley pods may eventually become too overloaded and we might need more replicas to cope with the additional traffic/load. Increasing/reducing the number of replicas is as easy as writing down the number of replicas or letting cloud providers auto-scale it for you. Cloud providers also allow us to increase replicas based on CPU cycles, memory, etc. Before the rise of container orchestrators like Kubernetes, we would have to scale out the whole application stack, unnecessarily overloading servers. Kubernetes also allows us to scale out only parts of the application that needs it, effectively reducing unnecessary overload on servers and costs. The other advantage is that we can upgrade part of our application with zero downtime without the overhead to re-deploy the whole application at once. Services: how traffic reaches Application within a Pod Scheduler spreads out pods throughout Kubernetes cluster. However, it is usually a good idea to group pod replicas behind a single entry-point for reachability purposes as a pod's IP may change. This is where a Kubernetes Services comes in: Services act as the single point of access for a groups of pods with fixed DNS name and port. The way Services work out which pod should belong to it is by the use of labels. There is a label selector on a service and pods with same label are grouped into the Service. Understanding the 3 ways Services can be exposed ClusterIP Services can be exposed internally when one group of pods want to communicate with another one. This is the default and is called ClusterIP type. A private IP address reachable only within Kubernetes cluster is used as single point of access for the group of pods. NodePort Services can be exposed externally by using Node's public IP address and port as cluster's entry point for client's external traffic. This is called NodePort type. If we use NodePort, external clients would have to directly reach one of the nodes so NodePort might not be suitable for most production environments. If we need to load balance traffic among nodes, the next type is the solution. LoadBalancer This is another layer on top of NodePort that load balances traffic in a round robin fashion to all nodes. However, the LoadBalancer type can only be tied to a single Service, i.e. if we have multiple Services, we would need one LoadBalancer per Service which could become quite costly. If we want to use a single Public IP address to direct external traffic to the right service based on URL, the next type is the solution. Ingress Resource The Ingress type reads the HTTP Host header and forwards connection to a Service based on the URL/PATH. An Ingress can point to multiple Services based on URL using a single Public IP address as entry point. This overcomes the limitation of one LoadBalancer per Service on LoadBalancer type. Final Remarks Kubernetes is currently a well-established DevOps tool but it is a very extensive topic and is constantly evolving. For release update, please watch Kubernetes official blog. There are many Kubernetes objects that were not covered here but should be covered in a future article. Appendix: Why Pods? Why not using containers directly? Design The underlying Container technology is independent from Kubernetes. Pods act as a layer of abstraction on top of it. With that in mind, Kubernetes doesn't really have to adapt to different container technology (such as Docker, Rocket, etc) and avoids runtime lock-in, i.e. each container runtime has its own strength. Application Requirements Within pods, containers can potentially share resources more easily. For example, one container might run to perform a certain task and the other one to take care of authentication. Another example would be one container writing to a shared storage volume and another one reading from it to perform additional processing. Containers in the same pod share the same network and IPC namespaces.Introducing the New Docker Compose Installation Option for F5 NGINX Instance Manager
F5 NGINX Instance Manager (NIM) is a centralised management solution designed to simplify the administration and monitoring of F5 NGINX instances across various environments, including on-premises, cloud, and hybrid infrastructures. It provides a single interface to efficiently oversee multiple NGINX instances, making it particularly useful for organizations using NGINX at scale. We’re excited to introduce a new Docker Compose installation option for NGINX Instance Manager, designed to help you get up and running faster than ever before, in just a couple of steps. Key Features: Quick and Easy Installation: With just a couple of steps, you can pull and deploy NGINX Instance Manager on any Docker host, without having to manually configure multiple components. The image is available in our container registry, so once you have a valid license to access it, getting up and running is as simple as pulling the container. Fault-Tolerant and Resilient: This installation option is designed with fault tolerance in mind. Persistent storage ensures your data is safe even in the event of container restarts or crashes. Additionally, with a separate database container, your product’s data is isolated, adding an extra layer of resilience and making it easier to manage backups and restores. Seamless Upgrades: Upgrades are a breeze. You can update to the latest version of NGINX Instance Manager by simply updating the image tag in your Docker Compose file. This makes it easy to stay up-to-date with the latest features and improvements without worrying about downtime or complex upgrade processes. Backup and Restore Options: To ensure your data is protected, this installation option comes with built-in backup and restore capabilities. Easily back up your data to a safe location and restore it in case of any issues. Environment Configuration Flexibility: The Docker Compose setup allows you to define custom environment variables, giving you full control over configuration settings such as log levels, timeout values, and more. Production-Ready: Designed for scalability and reliability, this installation method is ready for production environments. With proper resource allocation and tuning, you can deploy NGINX Instance Manager to handle heavy workloads while maintaining performance. The following steps walk you through how to deploy and manage NGINX Instance Manager using Docker Compose. What you need A working version of Docker Your NGINX subscription’s JSON Web Token from MyF5 This pre-configured docker-compose.yaml file: Download docker-compose.yaml file . Step 1 - Set up Docker for NGINX container registry Log in to the Docker registry using the contents of the JSON Web Token file you downloaded from MyF5 : docker login private-registry.nginx.com --username=<JWT_CONTENTS> --password=none Step 2 - Run “docker login” and then “docker compose up” in the directory where you downloaded docker-compose.yaml Note: You can optionally set the Administrator password for NGINX Instance Manager prior to running Docker Compose. ~$ docker login private-registry.nginx.com --username=<JWT_CONTENTS> --password=none ~$ echo "admin" > admin_password.txt ~$ docker compose up -d [+] Running 6/6 ✔ Network nim_clickhouse Created 0.1s ✔ Network nim_external_network Created 0.2s ✔ Network nim_default Created 0.2s ✔ Container nim-precheck-1 Started 0.8s ✔ Container nim-clickhouse-1 Healthy 6.7s ✔ Container nim-nim-1 Started 7.4s. Step 3 – Access NGINX Instance Manager Go to the NGINX Instance Manager UI on https://<<DOCKER_HOST>>:443 and license the product using the same JSON Web Token you downloaded from MyF5 earlier. Conclusion With this new setup, you can install and run NGINX Instance Manager on any Docker host in just 3 steps, dramatically reducing setup time and simplifying deployment. Whether you are working in a development environment or deploying to production, the Docker Compose-based solution ensures a seamless and reliable experience. For more information on using the docker compose option with NGINX Instance manager such as running a backup and restore, using secrets, and many more, please see the instructions here.Exploring Kubernetes API using Wireshark part 2: Namespaces
Related Articles: Exploring Kubernetes API using Wireshark part 1: Creating, Listing and Deleting Pods Exploring Kubernetes API using Wireshark part 3: Python Client API Quick Intro Using kubectl command is pretty useful: When you execute the above command, kubectl sends a GET request to /api/v1/namespaces/default/pods: Kubernetes master node replies with a JSON file containing all pods (along with their info) that belong to namespace 'default'. In this article, I'm going to explain what Kubernetes namespaces are by showing you real HTTP traffic reaching Kubernetes master node. I've removed the TLS complexity by using proxy so we can just focus on HTTP headers only. Understanding namespaces Initially, I'd say just memorise that /api/v1 is like the root directory of Kubernetes master node's API where client is going to retrieve all sorts of information. Have you noticed the namespaces in /api/v1/namespaces/default/pods? default just happens to be the namespace that our pods listed here belong to. Think of namespaces for Kubernetes as virtual Kubernetes clusters just like Virtual Machines (VMs) for OS. We can have identical objects with same name that belong to different namespaces and therefore are isolated from each other from the point of view of the API. Creating a new custom namespace I can create a new namespace like this using kubectl command: I can then create the same identical pods from default namespace in rodrigo's namespace. Let's see what happened under the hood when I typed the above command. When we create a new namespace, kubectl sends an HTTP POST request Kubernetes master node: pcap: creating-rodrigo-namespace.pcap The kubectl client then sends a JSON file like this in the POST request: Then, Kubernetes Master responds with HTTP 201 Created message and another JSON file with all newly created namespace's info: I've described some of the JSON info that came back from API just out of curiosity. Note that many different objects are 'namespaced', i.e. they belong to a namespace. Others like nodes are namespace-independent. I used pods as an example here to explain namespaces as pods are most popular and well-known object in Kubernetes world. Keeping 2 identical pods in 2 namespaces Let me create a new NGINX pod in the new namespace: Ops! We need to specify that we're creating the same pod in the new namespace we've just created, otherwise it defaults to default namespace where the nginx pod already exists: It now worked. Let's list only pods from rodrigo's namespace only with kubectl: When we capture the above request on Wireshark, we now see that our GET request to Kubernetes Master now uses rodrigo's namespace so we're now listing only pods from rodrigo namespace only : We also have this same exact pod using same name in default namespace. Remember? Deleting my custom namespace Now, let's delete our pod: And that's the API call under the hood (an HTTP DELETE request to complete path of namespace - just like we're deleting a folder): pcap: deleting-namespace.pcap Listing pods from all namespaces If you're curious about how the URL would look like when we list pods from all namespaces with kubectl: The answer is this: This request will list all pods from all namespaces. Troubleshooting Namespaces Remember I mentioned the finalizer attribute? When I was creating this article and I tried to delete the my custom namespace (rodrigo), it got stuck in Terminating state: Initially I thought it was just Google Cloud slowness but 40 minutes? That's a lot. So I suspected it could be because of finalize attribute and googled it so found that it was a bug and here's the solution: Retrieve namespace's JSON declaration to temporary file: Delete kubernetes keyword from finalizers attribute: Now send a PUT request to API and the JSON file above: Then, when I looked back it was finally gone:
