Templating Enhanced Kubernetes Load Balancing with a Helm Operator
Basic L4 load balancing only requires a few inputs, IP and Port, but how do provide enhanced load balancing and not overwhelm an operator with hundreds of inputs? Using a helm operator, a Kubernetes automation tool, we can unlock the full potential of a F5 BIG-IP and deliver the right level of service. In the following article we’ll take a look at what is a helm operator and how we can use it to create a service catalog of BIG-IP L4-L7 services that can be deployed natively from Kubernetes. Helm Helm is a tool that is used to automate Kubernetes application and infrastructure. You might use it to deploy a simple application with a deployment and service resource or use it to deploy a service mesh like Istio that contains custom resources, cluster roles, mutating webhooks, pilots, ingress gateways, egress gateways, prometheus, etc.. It’s kinda like Ansible, but for Kubernetes Helm Operator It is helpful to be able to automate via helm; but how do you know that the state of your cluster is consistent? Did somebody go in later and modify your deployment from the original template and create a snowflake? A helm operator are part of the Operator Framework; nannies parents for your Kubernetes services. They ensure that your services get started properly, clean-up when they have an accident, and put the resources to bed at the end of the day. Declarative L4-L7 K8S LB w/ AS3 F5 Container Ingress Services (CIS) (the product formerly known as Container Connector) enables an end-user to deploy a control plane process that monitors the Kubernetes API to deploy load balancer (LB) services when needed removing the need for the traditional change request queue. Version 1.9 of CIS introduces the ability to use Application Services Extension 3 (AS3) to deploy both basic and enhanced L4-L7 services. A basic service might be: L4 TCP L7 HTTP/HTTPS This is similar to what was possible with previous versions of CIS. AS3 introduces the ability to enhance these services with capabilities like Visibility of Client IP with Proxy Protocol End-to-end SSL encryption (including mutual TLS with the use of C3D) L4 and L7 DDoS protection using IP threat feeds and advanced WAF For the basic service this can be represented by a Kubernetes ConfigMap resource that contains a JSON file of the desired output. Something like: An enhanced service looks something more like: Ideally we can create a template for both basic and enhanced services to simplify deployment and ensure that both local policy and best practices are being adhered to. Helm Chart We can create a helm chart (template) that represents both a basic and advanced service. The basic template for TCP looks like: The inputs/values for these templates gets boiled down to a few input parameters. This makes it easier to make changes without having to modify JSON in a text editor! Helm Operators We could use helm to generate static AS3 ConfigMaps, but we can optionally use a helm operator to create a new resource (or service catalog) of values that can dynamically generate AS3 ConfigMaps. Following the guide from the helm operator user guide we can import a helm chart to build a new operator (container) that will monitor the Kubernetes API. In the following I created the “f5demo” operator. Once I install the f5demo custom resource I can query for the resource similar to any native Kubernetes resource like a ConfigMap. node1$ kubectl get f5demo -n ingress-bigip NAME AGE example-f5demo 18m The contents of the resource are the helm values that were used previously. node1$ kubectl get f5demo -n ingress-bigip -o yaml apiVersion: v1 items: - apiVersion: charts.helm.k8s.io/v1alpha1 kind: F5Demo ... spec: applications: - frontend: name: frontend template: f5demo.tcp.v1 virtualAddress: 10.1.10.81 virtualPort: 80 ... The helm operator builds a new ConfigMap based on the input values node1$ kubectl get cm -n ingress-bigip NAME DATA AGE example-f5demo-bzr48kbg2peco4p5g4wc0jy32-as3-configmap 1 20m Building Blocks To recap we’ve looked at using helm to build templates of BIG-IP L4-L7 services using AS3. To ensure day-to-day consistency in a cluster we are using an operator to keep track of the state of a service and make updates as appropriate. These patterns could be deployed in other ways, for example my colleague uses Jenkins and Python to templatize or maybe you’d rather just use Ansible with AS3. My recommendation is to: Figure out what L4-L7 services you need Build an AS3 declaration (JSON) of what you want Use a tool like Helm, Ansible, BIG-IQ, Perl etc... to deliver your infrastructure (as code)
Using F5 BIG-IP Controller Operator for OpenShift
Today A&O PM and PD team announced the availability of Certified F5 BIG-IP Controller Operator (using Helm Charts) on OpenShift 4.x platforms. In this document we discuss about Install, Configure and Deploy CIS using RedHat Certified F5 BIG-IP Controller Operator on OpenShift 4.x Platforms. Introduction What is an Operator? - A method of packaging, deploying and managing a Kubernetes application. A Kubernetes application is an application that is both deployed on Kubernetes and managed using the Kubernetes APIs and kubectl/oc tooling. You can think of Operators as the runtime that manages this type of application on Kubernetes. Conceptually, an Operator takes human operational knowledge and encodes it into software that is more easily packaged and shared with consumers. F5 BIG-IP Controller Operator is a Service Operator which installs F5 BIG-IP Controller (Container Ingress Services) on OpenShift platforms 4.x. Prerequisites OpenShift 4.x BIG-IP (F5 CIS supported versions) In this document we will use Code Ready Containers to install, Configure and deploy CIS using F5 BIG-IP Controller Operator. CRC 1.7.0 installs OCP 4.3.1 on you laptop. Get your suitable image from CRC Repo and follow the instructions to install CRC and bringup your single node OCP 4.3.1 cluster. Install, Configure and Deploy CIS using Operator Accessing OCP 4.3.1 web console From CLI, login as admin using CRC given credentials. $ eval $(crc oc-env) $ oc login -u kubeadmin -p db9Dr-J2csc-8oP78-9sbmf https://api.crc.testing:6443 Here, the username is 'kubeadmin'. and password is 'db9Dr-J2csc-8oP78-9sbmf' to login OCP web console. Installing Operator From the left Menu bar, access Operator Hub and search for "f5" to see the Certified F5 BIG-IP controller Operator in the listing as below. Click on Install to install this Operator. Installing Operator is a guided process. The below screen shows different options to subscribe for this Operator. Select the highlighted options. Click subscribe. Approval Strategy: Manual: Requires administrator approval to install new updates. Automatic: When a new release is available, updated automatic. (default) When Operator is Subscribed, Operator is installed based on approval strategy. An Installed Operator screen is as below. Configuring and Deploying F5 BIG-IP Controller Instance Click on "F5 BIG-IP Controller" or "F5BigIPCtlr" under Provided APIs column to create an Instance of F5 BIG-IP Controller. Creating a F5BigIpCtlr instance screen is as shown below. The Screen provides an editor to configure CIS/F5 BIG-IP Controller with required deployment options. A sample Controller deployment configuration is as shown below apiVersion: cis.f5.com/v1 kind: F5BigIpCtlr metadata: name: f5-server namespace: openshift-operators spec: args: manage_routes: true agent: as3 log_level: DEBUG route_vserver_addr: 172.16.1.4 bigip_partition: ocp openshift_sdn_name: /Common/openshift_vxlan bigip_url: 172.16.2.23 log_as3_response: true insecure: true pool-member-type: cluster bigip_login_secret: f5-bigip-ctlr-login image: pullPolicy: Always repo: k8s-bigip-ctlr user: f5networks namespace: kube-system rbac: create: true resources: {} serviceAccount: create: true version: latest Create BIG-IP controller login secret and update the same in above configuration. Update the YAML and click on Create. Based on Namespace and configuration options, CIS is installed. When Operator deploys the controller, we can see the updated YAML of the CustomResource Instance. An example below. Name: f5-server Namespace: openshift-operators Labels: <none> Annotations: <none> API Version: cis.f5.com/v1 Kind: F5BigIpCtlr Metadata: Creation Timestamp: 2020-02-08T00:31:21Z Finalizers: uninstall-helm-release Generation: 1 Resource Version: 245330 Self Link: /apis/cis.f5.com/v1/namespaces/openshift-operators/f5bigipctlrs/f5-server UID: 546d3890-4a0a-11ea-a1cf-0ef0e3c74fbe spec: args: agent: as3 bigip_partition: ocp bigip_url: 172.16.2.23 insecure: true log_as3_response: true log_level: DEBUG manage_routes: true openshift_sdn_name: /Common/openshift_vxlan pool_member_type: cluster route_vserver_addr: 172.16.1.4 bigip_login_secret: f5-bigip-ctlr-login Image: PullPolicy: Always Repo: k8s-bigip-ctlr Tag: latest User: f5networks Namespace: kube-system Rbac: Create: true Resources: Service Account: Create: true Name: <nil> Status: Conditions: Last Transition Time: 2020-02-08T00:31:21Z Status: True Type: Initialized Last Transition Time: 2020-02-08T00:31:23Z Message: F5 BIG-IP controller: f5-server General Controller Documentation: - Kubernetes: http://clouddocs.f5.com/containers/latest/kubernetes/index.html - OpenShift: http://clouddocs.f5.com/containers/latest/openshift/index.html Using Ingress? There's a helm chart for that: - https://github.com/F5Networks/charts/tree/master/src/stable/f5-bigip-ingress Using Routes in OpenShift? No helm chart yet, but we do have great documentation: - http://clouddocs.f5.com/containers/latest/openshift/kctlr-openshift-routes.html Reason: InstallSuccessful Status: True Type: Deployed Deployed Release: Manifest: . . . . . . . . . . We can verify from CLI or GUI. $ oc get pods -n kube-system NAME READY STATUS RESTARTS AGE f5-server-f5-bigip-ctlr-7c77d6846f-z7bhp 1/1 Running 0 112s Congratulations! Your F5 BIG-IP Controller is deployed using F5 BIG-IP Controller Operator. Additional Resources Operator Code: https://github.com/F5Networks/k8s-bigip-ctlr/tree/master/operator Operator Image: https://access.redhat.com/containers/#/registry.connect.redhat.com/f5networks/k8s-bigip-ctlr-operator Known Issues When Custom Resource Instance is created, instance listing doesn’t show Status [1] in the GUI. [1] https://github.com/operator-framework/operator-sdk/issues/2491Protect Your Kubernetes Cluster Against The Apache Log4j2 Vulnerability Using BIG-IP
Whenever a high profile vulnerability like Apache Log4j2 is announced, it is often a race to patch and remediate. Luckily, for those of us with BIG-IP's with AWAF (Advanced Web Application Firewall) in our environment, we can take care of some mitigation through updating and applying signatures. When there is a consolidation of duties, or both SecOps and NetOps work together on the same cluster of BIG-IP's then an AWAF policy can simply be applied to a virtual server. However, as we move into a world of modern application architectures, the Kubernetes administrators are very often a different set of individuals falling within DevOps. The DevOps team will work with NetOps to incorporate BIG-IP as the Ingress to the Kubernetes environment through the use of Container Ingress Services. This allows for a declarative configuration and objects can be called upon to incorporate into the Ingress configuration. In Container Ingress Services version 2.7, using the Policy CRD (Custom Resource Definitions) feature, an AWAF policy can be one of these objects incorporated. Here is some example code for defining the Policy CRD and specifying the WAF policy: apiVersion: cis.f5.com/v1 kind: Policy metadata: labels: f5cr: "true" name: policy-mysite namespace: default spec: l7Policies: waf: /Common/WAF_Policy profiles: http: /Common/Custom_HTTP logProfiles: - /Common/Log all requests And here is an example of associating this Policy CRD with the VirtualServer CRD: apiVersion: "cis.f5.com/v1" kind: VirtualServer metadata: name: vs-myapp labels: f5cr: "true" spec: # This is an insecure virtual, Please use TLSProfile to secure the virtual # check out tls examples to understand more. virtualServerAddress: "10.192.75.117" virtualServerHTTPSPort: 443 httpTraffic: redirect tlsProfileName: reencrypt-tls policyName: policy-mysite host: myapp.f5demo.com pools: - path: / service: f5-demo servicePort: 443 Mark Dittmer, Sr. Product Management Engineer here at F5, recently teamed up with Brandon Frelich, Security Solutions Architect, to create a how-to video on this. Mark's associated Github repo: https://github.com/mdditt2000/kubernetes-1-19/blob/master/cis%202.7/log4j/README.md This is going to now allow for the SecOps teams to focus on creating and providing AWAF policies while the DevOps can focus on their domain and incorporate the AWAF policy quickly. As we see microservices sprawl, we need every speed advantage we can get!
