In our previous article, we have setup Kubernetes and calico with our BIG-IPs. Now we will setup F5 Container Ingress Services (F5 CIS) and deploy an ingress service.
Note: in this deployment, we will setup F5 CIS in the namespace kube-system
BIG-IPs Setup
Setup our Kubernetes partition
Before deploying F5 Container Ingress Services, we need to setup our BIG-IPs:
- Setup a partition that CIS will use.
- Install AS3
On EACH BIG-IP, you need to do the following:
In the GUI, go to System > User > Partitions List and click on the Create button
Create a partition called “kubernetes” and click Finished
Next we need to download the AS3 extension and install it on each BIG-IP.
Install AS3
Go to GitHub . Select the release with the “latest” tag and download the rpm.
Once you have the rpm, go to iApps > Package Management LX and click the Import button (you need to run BIG-IP v12.1 or later) on EACH BIG-IP
Chose your rpm and click Upload. Once the rpm has been uploaded and loaded, you should see this:
Our BIG-IPs are setup properly now.
Next, we will deploy F5 CIS
CIS Deployment
Connect to your Kubernetes cluster. We will need to setup the following:
- Create a Kubernetes secret to store our BIG-IPs credentials
- Setup a service account for CIS and setup RBAC
- Setup our CIS configuration. We will need 1xCIS per BIG-IP (even when BIG-IP is setup as a cluster - we don’t recommend automatic sync between BIG-IPs)
- Deploy one CIS per BIG-IP
Store our BIG-IP credentials in a kubernetes secret
To store your credentials (login/password) in a kubernetes secret, you can run the following command (https://clouddocs.f5.com/containers/v2/kubernetes/kctlr-secrets.html#secret-bigip-login):
kubectl create secret generic bigip-login --namespace kube-system --from-literal=username=<your_login> --from-literal=password=<your_password>
In my setup, my credentials are the following:
- Login: admin
- Password: D3f4ult123
So we’ll run the following command:
kubectl create secret generic bigip-login --namespace kube-system --from-literal=username=admin --from-literal=password=D3f4ult123
ubuntu@ip-10-1-1-4:~$ kubectl create secret generic bigip-login --namespace kube-system --from-literal=username=admin --from-literal=password=D3f4ult123
secret/bigip-login created
Create a Service Account /RBAC for CIS
Now we need to create our service account (https://clouddocs.f5.com/containers/v2/kubernetes/kctlr-app-install.html#set-up-rbac-authentication):
Run the following command: :
kubectl create serviceaccount bigip-ctlr -n kube-system
ubuntu@ip-10-1-1-4:~$ kubectl create serviceaccount bigip-ctlr -n kube-system
serviceaccount/bigip-ctlr created
Use your favorite editor to create this file:
f5-k8s-sample-rbac.yaml:
# for use in k8s clusters only
# for OpenShift, use the OpenShift-specific examples
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: bigip-ctlr-clusterrole
rules:
- apiGroups: ["", "extensions"]
resources: ["nodes", "services", "endpoints", "namespaces", "ingresses", "pods"]
verbs: ["get", "list", "watch"]
- apiGroups: ["", "extensions"]
resources: ["configmaps", "events", "ingresses/status"]
verbs: ["get", "list", "watch", "update", "create", "patch"]
- apiGroups: ["", "extensions"]
resources: ["secrets"]
resourceNames: ["<secret-containing-bigip-login>"]
verbs: ["get", "list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: bigip-ctlr-clusterrole-binding
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: bigip-ctlr-clusterrole
subjects:
- apiGroup: ""
kind: ServiceAccount
name: bigip-ctlr
namespace: kube-system
Deploy this config by running the following command:
kubectl apply -f f5-k8s-sample-rbac.yaml
ubuntu@ip-10-1-1-4:~$ kubectl apply -f f5-k8s-sample-rbac.yaml
clusterrole.rbac.authorization.k8s.io/bigip-ctlr-clusterrole created
clusterrolebinding.rbac.authorization.k8s.io/bigip-ctlr-clusterrole-binding created
Next step is to setup our CIS deployment files
Create our CIS deployment configurations
Use your favorite editor to create the following files:
setup_cis_bigip1.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-bigip1-ctlr-deployment
namespace: kube-system
spec:
selector:
matchLabels:
app: k8s-bigip1-ctlr
# DO NOT INCREASE REPLICA COUNT
replicas: 1
template:
metadata:
labels:
app: k8s-bigip1-ctlr
spec:
# Name of the Service Account bound to a Cluster Role with the required
# permissions
serviceAccountName: bigip-ctlr
containers:
- name: k8s-bigip-ctlr
image: "f5networks/k8s-bigip-ctlr"
env:
- name: BIGIP_USERNAME
valueFrom:
secretKeyRef:
# Replace with the name of the Secret containing your login
# credentials
name: bigip-login
key: username
- name: BIGIP_PASSWORD
valueFrom:
secretKeyRef:
# Replace with the name of the Secret containing your login
# credentials
name: bigip-login
key: password
command: ["/app/bin/k8s-bigip-ctlr"]
args: [
# See the k8s-bigip-ctlr documentation for information about
# all config options
# https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest
"--bigip-username=$(BIGIP_USERNAME)",
"--bigip-password=$(BIGIP_PASSWORD)",
"--bigip-url=10.1.20.11",
"--bigip-partition=kubernetes",
"--insecure=true",
"--pool-member-type=cluster",
"--agent=as3"
]
imagePullSecrets:
# Secret that gives access to a private docker registry
- name: f5-docker-images
# Secret containing the BIG-IP system login credentials
- name: bigip-login
setup_cis_bigip2.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-bigip2-ctlr-deployment
namespace: kube-system
spec:
selector:
matchLabels:
app: k8s-bigip2-ctlr
# DO NOT INCREASE REPLICA COUNT
replicas: 1
template:
metadata:
labels:
app: k8s-bigip2-ctlr
spec:
# Name of the Service Account bound to a Cluster Role with the required
# permissions
serviceAccountName: bigip-ctlr
containers:
- name: k8s-bigip-ctlr
image: "f5networks/k8s-bigip-ctlr"
env:
- name: BIGIP_USERNAME
valueFrom:
secretKeyRef:
# Replace with the name of the Secret containing your login
# credentials
name: bigip-login
key: username
- name: BIGIP_PASSWORD
valueFrom:
secretKeyRef:
# Replace with the name of the Secret containing your login
# credentials
name: bigip-login
key: password
command: ["/app/bin/k8s-bigip-ctlr"]
args: [
# See the k8s-bigip-ctlr documentation for information about
# all config options
# https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest
"--bigip-username=$(BIGIP_USERNAME)",
"--bigip-password=$(BIGIP_PASSWORD)",
"--bigip-url=10.1.20.12",
"--bigip-partition=kubernetes",
"--insecure=true",
"--pool-member-type=cluster",
"--agent=as3"
]
imagePullSecrets:
# Secret that gives access to a private docker registry
- name: f5-docker-images
# Secret containing the BIG-IP system login credentials
- name: bigip-login
To deploy our CIS, run the following commands:
kubectl apply -f setup_cis_bigip1.yaml
kubectl apply -f setup_cis_bigip2.yaml
You can make sure that they got deployed successfully:
kubectl get pods -n kube-system
You can review if it launched successfully with the commands:
kubectl logs <pod_name> -n kube system
ubuntu@ip-10-1-1-4:~$ kubectl logs k8s-bigip2-ctlr-deployment-6f674c8d58-bbzqv -n kube-system
2020/01/03 12:04:06 [INFO] Starting: Version: 1.12.0, BuildInfo: n2050-623590021
2020/01/03 12:04:06 [INFO] ConfigWriter started: 0xc0002bb950
2020/01/03 12:04:06 [INFO] Started config driver sub-process at pid: 14
2020/01/03 12:04:07 [INFO] NodePoller (0xc000b7c090) registering new listener: 0x11bfea0
2020/01/03 12:04:07 [INFO] NodePoller started: (0xc000b7c090)
2020/01/03 12:04:07 [INFO] Watching Ingress resources.
2020/01/03 12:04:07 [INFO] Watching ConfigMap resources.
2020/01/03 12:04:07 [INFO] Handling ConfigMap resource events.
2020/01/03 12:04:07 [INFO] Handling Ingress resource events.
2020/01/03 12:04:07 [INFO] Registered BigIP Metrics
2020/01/03 12:04:07 [INFO] [2020-01-03 12:04:07,663 __main__ INFO] entering inotify loop to watch /tmp/k8s-bigip-ctlr.config563475778/config.json
2020/01/03 12:04:08 [INFO] Successfully Sent the FDB Records
You may also check your BIG-IPs configuration: if CIS has been able to connect successfully, it will have created *another* partition called “kubernetes_AS3”.This is explained here: F5 BIG-IP Container Ingress Services
CIS will create partition kubernetes_AS3 to store LTM objects such as pools, and virtual servers. FDB, and Static ARP entries are stored in kubernetes. These partitions should not be managed manually.
Application Deployment
In this section, we will do the following:
- Deploy an application with 2 replicas (ie 2 instances of our app)
- Setup an ingress service to connect to your application
Deploy our application and service
Create the following service and deployment configuration files:
f5-hello-world-app-http-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: f5-hello-world
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: f5-hello-world
template:
metadata:
labels:
app: f5-hello-world
spec:
containers:
- env:
- name: service_name
value: f5-hello-world
image: f5devcentral/f5-hello-world:latest
imagePullPolicy: Always
name: f5-hello-world
ports:
- containerPort: 8080
protocol: TCP
f5-hello-world-app-http-service.yaml
apiVersion: v1
kind: Service
metadata:
name: f5-hello-world
namespace: default
labels:
app: f5-hello-world
spec:
ports:
- name: f5-hello-world
port: 8080
protocol: TCP
targetPort: 8080
type: NodePort
selector:
app: f5-hello-world
Apply your deployment and service:
kubectl apply -f f5-hello-world-app-http-deployment.yaml
kubectl apply -f f5-hello-world-app-http-service.yaml
ubuntu@ip-10-1-1-4:~$ kubectl apply -f f5-hello-world-app-http-deployment.yaml
deployment.apps/f5-hello-world created
ubuntu@ip-10-1-1-4:~$ kubectl apply -f f5-hello-world-app-http-service.yaml
service/f5-hello-world created
You can review your deployment and service with kubectl get :
ubuntu@ip-10-1-1-4:~$ kubectl get pods
NAME READY STATUS RESTARTS AGE
f5-hello-world-847698f5c6-59mmn 1/1 Running 0 2m3s
f5-hello-world-847698f5c6-pcz9v 1/1 Running 0 2m3s
ubuntu@ip-10-1-1-4:~$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
f5-hello-world NodePort 10.97.83.208 <none> 8080:31380/TCP 2m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 63d
Define our ingress service
CIS ingress annotations can be reviewed here: F5 BIG-IP Controller for Kubernetes
Create the following file:
f5-as3-ingress.yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: singleingress1
namespace: default
annotations:
# See the k8s-bigip-ctlr documentation for information about
# all Ingress Annotations
# https://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest/#supported-ingress-annotations
virtual-server.f5.com/ip: "10.1.10.80"
virtual-server.f5.com/http-port: "443"
virtual-server.f5.com/partition: "kubernetes_AS3"
virtual-server.f5.com/health: '[{"path": "/", "send": "HTTP GET /", "interval": 5, "timeout": 10}]'
spec:
backend:
# The name of the Service you want to expose to external traffic
serviceName: f5-hello-world
servicePort: 8080
you can review the BIG-IP configuration by going into the kubernetes_AS3 partition
Summary
In this article we saw how to deploy CIS with AS3 to handle ingress services.
CIS has more capabilities than ingress that you may review here: Kubernetes (configmap, routes, …)
----
Relevant links to this article:







