BIG-IP Next SPK: a Kubernetes native ingress and egress gateway for Telco workloads

Kubernetes has a simplified networking model which was designed for general IT workloads which only use TCP/HTTP protocols, and a simplified networking model with a single IP address per POD (the smallest deployable unit of computing in Kubernetes) and a single external gateway. On the other hand, Telco deployments require:

• 3GPP protocols support.
• Transitional 4G to 5G facilities.
• Network capabilities to match Telco's networks by providing multiple external network connectivity, allowing different paths depending on the Network Function (NF), dynamic routing, etc...

Hopefully, Kubernetes has been designed to be extensible yet it's up to the software and infrastructure architects to design solutions with good practices by following Kubernetes patterns.

It´s usual that NF vendors use additional interfaces (multus CNI) for each NF PODs. By doing this, there is no dynamic advertising of addresses as the PODs in the deployment change or a good way to track these addresses outside the cluster by network elements such as firewalls.

Alternatively, NF vendors try to hide this complexity by selling turn-key solutions with dedicated Kubernetes clusters for each NF or vendor. Therefore the customer ends up with multiple clusters typically disimilar, ultimately defeating the whole purpose of Kubernetes which aims to homogenize application's environment in a single platform. In a way, this this approach is equivalent to having a load balancer for each application. Also, the management of addresses by external network elements is still inaddecuate because identification of NFs is done in a coarse manner by identifying clusters' addresses.

These two approaches break Kuberentes patterns by adding complexity in the form of non-homogenous networking to the different NFs.

In this post we introduce F5 BIG-IP Next Service Proxy Kubernetes (SPK) -- BIG-IP Next SPK for short -- architecture which overcomes these limitations while being Network Functions-agnostic. We will use Red Hat's Openshift as reference platform.

BIG-IP Next SPK software architecture

BIG-IP Next SPK is a cloud-native solution which runs inside the Kubernetes cluster and is made out of independent components which can be scaled-out. It's headless software (no graphical UI) and it's managed using the Kubernetes API. The major software components are shown next.

BIG-IP Next SPK's data plane makes use of the widely trusted BIG-IP’s Traffic Management Microkernel (TMM) data plane. This allows for a high performance, dependable product from the start. A dynamic routing component configures the BGP peering with the upstream routers for ECMP load distribution. The BFD feature for fast failure detection is available. The session persistence is a distributed database which allows to store connection related state such as pool member persistence, SNATs, NAT46 translations, etc... This database is backed in Kubernetes Persistent Volume which allows this information to be available even after POD restarts. The controller is the component which interacts with the Kubernetes API that customers use to configure BIG-IP Next SPK. Fluentd is a high-performance industry standard for exposing BIG-IP Next SPK metrics and logs to external tools.

BIG-IP Next SPK network architecture

The overall network architecture is shown next. Out of this picture we would like to emphasise the following items:

• Independent BIG-IP Next SPK instances, with completely different external network config, can handle ingress & egress traffic for each namespace individually.
• BIG-IP Next SPK is highly scalable at POD level (1-24 cores) and at cluster level, limited by the upstream ECMP capabilities.
  
  

A more detailed view of the network path is shown next. From this diagram we want to emphasise:

• PODs make use of BIG-IP Next SPK transparently by continue using the CNI as usual.
• BIG-IP Next SPK is a single tier ingress/egress solution not requiring external LB.
• BIG-IP Next SPK has direct POD IP visibility, there is no kube-proxy or other IP-translating mechanism in between.

As depicted above, BIG-IP Next SPK has two types of interfaces: external facing the upstream routers and internal facing the Kubernetes networking.

Openshift's networking facilitates that using BIG-IP Next SPK in a cluster is optional in a per-namespace basis and this is done transparently to the applications.  No change or configuration needs to be done in the applications.

Openshift uses for its networking the OVNKubernetes CNI. It can be seen in the picture that the applications continue using the OVNKubernetes router as default and only gateway. 

Finally we will show a L2 view of the networks in a cluster with BIG-IP Next SPK. From this diagram we want to emphasise:

• How regular nodes hosting applications have no modifications either.
• How BIG-IP Next SPK is typically setup with link aggregations and SR-IOV wire-speed interfaces.
• How the L3 path between BIG-IP Next SPK and the application's nodes is validated by means of using BFD.

Using BIG-IP Next SPK

In order to use BIG-IP Next SPK no changes need to be done in the applications or in the namespace hosting the applications. At BIG-IP Next SPK configuration time we will instruct which namespace we want BIG-IP Next SPK to handle and voilà: BIG-IP Next SPK becomes the next-hop of OVNKubernetes router for that namespace. No labels or other artifacts need to be configured manually.

Defining BIG-IP Next SPK services configurations is done thorugh the Kubernetes API using Custom Resource Definitions (CRDs). At time of this writting the following resources are avaiilable: