Why K8s Egress Traffic Policy Control is Critical to Security Architecture - Part 2

(This is the 2/2, For the part 1 please click here)

Service Mesh

Service Mesh is not a special solution for egress traffic control. Therefore, to implement egress control through Service Mesh means that an overall Service Mesh solution, such as Istio, needs to be deployed first. If it is only for the control of egress, such a scheme will be heavier. Service Mesh also supports a smaller range of protocols, which is not enough for an enterprise's security strategy.

In Istio, when

meshConfig.outboundTrafficPolicy.mode

is set to REGISTRY_ONLY, the whitelist for external service access can be implemented through sidecar combined with ServiceEntry resources. It is also possible to direct traffic to a dedicated Egress Gateway by combining with the Egress Gateway. Compared with the ServiceEntry method, the Egress Gateway combines VirtualService and DestinationRule to achieve more control, and with AuthorizationPolicy, it can control the granularity more finely.

Either way, you must rely on the sidecar for traffic hijacking. If there is a threat to bypass or destroy the sidecar, it means that harmful access can directly bypass the control. This security issue is repeatedly mentioned in Istio's documentation. So essentially it is not a good egress traffic control solution. At the same time, the thinking of Service Mesh is more for developers (although it often reflects platform-level capabilities), so we still need to answer the following question: When developers set the whitelist for external service access, whether the outside of the cluster trust such a setup, can an external security appliance be set up to allow arbitrary external access for the cluster?

Micro Segmentation

Micro-segmentation is generally a popular concept in the field of Zero Trust Architecture (ZTA). Through technologies (such as TC, IPtables, DPI), the underlying traffic can be probed, manipulated and controlled to realize the traffic of intra-container, inter-container communication, and out-of-cluster traffic. Generally speaking, the DaemonSet container is installed on each nodes of the cluster to detect the underlying traffic. This type of solution can control the egress policy based on finer granularity, such as which application-related pods access which external services through which protocols, and can also select elements such as Service account or Label. For application-layer encrypted data, if it is an Istio environment, plaintext detection can be achieved by detecting the traffic between the sidecar and the application container; if the application itself is directly encrypted, the detection cannot be achieved, but it can be achieved by combining DNS resolution and SNI.

Fusion

For the above-mentioned various types of solutions, the main entry point is in the cluster. In the actual production environment, the Kubernetes (K8s) cluster is a resource object. From the perspective of the overall security of the enterprise, it is still necessary for external security devices to control the outgoing traffic of the Kubernetes cluster. The difficulty in integrating external security devices with Kubernetes clusters is that traditional security devices are not designed directly for Kubernetes. High dynamics and IP independence will make it difficult for traditional devices to control Kubernetes outbound traffic. But this is not an unsolvable technical problem. If the external security device has a good API interface, the above technical problem can be solved by a specially designed controller. In this way, external security protection measures can be applied to the Kubernetes cluster to form a complete defense-in-depth system. At the same time, it can protect the existing investment of enterprises and save costs. Through the CRD design for Kubernetes, the team responsible for external security devices can also be involved in the overall security work of the Kubernetes cluster, avoiding the division of the team.

F5's Application Firewall Manager (AFM) realizes the control of Egress policies in the form of Kubernetes native custom resources (CRD) through a dedicated free controller (CES), and realizes the hierarchical setting of security rules and roles, making security administrators are involved into the Kubernetes platform. With the capabilities of AFM, advanced access rules, rate limiting, protocol inspection, log and event visualization of egress traffic can be implemented.

Fortinet itself and Calico Enterprise Edition have also integrated with Kubernetes, but its main feature is that Kubernetes resource objects are converted and written into Fortinet's address group, and its management perspective is still the firewall administrator's perspective, not Kubernetes native Way.

Others

Proxy pod is a common forward proxy, which is used by applications to access external services. This method is generally only suitable for small scenarios, and is not suitable for large-scale clusters and complex services.

The principle of DNS interception is through patching coredns. If the application accesses the external service set in the ExternalService object, it will direct the request to a dedicated proxy pod (such as Envoy, etc.) to process the traffic. This solution is also not suitable for large-scale scenarios.

After completing the analysis of the above 6 types of Egress traffic control solutions, let us summarize and compare the advantages and disadvantages of these solutions:

It can be seen that although Network Policy is a native method of Kubernetes, it is obviously not suitable for the management and control of cluster Egress traffic. CNI products have been enhanced to a certain extent, but they are still insufficient in terms of protocol detection and enterprise-level support. Micro-segmentation products have relatively complete capabilities, but micro-segmentation is a holistic solution. Only using micro-segmentation to control the outgoing traffic of a cluster will require a lot of investment, and micro-segmentation products generally have more complex underlying technologies, and higher operation and maintenance costs. The solution of integrating external security facilities into Kubernetes to realize outbound traffic control is more suitable for enterprises, both in terms of functional characteristics and operation and maintenance complexity. More importantly, this type of solution integrates traditional security assets of enterprises with modern application architecture, so that different departments can work closely together to form a defense-in-depth system.

Summarize

We often pay attention to the capabilities of Ingress, while ignoring the security control of egress traffic. However, no matter from the perspective of security or compliance, Egress traffic should be strictly controlled. After the application has been invaded as vulnerability, Egress traffic control is often the last protection. Among the current solutions, most of them are based on the implementation of Network Policy in Kubernetes, some rely on a specific CNI, and some rely on a specific orchestration platform. But when we consider the overall security architecture of the enterprise, it is very necessary to introduce external security devices into the kKubernetes security. Only in this way, a truly comprehensive defense can be achieved. When we discuss DevSecOps, we need to involve different teams(development, platform, security and even network) into the overall security work to achieve close collaboration across teams.

In the next article, I will introduce how the F5 CES solution implements Kubernetes egress traffic security control and how to achieve close collaboration between different teams.