Building Trust with Service Mesh

The Istio Service Mesh is built on a “zero-trust network”; but how do you ensure that you can trust Istio? When trusted and untrusted services are co-mingled in a single Kubernetes cluster all services appear to originate from a single IP address. Only being able to use an IP address as an identifier makes traditional firewall and forward web proxies unable to provide different levels of assurance. We’ll look at 3 different ways that we can secure egress traffic from the service mesh Istio using both built-in mechanisms as well as BIG-IP and F5 SSL Orchestrator.

What is a Service Mesh

A Service Mesh is a pattern of deploying many “small” proxies to secure communication, provide additional audit capability, and monitoring. The Istio project uses the Envoy proxy and mutual TLS to secure communications between services deployed in a Kubernetes cluster.

Trusting Istio

When you deploy Istio you can opt to have all egress traffic blocked and create specific rules to permit traffic to specific endpoints. The first method to secure egress traffic is to use the built-in methods of creating an “egress gateway”. This allows you to restrict outbound connections from the service mesh by IP, Port, Name, and/or TLS ServerName. Ideally you would also like to audit that Istio’s policies are accurate by layering in additional security.

Trust and Audit

Istio uses mutual TLS to secure communication within the service mesh, but when traffic departs the mesh it loses the identity of the original service. A request from a trusted service will look the same as an untrusted service. Using Istio’s built-in mutual TLS we can extend the circle of trust to include a BIG-IP. This enables the BIG-IP to capture the original identity of the service and distinguish between trusted and untrusted services. An example of the information that we can capture when the BIG-IP has joined the mesh and configured as an explicit proxy:

..<http_proxy_request>: service: spiffe://cluster.local/ns/trusted/sa/sleep GET http://www.f5.com/company
..<http_proxy_request>: service: spiffe://cluster.local/ns/trusted/sa/sleep CONNECT www.f5.com:443
..<http_proxy_request>: service: spiffe://cluster.local/ns/untrusted/sa/sleep GET http://www.f5.com/company
..<http_proxy_request>: service: spiffe://cluster.local/ns/untrusted/sa/sleep CONNECT www.f5.com:443

Service Chain of Trust

In the previous example we saw the complete request for cleartext, HTTP, traffic, but encrypted, HTTPS, traffic is not visible. We can create a proxy chain from the BIG-IP to another F5 SSL Orchestrator device that can selectively intercept HTTPS traffic for untrusted services. In this example we can configure the SSL Orchestrator to act as an explicit proxy and send decrypted traffic to a passive L2 device.

The Demo

Please watch the demo video to see an example of this in a lab environment. You may be also interested in the following article about connecting legacy services (like a database) to a service mesh and securing its communication over mutual TLS.