Forum Discussion
Altocumulusfeature request: container egress service
After installing cis in a test environment and getting ready to install in a new production environment I wonder if there also will be a container egress service (CES)? It is very easy to set a gateway for selected namespaces with AdminPolicyBasedExternalRoute in Openshift. See, F5 BIG-IP deployment with Red Hat OpenShift - keeping client IP addresses and egress flows | DevCentral
The solution above does not scale well if multiple namespace-egress IP address mappings are desired. A nice solution would be a CES that watches the creating and deletion of pods in selected namespaces. Then it can manage address lists with the pods ip addresses in the F5 ltm. Forwarding ip virtual services will use these address lists to match pod ip addresses to an egress ip defined in a snat pool.
Also the creation and deletion of forwarding ip virtual servers and address lists could be managed with a "CES".
A possible issue is that a container in a pod can start network connections before the forwarding IP virtual server accepts the new pod IP address. But this can easily be solved with adding an initcontainer in the pod that tests the network connectivity.
This would be a good alternative for Openshift egress IPs or Istio gateways.
Reason to want this, is to offer applications on Openshift an own egress IP address and stop using the node IP address for external network connections of the pods.
3 Replies
tomas573Nimbostratus
Hello,
You've outlined a very interesting and valuable problem, along with a potential solution for managing egress traffic in OpenShift using F5 BIG-IP. Let's break down the concepts and explore the feasibility of a Container Egress Service (CES) as you've described.
Understanding the Problem:
Namespace-Specific Egress: You want to assign dedicated egress IP addresses to individual namespaces in OpenShift, rather than relying on node IPs or a single shared egress IP.
Scalability: The AdminPolicyBasedExternalRoute approach, while functional, doesn't scale well when dealing with numerous namespaces and their respective egress IP mappings.
Dynamic Pod IPs: Pod IPs are ephemeral, changing with pod restarts or scaling. Manual management of these IPs in F5 BIG-IP is impractical.
Egress Control: You need a dynamic and automated mechanism to manage the lifecycle of pod IPs and their corresponding egress configurations on the F5 BIG-IP.
Best Regards
PaulVogtHi Tomas
Exactly. Using a gateway device or F5 ltm interface per namespace is impractical. Using an address list for the source of a forwarding ip virtual server would give the desired functionality. A reverse CIS solution. An outline of the controller flow. Which will need some more refinement,watch the configmap if a tenant is removed from the configmap then delete the external gateway definition delete the forwarding ip virtual server delete the address list delete the snat pool if egress ip is changed then update snat pool with new ip address only for the tenants listed in the configmap watch the tenant labeled namespaces for AdminPolicyBasedExternalRoute definition if AdminPolicyBasedExternalRoute not defined then define AdminPolicyBasedExternalRoute watch the tenant labeled namespaces for pods if a pod is created if address list does not exist yet then create an address list with pod ip. else add the pod ip address to the address list of the tenant if forwarding ip virtual server does not exist then if snat pool does not exist then create snat pool with tenant ip address from the configmap else if snat pool ip is not the egress ip then update the snat pool ip with the egress ip create forwarding ip virtual server with the previously defined snat and address list if a pod is deleted then delete the pod ip from the address list of the corresponding tenant if tenant has no pods and no_tenant_vs = false then if exists then delete the virtual server if exists then delete the snat pool if exists delete the address list set no_tenant_vs = true- PaulVogt
A pod could start making connections before the new pod ip is added to the forwarding ip virtual server. This can be solved with using an initcontainer that tests the network connectivity.
