on - edited on by
LiefZimmerman
Summary
Features like subrequests for authentication, adding/removing HTTP headers, and supporting services of type ExternalName in Kubernetes (K8s) are great reasons to use F5 NGINX Ingress Controller. Documentation and examples of NGINX configuration are easy to find, but this article specifically discusses how to achieve these when running NGINX Ingress Controller within K8s.
Introduction
Just like last time, I had a customer in need recently who was able to find documentation to meet their requirements that applied to a typical NGINX installation. By "typical", I mean an installation where we can modify NGINX configuration files directly. However, when using NGINX Ingress Controller, we create K8s resources, not configuration files. The Ingress controller watches these and then modifies it's own configuration files. Accordingly, I had to adapt the documentation I found to help my customer approach NGINX Ingress Controller, and so I've written this article in case anyone else finds it helpful.
Requirements
I'll simplify my customer's requirements here:
- Authentication subrequests. The customer had a custom authentication service that would receive a HTTP request, analyze a JWT in a certain header, and respond with either a HTTP Status of 204 (authentication successful) or 401 (unsuccessful). Note, this is not OIDC with Ingress Controller, and it is not the same as JWT policy (like the previous case I wrote about). I have created an extremely basic authentication service using an iRule to simply respond to requests with HTTP status 204 or 401 so that we can emulate their service.
- ExternalName service. Proxying to a service of type ExternalName within K8s has been supported since NGINX Ingress Controller 1.5 and is a Plus-only feature. Offical example from NGINX documentation is here.
- HTTP Header append and remove. This is straightforward functionality to append or remove headers from requests to proxied upstream servers. We will use the native functionality of the VirtualServer and VirtualServerRoute Custom Resource Definitions.
Overview of demo architecture
I have a very basic app within K8s with only a few paths: a default path, a path that is proxied to an external website, and a path that is only available to authenticated users.
- http://demo.my-f5.com/ - this is the default page
- http://demo.my-f5.com/external - this path is proxied to an external service of your choice
- http://demo.my-f5.com/private - this is only available if subrequest to auth service receives a 204 response
- http://demo.my-f5.com/auth - This is our subrequest to an authentication service, which can be hosted inside or outside of the cluster (in my example it is outside of the cluster). The /auth endpoint cannot be reached directly by users.
Kubernetes resources to configure this demo
I have documented this use case along with examples here on Github.
I will call out a few of these resources.
- We primarily use VirtualServer and VirtualServerRoute resources. These are represented by the "vs" and "vsr" boxes in the diagram above.
- We use snippets, which allow us to insert raw NGINX config into these CRD's, in cases where the YAML-based CRD doesn't meet our requirement. So, http-snippets, server-snippets, and location-snippets insert config into the http, server, and location contexts. Here's an example that uses all three. In this demo, we need use location-snippets in the VirtualServerRoute resources.
- In my demo above, I have a very basic external service. In truth, it was a VIP on a BIG-IP with this iRule.
- In my demo above, I have a very basic authentication service. In truth, it was another VIP and iRule.
Here's an example of an iRule you can use for an external service, attached to a BIG-IP Virtual Server listening on HTTP Port 80 with a HTTP profile.
when HTTP_REQUEST {
foreach aHeader [HTTP::header names] {
log local0. "HTTP Request Headers: $aHeader: [HTTP::header value $aHeader]"
}
HTTP::respond 200 content "hi, i am your external service"
}Here's an example of a very basic authentication service. You can edit this iRule if/when needed.
when HTTP_REQUEST {
foreach aHeader [HTTP::header names] {
log local0. "HTTP Request Headers: $aHeader: [HTTP::header value $aHeader]"
}
# uncomment one of the lines below to respond with either 204 or 401
# HTTP::respond 401
HTTP::respond 204
}
Advanced features you might add yourself
This demo assumes a basic lab with a single Ingress controller. However in production environments it is typical to see multiple Ingress controllers, and some additional requirements:
- Caching of authentication results. The user experience can be improved by caching validation responses for a short time. Caching is already documented in this existing guide NGINX Plus, however this example uses directives in the http and location blocks. To implement the same in K8s, we would use http-snippets and location snippets in the VirtualServer and VirtualServerRoute resources, respectively.
- Distributed caching. (Distributed caching is a Plus-only feature). For NGINX Plus, we could show how the cache can be distributed across a cluster of NGINX Plus instances, by updating the key‑value store with the JavaScript module, as introduced in NGINX Plus R18. Distributed caching is also already documented well in the same existing guide (see Optimization 2) and again, we would use snippets to do the same in our K8s environment.
- Revocation of records from the cache. If we cache successful authentications so as to improve user experience, it is wise to consider how we might revoke a cached authentication. This can be done with an API call (also a Plus-only feature).
Summary
With the linked GitHub repo, you should be able to see how to implement the features that meet my customers requirements with NGINX Ingress Controller and the associated resources. Perhaps the most interesting is authe subrequests (which could actually be used for purposes other than authentication, if you wanted).
When I come across NGINX documentation but would like to configure NGINX Ingress Controller within K8s, I typically look to the NGINX CRD's of VirtualServer, VirtualServerRoute, and Policy. I refer closely to documentation, and, of course, folks like me are always here to help so reach out to your account teams or via the comments below, thanks!
