F5 NGINX Plus R34 Release Now Available
We’re excited to announce the availability of F5 NGINX Plus Release 34 (R34). Based on NGINX Open Source, NGINX Plus is the only all-in-one software web server, load balancer, reverse proxy, content cache, and API gateway. New and enhanced features in NGINX Plus R34 include: Forward proxy support for NGINX usage reporting: With R34, NGINX Plus allows NGINX customers to send their license usage telemetry to F5 via an existing enterprise forward proxy in their environment. Native support for OpenID Connect configuration: With this release, we would like to announce the availability of native OpenID Connect (OIDC) module in NGINX Plus. The native module brings simplified configuration and better performance while addressing many of the complexities with the existing njs-based solution. SSL Dynamic Certificate Caching: NGINX Plus R34 builds upon the certificate caching improvements in R32 and introduces support for caching of dynamic certificates and preserving this cache across configuration reloads. Important Changes In Behavior Removal of OpenTracing Module: In NGINX Plus R32, we announced deprecation of the OpenTracing module in favor of OpenTelemetry module introduced in NGINX Plus R29. , and was marked to be removed with NGINX Plus R34. The OpenTracing is now being removed from NGINX Plus effective this release. Changes to Platform Support Added Platforms: Alpine Linux 3.21 Removed Platforms: Alpine Linux 3.17 SLES12 Deprecated Platforms: Alpine Linux 3.18 Ubuntu 20.04 New Features in Detail Forward proxy support for NGINX usage reporting In the previous NGINX Plus release (NGINX Plus R33), we introduced major changes to NGINX Plus licensing requiring all NGINX customers to report their commercial NGINX usage to F5. One of the prime feedback we received for this feature was the need to enable NGINX instances to send telemetry via existing outbound proxies, primarily for environments where NGINX instances cannot connect to the F5 licensing endpoint directly. We are pleased to note that NGINX Plus R34 introduces support for using existing forward proxy solutions in customers environment for sending licensing telemetry to F5. With this update, NGINX Plus can now be configured to use the HTTP CONNECT proxy to establish a tunnel to the F5 licensing endpoint to send the usage telemetry. Configuration The following snippet shows the basic NGINX configuration needed in the ngx_mgmt_module module for sending NGINX usage telemetry via a forward proxy solution. mgmt { proxy HOST:PORT; proxy_username USER; #optional proxy_password PASS; #optional } For complete details, refer the docs here. Native support for OpenID Connect configuration Currently, NGINX Plus relies on a njs-based solution for OpenID Connect (OIDC) implementation that involves intricate JavaScript files and advanced setup steps, which are error-prone. With NGINX Plus R34, we're thrilled to introduce native OIDC support in NGINX Plus. This native implementation eliminates many of the complexities of the njs-based approach, making it faster, highly efficient, and incredibly easy to configure, with no burdensome overheads of maintaining and upgrading the njs module. To this effect, a new module ngx_http_oidc_module is introduced in NGINX Plus R34 that implements authentication as a relying party in OIDC using the Authorization Code Flow. The native implementation allows the flexibility to enable OIDC authentication globally, or at a more granular, per-server or a per-location level. It also allows effortless auto-discovery and retrieval of the OpenID providers' configuration metadata without needing complex external scripts for each Identity Provider (IdP), greatly simplifying the configuration process. For a complete overview and examples of the features in the native implementation of OIDC in NGINX Plus and how it improves upon the njs based implementation, refer the blog. Configuration The configuration to setup OIDC natively in NGINX Plus is relatively straightforward requiring minimal directives when compared to the njs based implementation. http { resolver 10.0.0.1; oidc_provider my_idp { issuer "https://provider.domain"; client_id "unique_id"; client_secret "unique_secret"; } server { location / { auth_oidc my_idp; proxy_set_header username $oidc_claim_sub; proxy_pass http://backend; } } } The example assumes that the “https://<nginx-host>/oidc_callback” redirection URI is configured on the OpenID Provider's side. For instructions on how to configure the native OIDC module for various identity providers, refer the NGINX deployment guide. SSL Certificate Caching improvements In NGINX Plus R32, we introduced changes to cache various SSL objects and reuse the cached objects elsewhere in the configuration. This provided noticeable improvements in the initial configuration load time primarily where a small number of unique objects were being referenced multiple times. With R34, we are adding further enhancements to this functionality where cached SSL objects are reused across configuration reloads, making the reloads even faster. Also, SSL certificates with variables are now cached as well. Refer the blog for a detailed overview of this feature implementation. Other Enhancements and Bug Fixes Keepalive timeout improvements Prior to this release, idle keepalive connections could be closed any time the connection needed to be reused for another client or when the worker was gracefully shutting down. With NGINX Plus R34, a new directive keepalive_min_timeout is being introduced. This directive sets a timeout during which a keepalive connection will not be closed by NGINX for connection reuse or graceful worker shutdown. The change allows clients that send multiple requests over the same connection without delay or with a small delay between them, to avoid receiving a TCP RST in response to one of them, if not for network reasons or non-graceful worker shutdown. As a side-effect, it also addresses the TCP reset problem described in RFC 9112, Section 9.6, when the last sent HTTP response could be damaged by a followup TCP RST. It is important for non-idempotent requests, which cannot be retried by client. It is however recommended to not set keepalive_min_timeout to large values as this can introduce an additional delay during worker process shutdown and may restrict NGINX from effective connection reuse. Improved health check logging NGINX Plus R34 adds logging enhancements in the error log for better visibility while troubleshooting upstream health check failures. The server status code is now logged on health check failures. Increased session key size Prior to R34, NGINX accepted an SSL session with maximum 4k(4096) bytes. With NGINX Plus R34, the maximum session size has been increased to 8k(8192) bytes to accommodate use cases where the sessions could be larger than 4k bytes. For ex. in cases where a client certificate is saved in the session, with tickets (in TLS v1.2 or older versions), or with stateless tickets (in TLS v1.3) the sessions maybe of noticeably large size. Certain stateless session resumption implementations may store additional data as well. One such case is with JDK, which is known to include server certificates in the session ticket data which roughly doubles the decoded session size. The changes also include improved logging to capture cases when sessions are not saved in shared memory due to size. Changes in the Open Telemetry Module TLS support in OTEL traces: NGINX now allows enabling TLS for sending OTEL traces. It can be enabled by specifying "https" scheme in the endpoint as shown. otel_exporter { endpoint "https://otel.labt.fp.f5net.com:4433"; trusted_certificate “path/to/custom/ca/bundle“; # optional } By default, system CA bundle is used to verify endpoint's certificate which can be overridden with "trusted_certificate" directive if required. For a complete list of changes to the OTEL module, refer the NGINX OTEL change log. Changes Inherited from NGINX Open Source NGINX Plus R34 is based on NGINX mainline release and inherits all functional changes, features, and bug fixes made since NGINX Plus R33 was released (in NGINX Open source 1.27.3 and 1.27.4 mainline versions) Features: SSL Certificate Caching "keepalive_min_timeout" directive The "server" directive in the "upstream" block supports the "resolve" parameter. The "resolver" and "resolver_timeout" directives in the "upstream" block. SmarterMail specific mode support for IMAP LOGIN with untagged CAPABILITY response in the mail proxy module. Changes: Now TLSv1 and TLSv1.1 protocols are disabled by default. An IPv6 address in square brackets and no port can be specified in the "proxy_bind", "fastcgi_bind", "grpc_bind", "memcached_bind", "scgi_bind", and "uwsgi_bind" directives, and as client address in ngx_http_realip_module. Bug Fixes: gzip filter failed to use preallocated memory" alerts appeared in logs when using zlib-ng. nginx could not build libatomic library using the library sources if the --with-libatomic=DIR option was used. QUIC connection might not be established when using 0-RTT; the bug had appeared in 1.27.1. NGINX now ignores QUIC version negotiation packets from clients. NGINX could not be built on Solaris 10 and earlier with the ngx_http_v3_module. Bugfixes in HTTP/3. Bugfixes in the ngx_http_mp4_module. The "so_keepalive" parameter of the "listen" directive might be handled incorrectly on DragonFly BSD. Bugfix in the proxy_store directive. Security: Insufficient check in virtual servers handling with TLSv1.3 SNI allowed to reuse SSL sessions in a different virtual server, to bypass client SSL certificates verification (CVE-2025-23419). For the full list of new changes, features, bug fixes, and workarounds inherited from recent releases, see the NGINX changes . Changes to the NGINX Javascript Module NGINX Plus R34 incorporates changes from the NGINX JavaScript (njs) module version 0.8.9. The following is a list of notable changes in njs since 0.8.7 (which was the version shipped with NGINX Plus R33). Features: Added fs module for QuickJS engine. Implemented process object for the QuickJS engine. Implemented the process.kill() method. Bug Fixes: Removed extra VM creation per server. Previously, when js_import was declared in http or stream blocks, an extra copy of the VM instance was created for each server block. This was not needed and consumed a lot of memory for configurations with many server blocks. This issue was introduced in 9b674412 (0.8.6) and was partially fixed for location blocks only in 685b64f0 (0.8.7). Fixed XML tests with libxml2 2.13 and later. Fixed promise resolving when Promise is inherited. Fixed absolute scope in cloned VMs. Fixed limit rated output. Optimized use of SSL contexts for the js_fetch_trusted_certificate directive. For a comprehensive list of all the features, changes, and bug fixes, see the njs Changelog.Lightboard Lessons: HTTP Cookie SameSite Attribute
In this episode of Lightboard Lessons, Jason covers the SameSite attribute on HTTP cookies, and the implications for site developers and end users when Chrome begins enforcing a default behavior set to "lax" later this month in a limited rollout for Chrome v80 stable users. This should be addressed in the applications, but BIG-IP can help via iRules and local traffic policies as briefly described in the video, as well as ASM module settings and through NGINX directives. Resources Start Here: Article: Handling Incompatible Clients AskF5 Knowledge Article on SameSite enforcement: K03346798 Article: Increased Security with First Party Cookies Additional iRule options: Codeshare: Setting SameSite on LTM Persistence Cookies Codeshare: Setting SameSite on All Web App & BIG-IP Cookies Codeshare: Add SameSite Attribute to APM Cookies Article: Detecting SameSite=None Incompatible Browsers ASM & NGINX Configuration Options ASM Manual info on SameSite NGINX proxy_cookie_path (Example: proxy_cookie_path / "/; secure; HttpOnly; SameSite=none";) NGINX sticky cookie (Example: sticky cookie srv_id expires=1h httponly secure “path=/; SameSite”;) Industry Insight Chromium Updates on SameSite (offsite) CSRF is (really) dead (offsite)2022 DevCentral MVP Announcement
Congratulations to the 2022 DevCentral MVPs! Without users who take time from their busy days to share their experience and knowledge for others, DevCentral would be more of a corporate news site and not an actual user community. To that end, the DevCentral MVP Award is given annually to the outstanding group of individuals – the experts in the technical F5 user community who go out of their way to engage with the user community. The award is our way of recognizing their significant contributions, because while all of our users collectively make DevCentral one of the top community sites around and a valuable resource for everyone, MVPs regularly go above and beyond in assisting fellow F5 users. We understand that 2021 was difficult for everyone, and we are extra-grateful to this year's MVPs for going out of their ways to help others. MVPs get badges in their DevCentral profiles so everyone can see that they are recognized experts. This year’s MVPs will receive a glass award, certificate, exclusive thank-you gifts, and invitations to exclusive webinars and behind-the-scenes looks at things like roadmaps, new product sneak-previews, and innovative concepts in development. The 2022 DevCentral MVPs are: Aditya K Vlogs AlexBCT Amine_Kadimi Austin_Geraci Boneyard Daniel_Wolf Dario_Garrido David.burgoyne Donamato 01 Enes_Afsin_Al FrancisD iaine jaikumar_f5 Jim_Schwartzme1 JoshBecigneul JTLampe Kai Wilke Kees van den Bos Kevin_Davies Lionel Deval (Lidev) LouisK Mayur_Sutare Neeeewbie Niels_van_Sluis Nikoolayy1 P K Patrik_Jonsson Philip Jönsson Rob_Carr Rodolfo_Nützmann Rodrigo_Albuquerque Samstep SanjayP ScottE Sebastian Maniak Stefan_Klotz StephanManthey Tyler.HattonJanuary 2022 DevCentral Survey
Hi everyone, As 2021 winds down, we are excited about some changes coming to DevCentral in the new year. We'd like your input as we work to make DevCentral a community you WANT to visit. To that end, we hope that you'll fill out this 6-question survey. That's all for now! Stay tuned over the next few weeks to learn more about the changes. Happy new year! ~your DevCentral Team
What is NGINX?
Introduction NGINX started out as a high performance web-server and quickly expanded adding more functionality in an integrated manner. Put simply, NGINX is an open source web server, reverse proxy server, cache server, load balancer, media server and much more. The enterprise version of NGINX has exclusive production ready features on top of what's available, including status monitoring, active health checks, configuration API, and live dashboard for metrics. Think of this article as a quick introduction to each product but more importantly, as our placeholder for NGINX articles on DevCentral. If you're interested in NGINX, you can use this article as the place to find DevCentral articles broken down by functionality in the near future. By the way, this article here has also links to a bunch of interesting articles published on AskF5 and some introductory NGINX videos. NGINX as a Webserver The most basic use case of NGINX. It can handle hundreds of thousands of requests simultaneously by using an event-drive architecture (as opposed to process-driven one) to handle multiple requests within one thread. NGINX as a Reverse Proxy and Load Balancer Both NGINX and NGINX+ provide load balancing functionality and work as reverse-proxy by sitting in front of back-end servers: Similar to F5, traffic comes in, NGINX load balances the requests to different back-end servers. In NGINX Plus version, it can even do session persistence and health check monitoring. Published Content: Server monitoring - some differences between BIG-IP and NGINX NGINX as Caching Server NGINX content caching improves efficiency, availability and capacity of back end servers. When caching is on, NGINX checks if content exists in its cache and if that's the case, content is served to client without the need to contact back end server. Otherwise, NGINX reaches out to backend server to retrieve content. A content cache sits between a client and back-end server and saves copies of pre-defined cacheable content. Caching improves performance as strategically, content cache is supposed to be closer to client. It also has the benefit of offloads requests from back-end servers. NGINX Controller NGINX controller is a piece of software that centralises and simplifies configuration, deployment and monitoring of NGINX Plus instances such as load balancers, API gateway and even web server. By the way, NGINX Controller 3.0 has just been released. Published Content: Introducing NGINX Controller 3.0 Setting up NGINX Controller Use of NGINX Controller to Authenticate API Calls Publishing an API using NGINX Controller NGINX as Kubernetes Ingress Controller NGINX Kubernetes Ingress Controller is a software that manages all Kubernetes ingress resources within a Kubernetes cluster. It monitors and retrieves all ingress resources running in a cluster and configure the corresponding L7 proxy accordingly. There are 2 versions of NGINX Ingress Controllers. One is maintained by the community and the other by NGINX itself. Published Content: Lightboard Lesson: NGINX Kubernetes Ingress Controller Overview NGINX as API Gateway An API Gateway is a way of abstracting application services interaction from client by providing a single entry-point into the system. Clients may issue a simple request to the application, for example, by requesting to load some information from a specific product. In the background, API gateway may contact several different services to bundle up the information requested and fulfil client's request. NGINX API management module for NGINX Controller can do request routing, composition, applying rate limiting to prevent overloading, offloading TLS traffic to improve performance, authentication, and real-time monitoring and alerting. NGINX as Application Server (Unit) NGINX Unit provides all sorts of functionalities to integrate applications and even to migrate and split services out of older monolithic applications. A key feature of Unit is that we don't need to reload processes once they're reconfigured. Unit only changes part of the memory associated to the changes we made. In later versions, NGINX Unit can also serve as intermediate node within a web framework, accepting all kinds of traffic and maintaining dynamic configuration and acting as a reverse proxy for back-end servers. NGINX as WAF NGINX uses ModSecurity module to protect applications from L7 attacks. NGINX as Sidecar Proxy Container We can also use NGINX as side car proxy container in Service Mesh architecture deployment (e.g. using Istio with NGINX as sidecar proxy container). A service mesh is an infrastructure layer that is supposed to be configurable and fast for the purposes of network-based interprocess communication using APIs. NGINX can be configured as a Sidecar proxy to handle inter-service communication, monitoring and security-related features. This is a way of ensuring developers only handle development, support and maintenance while platform engineers (ops team) can handle the service mesh maintenance.
Guarding privacy and security using API Gateways: Implement the Phantom Token Flow using NGINX and the Curity Identity Server
By Damian Curry, NGINX Business Development Technical Director, F5, and Michal Trojanowski, Product Marketing Engineer, Curity ---- In today’s world, APIs are ubiquitous, either in communication between-backend services or from front ends to back ends. They serve all kinds of purposes and come in different flavors and can return data in various formats. The possibilities are countless. Still, they all share one common trait – an API needs to be secure. Secure access to an API should be paramount for any company exposing them, especially if the APIs are available externally and consumed by third-party clients. To help organizations address the critical topic of API security, OWASP has provided guidelines to ensure safety. In 2019, OWASP released a “top 10” compilation of the most common security vulnerabilities in APIs. OAuth and JWT as a Sign of Mature API Security Modern, mature APIs are secured using access tokens issued according to the OAuth standard or ones that build on top of it. Although OAuth does not require the use of JSON Web Token (JWT) format for access tokens, per se, it is common practice to use them. Signed JWTs (JWTs signed using JSON Web Signing, JWS) have become a popular solution because they include built-in mechanisms that protect their integrity. Although signed JWTs are a reliable mechanism when the integrity of the data is considered, they are not as good in regards to privacy. A signed JWT (JWS) protects integrity of the data, while an encrypted one (JWE) protects also the privacy. A JSON Web Token consists of claims which can carry information about the resource owner – the user who has granted access to their resources or even about the API itself. Anyone in possession of a signed JWT can decode it and read the claims inside. This can produce various issues: You have to be really careful when putting claims about the user or your API in the token. If any Personally Identifiable Information (PII) ends up in there, this data essentially becomes public. Striving to keep PII private should be the goal of your organization. What is more, in some countries, privacy is protected by laws like GDPR or CCPA, and it can be an offense not to keep PII confidential. Developers of apps consuming your access tokens can start depending on the contents of your tokens. This can lead to a situation where making updates to the contents of your access tokens breaks the integrations with your API. Such a situation would be an inconvenience both for your company and your partners. An access token should be opaque to the consumers of your API, but this can be hard to enforce outside of your organization. By-reference Tokens and the Phantom Token Flow When there is a problem, there must be a solution. In fact, there are two ways to address these aforementioned problems: Encryption Use by-reference (AKA handle) tokens To protect the contents of JWTs from being visible, the token can be encrypted (thus creating a JWE inside the JWS). Encrypted content will be safe as long as you use a strong encryption algorithm and safeguard the keys used for encryption. To use JWEs, though, you need a Public Key Infrastructure (PKI), and setting one up can be a big deal. If you do not use PKI, you need another mechanism to exchange keys between the Authorization Server and the client. As JWEs also need to be signed, your APIs have to be able to access over TLS the keys used to verify and decrypt tokens. These requirements add considerable complexity to the whole system. Another solution is to use by-reference tokens – an opaque string that serves as a reference to the actual data kept securely by the Authorization Server. Using opaque tokens protects the privacy of the data normally kept in tokens, but now your services, which process requests, need some other way to get the information that was earlier available in the JWT’s claims. Here, the Token Introspection standard can prove useful. Using a standardized HTTP call to the Authorization service, the API can exchange the opaque token for a set of claims in JSON format. Exchanging an opaque token for a set of claims comes to the API just as easily as extracting the data from decoding a JWT. Because the API connects securely to the Authentication Server and trusts it, the response does not have to be signed, as is the case with JWTs. Still, in a world dominated by microservices, a problem may arise with the proposed solution. Have a look at this diagram below. Note that every service which processes the request needs to perform the introspection. When there are many services processing one request, the introspection must be done repeatedly or else the APIs would have to pass unsigned JSONs and trust the caller with the authorization data – which can lead to security issues. Situation in which every service calls the Authorization Server to introspect the token can put excess load on the Authorization Server, which will have to return the same set of claims many times. It can also overload the network between the services and the Authorization Server. If your API consists of numerous microservices, there is probably some kind of a reverse proxy or an API gateway standing in front of all of them, e.g., an instance of NGINX. Such a gateway is capable of performing the introspection flow for you. This kind of setup is called a Phantom Token Flow. The Authorization Server issues opaque tokens and serves them to the client. The client uses the token as any other token – by appending it to the request in the Authorization header. When the gateway receives a request, it extracts the opaque token from the header and performs the introspection flow, but it is a bit different from the flow performed by APIs. According to the specification, when the API introspects a token, it receives a JSON with the claims that would usually end up in a JWT. For an API, it is enough, as it trusts the Authorization Server and just needs the data associated with the access token, the claims do not have to be signed. For the API gateway, though, it’s not enough. The gateway needs to exchange the opaque token for the set of data but will be passing this data to the downstream services together with the request. The APIs must be sure that the claims they received from the gateway have not been tampered with. That’s why some Authorization Servers allow returning a JWT from the introspection endpoint. The API gateway can introspect the opaque token, but in the response, it gets a JWT that corresponds to the access token. This JWT can then be added to the request’s Authorization header, and the downstream APIs can use it as any other JWT. Thanks to this solution, APIs can securely pass the access token between themselves and at the same time have simple access to the claims, without the need of contacting the Authorization Server. Using the Phantom Token Flow allows you to leverage the power of JWTs inside your infrastructure while at the same time keeping high levels of security and privacy outside of your organization. The network workload is greatly reduced because now only the gateway needs to contact the Authorization Server - the introspection is done only once for every request. Also, the amount of work the gateway needs to do is limited – it does not have to parse the body or decode the JWT in order to decide whether the token is valid or has not expired. The gateway can depend only on the status code of the response from the Authorization Server. An OK response means that the token is valid and has not expired. The amount of network traffic can be reduced even further. The API gateway can cache the Authorization Server’s response for as long as the server tells it to (which will usually be the lifetime of the access token). If the opaque reference token is globally unique, this works as an ideal cache key, and the mapping in the cache can even be shared across any API. Implementing the Phantom Token Flow using NGINX and the Curity Identity Server At Curity, we have created an open-source module for NGINX to facilitate implementing the Phantom Token flow. The module takes care of performing the introspection and keeping the result in the cache (if caching is enabled). As values of the opaque token are globally unique, so the cache can be as well; there is no need to keep a reference on a per-client basis. The NGINX module is easy to use as all parameters can be set using standard NGINX configuration directives. The module can be built from source, but there are binaries available for a few different Linux distributions and NGINX versions. You can find them in the releases section on GitHub. Installing and Configuring the Module Once downloaded, the module needs to be enabled in your instance of NGINX unless you want to create your own build of NGINX with all your modules embedded. To enable the module, add this directive in the main context of your configuration: load_module modules/ngx_curity_http_phantom_token_module.so; Then you can apply the phantom token flow to chosen servers and locations. Here’s an example configuration that enables the Phantom Token Flow for an /api endpoint. The configuration assumes that: Your NGINX instance runs on a different host (nginx.example.com) than your Curity Identity Server instance (curity.example.com). The API that will eventually process the request listens on example.com/api. The Curity Identity Server’s introspection endpoint is /oauth/v2/introspection. Responses from introspection are cached by NGINX. http { proxy_cache_path /path/to/cache/cache levels=1:2 keys_zone=my_cache:10m max_size=10g inactive=60m use_temp_path=off; server { server_name nginx.example.com; location /api { proxy_pass https://example.com/api; phantom_token on; phantom_token_client_credential "client_id" "client_secret"; phantom_token_introspection_endpoint curity; } location curity { proxy_pass "https://curity.example.com/oauth/v2/introspection"; proxy_cache_methods POST; proxy_cache my_cache; proxy_cache_key $request_body; proxy_ignore_headers Set-Cookie; } } } Now, your NGINX gateway is capable of performing the Phantom Token Flow! Conclusion The Phantom Token Flow is a recommended practice for dealing with access tokens that are available publicly. The flow is especially effective when there are many microservices processing one request, which would otherwise have to introspect the tokens on their own. Using Phantom Tokens enhances your security and the privacy of your users, which can never be underestimated. With the help of NGINX as your gateway and the Curity Identity Server as the Authorization Server implementing the Phantom Token Flow is as simple as loading a module in your NGINX server! Sign up now for a free API Gateway Webinar to learn more about guarding privacy and security with Curity and NGINX
NGINX ingress proxy for Consul Service Mesh
Disclaimer: This blog post is an abridged (NGINX specific) version of the original blog by John Eikenberry at HashiCorp. Consul service mesh provides service-to-service connection authorization and encryption using mutual Transport Layer Security (mTLS). Secure mesh networks require ingress points to act as gateways to enable external traffic to communicate with the internal services. NGINX supports mTLS and can be configured as a robust ingress proxy for your mesh network. For Consul service mesh environments, you can use Consul-template to configure NGINX as a native ingress proxy. This can be beneficial when performance is of utmost importance. Below you can walk through a complete example setup. The examples use consul-template to dynamically generate the proxy configuration and the required certificates for the NGINX to communicate directly with the services inside the mesh network. This blog post is only meant to demonstrate features and may not be a production ready or secure deployment. Each of the services are configured for demonstration purposes and will run in the foreground and output its logs to that console. They are all designed to run from files in the same directory. In this blog post, we use the term ingress proxy to describe Nginx. When we use the term sidecar proxy, we mean the Consul Connect proxy for the internal service. Common Infrastructure The example setup below requires Consul, NGINX, and Python. The former is available at the link provided, NGINX and Python should be easily installable with the package manager on any Linux system. The Ingress proxy needs a service to proxy to, for that you need Consul, a service (E.g., a simple webserver), and the Connect sidecar proxy to connect it to the mesh. The ingress proxy will also need the certificates to make the mTLS connection. Start Consul Connect, Consul’s service mesh feature, requires Consul 1.2.0 or newer. First start your agent. $ consul agent -dev Start a Webserver For the webserver you will use Python’s simple built-in server included in most Linux distributions and Macs. $ python -m SimpleHTTPServer Python’s webserver will listen on port 8000 and publish an index.html by default. For demo purposes, create an index.html for it to publish. $ echo "Hello from inside the mesh." > index.html Next, you’ll need to register your “webserver” with Consul. Note, the Connect option registers a sidecar proxy with the service. $ echo '{ "service": { "name": "webserver", "connect": { "sidecarservice": {} }, "port": 8000 } }' > webserver.json $ consul services register webserver.json You also need to start the sidecar. $ consul connect proxy -sidecar-for webserver Note, the service is using the built-in sidecar proxy. In production you would probably want to consider using Envoy or NGINX instead. Create the Certificate File Templates To establish a connection with the mesh network, you’ll need to use consul-template to fetch the CA root certificate from the Consul servers as well as the applications leaf certificates, which Consul will generate. You need to create the templates that consul-template will use to generate the certificate files needed for NGINX ingress proxy. The template functions caRoots and caLeaf require consul-template version 0.23.0 or newer. Note, the name, “nginx” used in the leaf certificate templates. It needs to match the name used to register the ingress proxy services with Consul below. ca.crt NGINX requires the CA certificate. $ echo '{{range caRoots}}{{.RootCertPEM}}{{end}}' > ca.crt.tmpl Nginx cert.pem and cert.key NGINX requires the certificate and key to be in separate files. $ echo '{{with caLeaf "nginx"}}{{.CertPEM}}{{end}}' > cert.pem.tmpl $ echo '{{with caLeaf "nginx"}}{{.PrivateKeyPEM}}{{end}}' > cert.key.tmpl Setup The Ingress Proxies NGINX proxy service needs to be registered with Consul and have a templated configuration file. In each of the templated configuration files, the connect function is called and returns a list of the services with the passed name “webserver” (matching the registered service above). The list Consul creates is used to create the list of back-end servers to which the ingress proxy connections. The ports are set to different values so you can have both proxies running at the same time. First, register the NGINX ingress proxy service with the Consul servers. $ echo '{ "service": { "name": "nginx", "port": 8081 } }' > nginx-service.json $ consul services register nginx-service.json Next, configure the NGINX configuration file template. Set it to listen on the registered port and route to the Connect-enabled servers retrieved by the Connect call. nginx-proxy.conf.tmpl $ cat > nginx-proxy.conf.tmpl << EOF daemon off; master_process off; pid nginx.pid; error_log /dev/stdout; events {} http { access_log /dev/stdout; server { listen 8081 defaultserver; location / { {{range connect "webserver"}} proxy_pass https://{{.Address}}:{{.Port}}; {{end}} # these refer to files written by templates above proxy_ssl_certificate cert.pem; proxy_ssl_certificate_key cert.key; proxy_ssl_trusted_certificate ca.crt; } } } EOF Consul Template The final piece of the puzzle, tying things together are the consul-template configuration files! These are written in HCL, the Hashicorp Configuration Language, and lay out the commands used to run the proxy, the template files, and their destination files. nginx-ingress-config.hcl $ cat > nginx-ingress-config.hcl << EOF exec { command = "/usr/sbin/nginx -p . -c nginx-proxy.conf" } template { source = "ca.crt.tmpl" destination = "ca.crt" } template { source = "cert.pem.tmpl" destination = "cert.pem" } template { source = "cert.key.tmpl" destination = "cert.key" } template { source = "nginx-proxy.conf.tmpl" destination = "nginx-proxy.conf" } EOF Running and Testing You are now ready to run the consul-template managed NGINX ingress proxy. When you run consul-template, it will process each of the templates, fetching the certificate and server information from consul as needed, and render them to their destination files on disk. Once all the templates have been successfully rendered it will run the command starting the proxy. Run the NGINX managing consul-template instance. $ consul-template -config nginx-ingress-config.hcl Now with everything running, you are finally ready to test the proxies. $ curl http://localhost:8081 Hello from inside the mesh! Conclusion F5 technologies work very well in your Consul environments. In this blog post you walked through setting up NGINX to work as a proxy to provide ingress to services contained in a Consul service mesh. Please reach out to me and the F5-HashiCorp alliance team here if you have any questions, feature requests, or any feedback to make this solution better.My first CRD deployment with CIS
Summary F5 Container Ingress Services (CIS) now supports the use of Custom Resource Definitions (CRD's) in Kubernetes (K8s) as a method for configuring services on the BIG-IP. This article walks through my first time deploying and what I learned. Goal of this article The goal of this article is to demonstrate that you can easily configure services on BIG-IP using CRD's, and this makes the configuration of services on F5 BIG-IP feel like a native Kubernetes experience, since you are creating Kubernetes resources. Other ways to configure BIG-IP from K8s Before F5 CIS supported CRD's, you could already use Ingress resources or ConfigMap resources to create VIPs and pool members on F5. Using CRD's now gives us a new way. As I found in my first deployment, the advantage of CRDs over Ingress resources is that F5 can develop their own CRD and have more attributes that can map to configurations in BIG-IP. And again based on my first impressions, the advantage of CRDs over ConfigMaps is that you don't need to create an AS3 declaration yourself. You now get to do that via a Kubernetes resource. This table summarizes my opinion only: Overview of my demo architecture Here's an image of the cloud infrastructure I built. Basically I have a pair of BIG-IP devices in Active/Standby config, and the K8s cluster is integrated via CIS. Here's a more application-focused image of what is deployed. NGINX Ingress Controller (open source version) is used here as an ingress controller running inside Kubernetes. What is this resource of kind VirtualServer doing? Here is the definition of my VirtualServer resource and the related TLSProfile resource: apiVersion: "cis.f5.com/v1" kind: VirtualServer metadata: name: hello-world-virtual-server namespace: nginx-ingress labels: f5cr: "true" spec: tlsProfileName: hello-world-tls # --> This will attach hello-world-tls TLSProfile virtualServerAddress: "10.0.2.100" pools: - path: / service: nginx-ingress servicePort: 80 monitor: type: http interval: 10 timeout: 31 send: "/" --- apiVersion: cis.f5.com/v1 kind: TLSProfile metadata: name: hello-world-tls namespace: nginx-ingress labels: f5cr: "true" spec: tls: termination: edge clientSSL: /Common/clientssl reference: bigip # --> reference profiles created in BIG-IP by User hosts: - hello-world.example.com Now, the above resources will create objects in the BIG-IP config. I'll take a screenshot of what's created in BIG-IP and highlight some fields that you can see were configured by the resources above. Run this demo yourself (code not supported) Here's a link to my Github repo if you want to see the resources created. I've included instructions for deploying this demo environment, for educational purposes only. I'll use this as a tool for showing others but this will not be a supported solution long term. Conclusion You can use CRD's to configure F5 services from within Kubernetes. This means the Kubernetes admins can configure BIG-IP themselves, without the traditional network admin who normally "owns" F5 being involved. This was possible before CRD's (with Ingress, ConfigMap, or Routes in OpenShift) but with CRD's we have a more native-feeling resource that F5 can now extend with future CIS development.
redirect from http (port 80) to https (port 443) not working
Hi, I am running nginx version: nginx/1.16.1 on CentOS Linux release 7.8.2003 (Core). When I hit https://marketplace.mydomain.com it works perfectly fine whereas when I hit http://marketplace.mydomain.com (port 80) does not get redirected to https://marketplace.mydomain.com (port 443). I have the below nginx.conf. server { > listen 443 ssl default_server; > #listen 80 default_server; > #server_name _; > server_name marketplace.mydomain.com; > ssl_protocols TLSv1.2; > ssl_certificate /etc/ssl/certs/ > marketplace.mydomain.com/fullchain1.pem; ssl_certificate_key > /etc/ssl/certs/marketplace.mydomain.com/privkey1.pem; > if ($scheme = http) { return 301 https://$server_name$request_uri; > } > ssl_ciphers > ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384; > ssl_prefer_server_ciphers on; > ssl_dhparam /etc/ssl/certs/marketplace.mydomain.com/dhparam.pem; > client_max_body_size 100M; > root /var/www/drupal/marketplace-v2/mpV2/web; > access_log /var/log/nginx/access.log; > error_log /var/log/nginx/error.log; > # Load configuration files for the default server block. > include /etc/nginx/default.d/*.conf; > location = /favicon.ico { > log_not_found off; > access_log off; > } I will appreciate it if someone can pitch in for help. Please let me know if you need any additional configurations and I look forward to hearing from you. Thanks in Advance. Best Regards, KaushalNGINX as an HTTP Load Balancer
Quick Intro You've probably used NGINX as a WebServer and you might be wondering how does it work as a load balancer. We don't need a different NGINX file or to install a different package to make it a load balancer. NGINX works as a load balancer out of the box as long as we specify we want it to work as a load balancer. In this article, we're going to introduce NGINX HTTP load balancer. If you need more details, please refer to NGINX official documentation. However, NGINX also supports TCP and UDP load balancing and health monitors which are not covered here. NGINX as a WebServer by default Once NGINX is installed, it works as a WebServer by default: The response above came from our client-facing NGINX Load Balancer-to-be which is currently just a webserver. Let's make it an HTTP load balancer, shall we? NGINX as a Load Balancer Disable WebServer Let's comment out the default file on /etc/nginx/sites-enabled to disable our local WebServer just in case: Then I reload the config: If we try to connect now, it won't work now because we disabled the default page: Now, we're ready to create the load balancer's file! Creating Load Balancer's file Essentially, the default NGINX config file (/etc/nginx/nginx.conf) already has an http block which references the /etc/nginx/conf.d directory. With that in mind, we can pretty much create our file in /etc/nginx/conf.d/ and whatever is in there will be in HTTP context: Within upstream directive, we add our backend servers along with the port they're listening to. These are the servers NGINX load balancer will forward the request to. Lastly, we create a server config for our listener with proxy_pass pointing to upstream name (backends). We now reload NGINX again: Lab tests NGINX has a couple of different load balancing methods and round robin (potentially weighted) is the default one. Round robin test I tried the first time: Second time: Third time: Fourth time it goes back to server 1: That's because the default load balancing method for NGINX is round robin. Weight test Now I've added weight=2 and I expect that server 2 will proportionally take 2x more requests than the rest of the servers: Once again I reload the configuration after the changes: First request goes to Server 3: Next one to Server 2: Then again to Server 2: And lastly Server 1: Administratively shutting down a server We can also administratively shut down server 2 for maintenance, for example, by adding the down keyword: When we issue the requests it only goes to server 1 or 3 now: Other Load Balancing methods Least connections Least connections sends requests to the server with the least number of active connections taking into consideration any optionally configured weight: If there's a tie, round robin is used, taking into account the optionally configured weights. For more information on least connections method, please click here. IP Hash The request is sent to the server as a result of a hash calculation based on the first 3 octets of Client IP address or the whole IPv6 address. This method makes sure requests from the same client are always sent to the same server, unless it's unavailable. Note: A hash from server that is marked down is preserved for when it comes back up. For more information on IP hash method, please click here. Custom or Generic Hash We can also define a key for the hash itself. In below example, the key is the variable $request_uri which represents the URI present in the HTTP request sent by client: Least Time (NGINX Plus only) This method is not supported in NGINX free version. In this case, NGINX Plus picks the server with lowest average latency + number of active connections. The lowest average latency is calculated based on an argument passed to least_time with the following values: header: time it took to receive the first byte from server last_byte: time it took to receive the full response from server last_byte inflight: time it took to receive full response from server taking into account incomplete requests In above example, NGINX Plus will make its load balancing decision based on the time it took to receive the full HTTP response from our server and will also include incomplete requests in the calculation. Note that /etc/nginx/conf.d/ is the default directory for NGINX config files. In above case, as I installed NGINX directly from Debian APT repository, it also added sites-enabled. Some Linux distributions do add it to make it easier for those who use Apache. For more information on least time method, please click here. Random In this method, requests are passed on to randomly selected servers but it's only ideal for environments where multiple load balancers are passing on requests to the same back end servers. Note that least_time parameter in this method is only available on NGINX Plus. For more details about this method, please click here.
