Custom HTTPS External Monitor

Happy to step aside and feature a community member's content as a guest article for this post. Please welcome Leonardo Souza to the ranks of DevCentral authors!

Introduction

If you have upgraded a F5 device (BIG-IP) before, you probably have faced SSL/TLS compatibility issues. That is not exclusive to F5 devices, but a general problem every time a new software version or functionality is released. In theory things should be compatible with previous version, but in reality most of the time it is not. When talking about encryption we use the term SSL, however when we start to discuss the technical details we need to differentiate SSL and TLS.

I will not extend the SSL/TLS topic, as there is a good article in DevCentral that explains how a F5 device uses SSL/TLS. Basically, TLS 1.0 is defined in RFC 2246 in 1999, TLS 1.1 RFC 2246 in 2006, TLS 1.2 in RFC 4346 in 2008, and TLS 1.3 still in draft. Searching the F5 release notes, TLS 1.2 started to be supported in 10.2.3 and 11.0.0, this was around 2010/2011. Starting in 10.2.0 we started to have the option to use the clientssl-insecure-compatible and serverssl-insecure-compatible, to allow the use of insecure SSL/TLS options. The insecure profiles are very well known today, and they provide a quick workaround for many SSL/TLS problems after the upgrade.

However, the insecure profiles apply only to traffic from users to servers via the F5 device. F5 devices must also monitor the servers, and sometimes because security requirements, both the client side and server sides must be encrypted. In this case F5 devices must monitor the servers using SSL/TLS, with the options that the servers are capable to accept. F5 devices use the daemon bigd to monitor nodes and pool members, and bigd works with OpenSSL to perform HTTPS monitoring. When OpenSSL changes the behaviour, bigd consequently changes as well. In 2012 OpenSSL 1.0.1 was released with support for TLS 1.1 and 1.2. BIG-IP version 11.4.1 uses OpenSSL version 0.9.8y, version 11.5.0 uses 1.0.1e. This is where the things started to complicate, and is what I’m trying to address in this article.

Problem

As expected when using software to provide a security layer, bigd/OpenSSL will try to use the highest SSL/TLS version available, and the server should either accept that version or suggest a different version. However some servers that don’t support TLS v1.2 are not able to downgrade to TLS 1.1 or TLS1.0, and they just terminate the connection. This behavior translates to a pool member been marked down after the timeout defined.

With clientssl and serverssl profiles we can change both the protocol version and the ciphers used. We can do that either changing the ciphers or using the options list, both in the profile configuration. However these options are handled by TMM (also OpenSSL for COMPAT, but let’s skip that), that is F5 TMOS Kernel.

As explained, bigd is responsible for monitoring tasks, and uses OpenSSL. The problem is that OpenSSL does not allow changing the protocol using the cipher string. OpenSSL have specific options to change the protocol version, like -tls1_2 to force the use of TLS 1.2. That limits the use of built-in HTTPS monitor, as we can’t force bigd to use TLS 1.0 for example. If we can’t use the build-in monitor, we need to use an external monitor to workaround the limitation.

Solution

Using an external monitor we can force OpenSSL to use the protocol version that is compatible with the server, and also change the cipher in case is needed. Curl is not a valid option here, because the options tlsv1.0/tlsv1.1/tlsv1.2 were only added in version 7.34.0 that was release end of 2013. At the time I’m writing this article the latest BIG-IP version is 12.1.1, and uses curl 7.19.7.

The script has extra information as comments, and should be simple to use. I will not go over the details of the script, or how to write external monitors, as there are multiple articles in DevCentral about that (see resources section below.) The script is available here in the codeshare. The script expects some variables (the order does not matter in this case,) and it will use those variables in the OpenSSL command.

Variable: cipher
Example: ECDHE-RSA-AES256-SHA
The cipher defines with cipher or ciphers OpenSSL should use.

Variable: protocol
Example: tls1
Indicates witch protocol version OpenSSL should use.

Variable: send_string
Example: GET /
Similar as in the HTTP/HTTPS monitor, indicates the string that is sent to server.

Variable: receive_string
Example: Welcome
Similar as in the HTTP/HTTPS monitor, indicates the string we should receive from the server.

Variable: receive_string_down
Example: Moved
Mark pool member down when the string is received.

Variable: debug
Example: enable or disable
Enable or disable debug.
If not present, debug is disabled.

Limitations

The HTTPS monitor has an option to disable the member when a defined string is received, that functionality exists in this external monitor, however the pool member is marked down. I could disable the pool member using tmsh, but that triggers a configuration change status, and is easier just mark the pool member down. I have successfully tested the external monitor with multiple versions and also with route domains.