Understanding the Authenticate Name Option on Server SSL profile on BIG-IP

Quick Intro

Have you ever wondered what this little option on Server SSL profile really does in practice?

This is what this article is all about.

If you're only interested about what I learnt during my lab tests, please feel free to read just Lab Test Results section.

Otherwise, enjoy the lab walkthrough :)

Lab Test Results

In case you just want to know what it is, this setting looks at both commonName and subjectAltName extension and if whatever name set on Authenticate Name doesn't match the name in any of these 2 fields I mentioned, certificate authentication fails and BIG-IP resets connection.

Note: for Authenticate Name to work, Server Certificate has to be set to Require, i.e. BIG-IP should be configured to check the validity of Server Side certificate. On top of that, ca-file (Trusted Certificate Authorities in the GUI) is also required to be set, otherwise BIG-IP has no trusted Root CA list to validate server's certificate.

What it is

This is like another layer of authentication. When server sends us a Certificate and peer-cert-mode is set to require, BIG-IP looks up Root CA present in ca-file and confirms that server's certificate is trusted. Then, when authenticate-name is also set, BIG-IP checks subjectAltName extension and commonName for a match of what we've typed in this field. If no match is found, certificate authentication fails and we do not trust certificate. Otherwise, certificate is trusted and we proceed with handshake.

Lab Test

When Authenticate Name does not match Certificate's commonName or subjectAltName

I created an end-entity X.509v3 TLS Certificate and set commonName (CN) to server1.rodrigo and subjectAltName extension to server01.rodrigo as seen below:

I set Server Certificate (peer-cert-mode in tmsh) to require, added the Root CA that signed back-end server's certificate to BIG-IP's Trusted Certificate Authorities (ca-file in tmsh) and authentication-name to fail.rodrigo:

On Wireshark, we see that authentication fails as soon as we receive Certificate message from server:

Note: If we want BIG-IP to display the specific alert such as unknown_ca above, we need to disable generic-alert on Server SSL settings.

It fails because fail.rodrigo is neither in CN nor in subjectAltName. I had set server1.rodrigo instead, remember?

Let's break it down into more details:

• back-end server sends Certificate message to BIG-IP
• because peer-cert-mode is set to require, BIG-IP looks up the Root CA list in ca-file (root-ca.crt here)
• BIG-IP answers the following question: was back-end certificate signed by any of the certificates listed in root-ca.crt?
• If not, authentication fails immediately and we never get to use authenticate-name
• In this case it was, so BIG-IP moves on to check if fail.rodrigo is in either commonName or subjectAltName fields of back-end's X.509v3 certificate
• Because fail.rodrigo doesn't match server1.rodrigo, authentication fails and BIG-IP resets connection.

When Authenticate Name matches Certificate's commonName or subjectAltName

I've now set authenticate-name to server1.rodrigo and TLS handshake suceeds:

Let's break it down into more details:

• back-end server sends Certificate message to BIG-IP
• because peer-cert-mode is set to require, BIG-IP looks up the Root CA list in ca-file (root-ca.crt here)
• BIG-IP answers the following question: was back-end certificate signed by any of the certificates listed in root-ca.crt?
• If not, authentication fails immediately and we never get to use authenticate-name
• In this case it was, so BIG-IP moves on to check if server1.rodrigo is in either commonName or subjectAltName fields of back-end's X.509v3 certificate
• Because server1.rodrigo matches server1.rodrigo in both commonName and SubjectAltName fields, authentication succeeds and TLS handshake proceeds

If you ever wondered where to find commonName and SubjectAltName on TLS headers, here's where they are:

The above snippet is from back-end's Certificate message that is sent to BIG-IP as part of TLS handshake.

Hope that's helpful.