Securing SSL Keys on your BIG-IP

Losing your keys is a real problem

While losing your car keys is indeed a pain, I mean losing your Web Server Keys. Lost keys can expose your website to a Man in The Middle (MiTM) Attack. While in the middle, an attacker can use those keys to decrypt the traffic between your clients and your servers.

All of the data is open to be read by the hacker.

How can we help make sure you're not the poor soul in the picture?

Let's start with a little Crypto background.

What is a Cryptographic key?

A cryptographic key is pretty much like any other key. It unlocks a door; in this case, the door is a ciphered piece of text.

I'll save the mechanics for what enciphering and deciphering are for another time, but suffice to say when traffic is encrypted, we need to encrypt it in such a way that it cannot be easily decrypted. In asymmetric encryption, there are two files – one is a certificate (the lock on the door—also known as the “public key”), and the other one is the key (the, uh, key?--also known as the “private key”). 

The certificate is the public part anybody can have. We share that readily, possibly through infrastructure that automatically transfers the info or delivers the certificate over a handshake or an email. 

An example of an RSA certificate file … slightly obfuscated because I am somewhat paranoid.

The key itself is what stores the secret component, and it is the file we need to protect.

An example of an RSA key file.

Storing Cryptographic Files

We can protect our key by storing it somewhere securely and ensuring that access is extremely limited. There are tools, like hardware security modules (HSM) that do this for us, and there are standards like Federal Information Processing Standard (FIPS) 140 (https://csrc.nist.gov/publications/detail/fips/140/3/final) which define many of the specifications around storing keys, including access and retention. When there are 1000s of keys to manage, it makes sense to invest in these tools. For others, we might just want to store them in a secure location, and there are several ways to do that.

File Storage

Indeed, the first is file access controls. We can limit access to these files using ACLs and read/write protection, and we definitely should do that. By default, on Linux distributions, you haveto correctly set the file permissions for some applications to even use your keys.

Properly setting the file permissions is necessary, but with this mechanism alone, anyone who is able to get the necessary access to any of the systems storing the key can copy the key—it is just a small string of text--and do whatever they want with it. While gaining access to a BIGIP is not simple, there are many ways this can be done such as a disgruntled administrator, a vulnerability, a weak access password, and so on.

Further, OpenSSL (www.openssl.org) provides a means to add a passphrase to this file so that instead of storing the file data as plain text, the file itself is enciphered with a passphrase even to read it.

Shown here is a file – the same "key" even – without and then with a passphrase:

Plaintext key file.

You will notice that in the image above, we can read the key's text is displayed right after the header – that is THE KEY itself.

Key file protected with an OpenSSL Passphrase enciphered with AES256.

However, in the next image, there's a "Proc-Type:" and "DEK-Info:" header. The text of the file is NOT the actual key but an enciphered version of the key.

Doing this means that, even with file access, an attacker would still need to get the Passphrase to use the key. It adds an extra factor of authentication to use the file. It is not as secure as a Hardware Security Module, but it is at least another layer of protection from unwanted access to your keys.

Now, note that the key itself has not changed. The file is the only change. Once you read the file with the proper access, the resulting key string remains the same.

Passphrases and TMOS

Within the BIG-IP configuration, these files are stored in an unencrypted version or an encrypted version depending on whether or not a Passphrase was supplied when they were loaded. To protect the Passphrase-protected files, TMOS includes Secure Vault (https://support.f5.com/csp/article/K73034260), which uses a "Master Key" which encrypts the file passphrases into the BIG-IP configuration. TMOS stores the OpenSSL Passphrase, but the Passphrase itself is stored in an encrypted format in the configuration files. To access the keys, you now need both the required TMOS access permissions and the Passphrase to read the contents.

So, merely gaining access to the system is not enough to steal the key and use it to decrypt confidential data! Neat, huh?

 

But Security is HARD!!!

Yup, security is hard, and I would say moderately annoying, and a lot of people skip this step of adding the Passphrase to their keys. 

It is easier to store them as plain text, which allows loading them into TMOS without a Passphrase. When the keys are used in a Client or Server SSL profile, that pesky Passphrase isn't required. 

More comfortable, sure, but this isn't a smart or secure way to work.

Plaintext keys came up recently with a customer. The customer had conducted a thorough security audit and determined that having keys available in plaintext on the file system was a possible threat to customer data integrity. This finding led to a requirement that all keys across the organization need to use a Passphrase.

 

Great idea, but how could they retrofit the 100s of keys already loaded without a Passphrase?

Luckily, OpenSSL provides a mechanism for managing the Passphrase associated with a key. You can change, add, or remove the Passphrase with a simple command.

Here's the command if you want to copy it:

openssl rsa -aes256 -in devcentral-example-key-open -out devcentral-key-protected

Using this command, OpenSSL takes the "in" file and adds a Passphrase to the "out" file, enciphering it in the desired format specified (AES256).

Fortunately, this is possible using the BIG-IP RESTful API and a few shell calls to manage the underlying files. The steps are:

• Copy the keys.
• Run the OpenSSL command to add a passphrase and encipher a copy of the file.
• Load the new, enciphered version of the key onto the BIG-IP.
• Get a list of the SSL Client and Server profiles using the plaintext key.
• Update these profiles with the new name of the encrypted key and Passphrase.
• Optionally remove the plaintext version of the key.

I have built a sample script that does all this for you! I've posted it on my Github here https://github.com/pmscheffler/securekey, and you're welcome to take a copy and give it a try. 

Since it uses your keys, I would suggest thorough testing and making sure you're not using it in Production the first time out!

Protecting the Keys to Your Door

If you haven't adopted Passphrases on your keys because you thought it was too much effort to maintain, I hope you take a moment to check out the script and see that it isn't a lot to manage them in this manner.

If you find the script useful or have another method you use – please let me know in the comments below.

 

Hopefully, you never have a case where your customer data is exposed in a Man in the Middle breach because you lost your keys.