Configuring OCSP Stapling on BIG-IP
When setting up an SSL connection the cert tells you its expiration, but how do you tell if the SSL Cert has been revoked? There are multiple ways to do this. The first is the Certificate Revocation List (CRL). When the client requests the CRL, the cert provider will send a list of revoked certificates. The problem is that over time this list becomes very large as certs are added. Case in point, heartbleed casued a spike in the number of certs added to CRL list.
To get around these large lists when all the client is looking for is one cert is the Online Certificate Status Protocol (OCSP.) This allows the client to query the CA for the validity of the cert in question. There are still problems, for example there can be a large number of clients that query the OCSP responder. If the OCSP responder takes too long and times out, then most clients will ignore the problem and move on.
With OCSP stapling the client can ask the server to staple the OCSP response with the SSL server certificate response from the server. This allows the server to deal with the OCSP message and not require the client to query the OCSP responder. Doing this offloads the process from the OCSP server to the server, spreading the load around, making it quicker in most cases and limiting OCSP timeout. Once the server gets the OCSP response it can cache the response for the valid period, so one request to the OCSP server will serve many responses to the client.
Requirements for OCSP Stapling on BIG-IP
- The OSCP responder for the cert in question. In this case startSSL.com allows you to generate a real cert for the test. The OSCP responder is http://ocsp.startssl.com
- You need to make sure the cert has the full chain for the client and f5 to trust
- Test both the responder and the chain with the following openssl command
To test the SSL chain run the following command:
openssl verify -CAfile \:Common\: \:Common\:"cert in question"
The output should be like the following
:Common:"cert in Question": OK
Now test OCSP response from openssl
openssl ocsp -issuer \:Common\: -cert \:Common\: -text -header "HOST" "host name of ocsp" -url http:/// -CAfile \:Common\:
You should get an output like below:
OCSP Response Data: OCSP Response Status: successful (0x0) Response Type: Basic OCSP Response Version: 1 (0x0) Responder Id: C = IL, O = StartCom Ltd., OU = StartCom Certification Authority, CN = StartCom Class 1 DV Server CA OCSP Responder Produced At: Jan 14 18:19:02 2016 GMT Responses: Certificate ID: Hash Algorithm: sha1 Issuer Name Hash: 5168159AB1999B3A55E860C022C9D519E0303D29 Issuer Key Hash: D7914E01C4B0BFF8C86793449CE733FAAD930CAF Serial Number: 346F8F9596F4A3AEB2B0C3AE693B5A7F Cert Status: good This Update: Jan 14 18:19:02 2016 GMT Next Update: Jan 18 18:29:02 2016 GMT
Once you have tested and know the cert and CA are correct you can connect to the correct OCSP responder now it time to setup the f5.
BIG-IP Configuration Steps
First we need to Create the DNS resolver this is located in Network -> DNS Resolvers. The defaults are fine here you just need to add a name
Next you will need to create a forward zone. For the name put . and enter the DNS server that the unit should use.
Now we can create the OCSP Stapling profile.
- For the DNS Resolver put the one in that you setup in the steps above
- For the Trusted Certificate Authorities this is the CA file you test with earlier
- Responder URL us the URL from earler that the f5 will direct all OCSP requests to.
- By default Sign Hash is set to SHA256 there responders that will not work with this and it will need to be set to SHA1
- Timeout is the amount of time till the f5 drop the connection to the OCSP server. This has to be smaller them the Handshake timeout in the SSL Profile.
- Status Age is the age of the from this update, most servers will have a range that this update is good for.
- By default the f5 1 day make sure you check your server to make sure the Status Age is large enough to handle this
- Cache Timeout this is how long the f5 will cache the response, default is Indefinite this allows the f5 to cache the request for the length of the validity period the server sets
Next step is to create the SSL profile and add the Stapling Profile
Now after the ClientSSL profile is add to the VIP you are ready to test.
Testing it out
If your site is open to the Internet you can use tools like Qualys SSL Labs, or you can test with OpenSSL with the following command:
certificate_d # openssl s_client -connect VIP IP:443 -tls1 -tlsextdebug -status | grep -i "ocsp response" -B 5 -A 10
The output from a successful OCSP stapling
OCSP response: ====================================== OCSP Response Data: OCSP Response Status: successful (0x0) Response Type: Basic OCSP Response Version: 1 (0x0) Responder Id: C = IL, O = StartCom Ltd., OU = StartCom Certification Authority, CN = StartCom Class 1 DV Server CA OCSP Responder Produced At: Jan 14 18:19:02 2016 GMT Responses: Certificate ID: Hash Algorithm: sha1 Issuer Name Hash: 5168159AB1999B3A55E860C022C9D519E0303D29 Issuer Key Hash: D7914E01C4B0BFF8C86793449CE733FAAD930CAF Serial Number: 346F8F9596F4A3AEB2B0C3AE693B5A7F Cert Status: good
Much thanks to Richard Harlan, one of F5's fine field systems engineer for contributing this write up!
- Sam_HallNimbostratus
We haven't looked at it for a while, but I think we ended up determining that the F5 was caching a bad or failed OSCP response which upset Firefox. This was on version 12. I also kind of lost interest when I realised that OSCP stapling doesn't support the chain certificate anyway, so it won't eliminate the delay, only halve it. That coupled with the risk of denying access to the site it's supposed to speed up, I found it pretty disappointing overall and there doesn't seem to be any activity towards adding support for stapling chain certs since draft was raised 6 years ago... https://bugzilla.mozilla.org/show_bug.cgi?id=611836
- webguy96Nimbostratus
Can you perhaps try the following and see if it gives you any better results from within bash or list the specific version of F5 you are running as this was put together against v13.1.5.
# set directory path to location of certs (change to custom partition if not stored in /Common)
cd /config/filestore/files_d/Common_d/certificate_d
# obtain the full name of cert + ca bundle
ls
# test OCSP response using openssl
# -issuer & -CAfile are both using a cert bundle containing both the Intermediate + Root CA for DigiCert
openssl ocsp -no_nonce -header Host ocsp.digicert.com -issuer \:Common\:digicert_bundle.crt_198653_1 -cert \:Common\:your.website.com.crt_234232_1 -url http://ocsp.digicert.com -CAfile \:Common\:digicert_bundle.crt_198653_1
- gscholzNimbostratus
Hi Jason!
Could it be the migration of devcentral destroyed the code snippets above? As an example:
openssl verify -CAfile \:Common\: \:Common\:"cert in question"
If I use the command as specified I get the following errors:
Error loading file :Common:
47706134397416:error:02001002:system library:fopen:No such file or directory:bss_file.c:165:fopen(':Common:','r')
47706134397416:error:2006D080:BIO routines:BIO_new_file:no such file:bss_file.c:168:
47706134397416:error:0B084002:x509 certificate routines:X509_load_cert_crl_file:system lib:by_file.c:258:
If I use the command like this:
openssl verify -CAfile \:Common\:"ca cert in question" \:Common\:"cert in question"
I get the following output:
:Common:"cert in question": C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
error 2 at 2 depth lookup:unable to get issuer certificate
The individual certificates I use seem to be okay though:
# openssl verify \:Common\:"cert in question"
:Common:"cert in question": OK
# openssl verify \:Common\:"ca cert in question"
:Common:"ca cert in question": OK
Also, I found that I have to run the command from the /config/filestore/files_d/Common_d/certificate_d directory if I don't want to specify the complete path, and I think adding this information to your article might be helpful.
- Torsten_SorgerNimbostratus
@Sam: I would expect this too but after waiting for roughly one day the certificate was still shown as valid until I manually deleted the OCSP cache...
Regarding the timeout of OCSP stapling I tested against the website of our CA:
echo HEAD / HTTP/1.0 | openssl s_client -connect www.digicert.com:443 -status 2> /dev/null | grep -A 17 'OCSP response:' | grep -B 17 'Next Update'
Result:
OCSP response: ====================================== OCSP Response Data: OCSP Response Status: successful (0x0) Response Type: Basic OCSP Response Version: 1 (0x0) Responder Id: 3DD350A5D6A0ADEEF34A600A65D321D4F8F8D60F Produced At: Oct 30 00:12:54 2017 GMT Responses: Certificate ID: Hash Algorithm: sha1 Issuer Name Hash: 49F4BD8A18BF760698C5DE402D683B716AE4E686 Issuer Key Hash: 3DD350A5D6A0ADEEF34A600A65D321D4F8F8D60F Serial Number: 0793EC89595DBA606D1FD9F7BE389802 Cert Status: good This Update: Oct 30 00:12:54 2017 GMT Next Update: Nov 5 23:27:54 2017 GMT
So it seems the timeout is seven days which I find rather irritating, or is this just a fixed renewal which is independent of revoking certificates? Nevertheless I did now set the timeout to 1800 seconds and see how this will perform...
Any further advise appreciated 🙂
- Sam_HallNimbostratus
@Torsten, I think you'll find that after a short time the cached response would be invalid as I'm lead to believe the OCSP response has it's own expiry. But I'm new to this OCSP stapling so maybe someone with more experience could chime in.
I generally don't revoke certificates, I just destroy them as best I can along with the private keys after I've got a replacement cert+key pair setup on the F5. The revocation process is more complicated and less reliable than starting over. I'd only use revocation if a private key was stolen.
I eventually got rid of my Firefox error, I think it had something to do with the order of the certificates. It turned out that my stapled OCSP data was actually useless. Browsers other than Firefox just ignored it and fetched it from the source. Once I got it working, I found still there was no chain certificate getting stapled so I figure I'll disable it for now as it seems like a flakey feature.
- Torsten_SorgerNimbostratus
I am wondering on the "Response Caching => Timeout: Indefinite" setting: Yesterday I revoked some certificate and this morning it was still shown as ok. When I deleted the OCSP cache for that certificate the status was updated to 'revoked' which is fine. If you follow the advice to set it to "Indefinite" - wouldn't this render OCSP stapling pretty much useless as the certificate will only be checked once then the cached response is used forever no matter if the certificate is still valid or revoked?
What timeout do you guys set on this usually? 1800-3600 seconds?
- Sam_HallNimbostratus
I have the same question as Jie.
When I follow the instructions, OCSP Stapling appears to be configured, SSL Labs reports "OCSP Stapling: Yes". However, Chrome still does it's own OCSP checks and Firefox throws an error MOZILLA_PKIX_ERROR_OCSP_RESPONSE_FOR_CERT_MISSING.
I have an EV certificate with an intermediate chain certificate issued by QuoVadis. It seems that the OCSP Stapling profile doesn't support this certificate configuration. Or am I missing something?
- JGCumulonimbusHow to do this for a certificate chain with an intermediate CA cert?