How to Troubleshoot SNI
SNI, Servername Indication, allows you to re-use a single IP address for many SSL certificates. The following article will highlight steps that I use to debug issues with SNI.
Review of SNI
SNI is part of the SSL/TLS “hello” that occurs in cleartext. This allows the BIG-IP/NGINX to select the appropriate SSL certificate or policy to apply when a SSL connection occurs. Prior to SNI you would need to allocate a separate IP address for each SSL certificate, now you can pile on many many certificates on a single IP address and use SNI to separate requests.
The Trouble with SNI
That sounds great! But how do you simulate a SNI connection? Netcat will fail you since you need to negotiate a SSL connection. There’s two tools that I use to simulate a connection using SNI. The first is openssl and the second is curl.
Using OpenSSL s_client
Many devices (including BIG-IP or your standard Linux box) will have the “openssl” command-line tool installed by default. If you read the manpage you’ll see the definition (for the millennial readers, a manpage is like a README.md before MD was a thing):
A “generic SSL/TLS client”, perfect! If you run “openssl s_client -h” you’ll see a nifty argument “-servername”
This allows you to set the SNI name to the value of your choice. So now you can run:
openssl s_client -connect ip:port -servername sni_value.example.com
Using curl
Curl is like your high school friend that keeps on showing up on your social media feed; you don’t want to see them all the time, but it’s nice to know that they’re doing well. Perhaps that analogy doesn’t work, but the following does.
Using curl you can add the “—resolve” flag to insert the SNI name.
The format would be:
curl –resolve sni_value.example.com:port:ip https://sni_value.example.com:port
What about the server?
In this article we focused on client-side tools to debug SNI issues, but you can also add additional server-side logging to ensure that you are getting the correct response. One method that you could stand-up a quick test server is to use openssl s_server. The following is an example from my Ubuntu workstation.
# Start the server $ openssl s_server -cert server.pem -key server.key -tlsextdebug # Connecting from openssl s_client $ openssl s_client -connect localhost:4433 -servername sni_value.example.com # Connecting from curl $ curl -k https://sni_value.example.com:4433/ --resolve sni_value.example.com:4433:127.0.0.1
Looking at the output from openssl s_server you can see that both openssl/curl are sending the desired SNI value.
What about using a Host header?
A comment from my YouTube video asks how this is better than a Host header.
My answer is below.
It depends. If the server/proxy will also look at host headers; then it could be OK to omit the servername. You may end up with the "default" SSL certificate instead of the one that matches the SNI value. The following is an example where it may not work.
Here's an extended example where you could see an issue with not using SNI. This gets verbose towards the end, but hopefully you/others find it helpful.
# server w/ "default.example.com" and "f5.example.com" certificates. requests for "f5.example.com" require SNI openssl s_server -cert default.example.com.crt -cert2 f5.example.com.crt -key server.key -key2 server.key -servername f5.example.com -servername_fatal -www # openssl client no SNI value $ openssl s_client -connect localhost:4433 -quiet depth=0 C = US, ST = WA, L = Seattle, O = F5 Networks, CN = default.example.com verify error:num=18:self signed certificate verify return:1 depth=0 C = US, ST = WA, L = Seattle, O = F5 Networks, CN = default.example.com verify return:1 # # note that default.example.com cert is returned and not f5.example.com # # openssl client w/ SNI value $ openssl s_client -connect localhost:4433 -servername f5.example.com -quiet depth=0 C = US, ST = WA, L = Seattle, O = F5 Networks, CN = f5.example.com verify error:num=18:self signed certificate verify return:1 depth=0 C = US, ST = WA, L = Seattle, O = F5 Networks, CN = f5.example.com verify return:1 # # note that f5.example.com cert is returned # # curl using host headers, errors out $ curl -k https://localhost:4433 -H host:f5.example.com curl: (35) error:14004458:SSL routines:CONNECT_CR_SRVR_HELLO:tlsv1 unrecognized name # # errors out b/c curl is sending a different SNI value than f5.example.com, this is visible on the server # # Hostname in TLS extension: "localhost" #4537142892:error:140270E2:SSL routines:ACCEPT_SR_CLNT_HELLO_C:clienthello #tlsext:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-47.11.1/libressl-2.8/ssl/ssl_srvr.c:1050: # # curl using SNI value + host headers via --resolve parameter $ curl -k https://f5.example.com:4433 --resolve f5.example.com:4433:127.0.0.1 -v * Added f5.example.com:4433:127.0.0.1 to DNS cache * Hostname f5.example.com was found in DNS cache * Trying 127.0.0.1... ... * Server certificate: * subject: C=US; ST=WA; L=Seattle; O=F5 Networks; CN=f5.example.com ... > GET / HTTP/1.1 > Host: f5.example.com:4433 > User-Agent: curl/7.64.1 > Accept: */* > * HTTP 1.0, assume close after body < HTTP/1.0 200 ok < Content-type: text/html < <HTML><BODY BGCOLOR="#ffffff"> <pre> s_server -cert default.example.com.crt -cert2 f5.example.com.crt -key server.key -key2 server.key -servername f5.example.com -servername_fatal -www Secure Renegotiation IS supported Ciphers supported in s_server binary .... # # response truncated for readability. note response and correct cert of f5.example.com #
When this comes in handy
During a reverse proxy connection this can come in handy in verifying that the client is able to negotiate correctly with the proxy and that the proxy is also able to connect to the backend. Let me know if the comments whether you found this helpful or if you have other ways that you’ve tested SNI connections. Thanks!
