Forum Discussion
Zuke_254875
Altostratus
AltostratusSSL handshake failure during SSO
Both our production and non-production service desk applications use SSO.
User connects to application VIP, which redirects users to the SSO VIP on 443.
The F5 configuration for these two environments are identical:
SSL bridging with default Client SSL profile as parent. No customizations except for the certificate/key/bundle.
However in the non-prod environment, the SSL handshake cannot complete. tcpdump shows a fatal error, certificate unknown, even though this is the same cert/key on the SSO server.
When I browse directly to the SSO VIP, the application works as expected.
Currently the work-around is to have the non-prod ITSD application server bypass the F5 and go directly to the SSO app server rather than the F5.
- Kevin_Stewart
Employee
Is the SSO VIP doing client certificate authentication?
Is there an application behind that you're re-encrypting to?
Is the above a client side or server side capture?
ZukeCirrostratus
Kevin, my understanding is that this setting is on the Client SSL profile? The Client Certificate is set to ignore.
We are SSL bridging to the SSO app server.
The above is clientside.
- Kevin_Stewart
Okay, so to be clear, "SSL Bridging" generally defines where the F5 both decrypts and re-encrypts the traffic to the server, which requires both client and server SSL profiles. If I can assume that there is NO client certificate auth going on, on either side, then the "certificate unknown" error is going to come from the client after consuming the server's certificate.
Browsers aren't usually that strict about bad server certificates. Worst case you'll get a certificate warning and click-to-continue.
So then I'd ask,
- Does the app server require HTTPS?
- Do you have a server SSL profile also applied?
- And if yes to the above, which server SSL profile are you using?
- Zuke
The application requires HTTPS
The default serverssl profile is applied to the virtual server.
- Kevin_Stewart
Okay, so I would then ask that you SSLDUMP on both sides.
ssldump -AdNn -i [client or server side VLAN] port 443This will show you if there's any issues in the respective TLS handshakes.
Are there ANY OTHER differences between the environments? Are you using the same browser version in both?
- Zuke
Kevin, same browser in both.
Here is the output on the current config:
3 1 0.0011 (0.0011) C>SV3.3(198) Handshake ClientHello Version 3.3 random[32]= 5b 89 77 3d 0b b2 cc 13 45 41 6f 92 af bb 53 51 84 7b 22 0c 4b cc 71 cf 17 9d 1d 2b 77 01 cc b8 cipher suites TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 TLS_RSA_WITH_AES_128_CBC_SHA256 TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA TLS_RSA_WITH_AES_128_CBC_SHA TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA TLS_ECDH_RSA_WITH_AES_128_CBC_SHA TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 TLS_DHE_DSS_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA TLS_RSA_WITH_3DES_EDE_CBC_SHA TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA TLS_EMPTY_RENEGOTIATION_INFO_SCSV compression methods NULL 3 2 0.0012 (0.0000) S>CV3.3(2) Alert level fatal value handshake_failure 3 0.0012 (0.0000) S>C TCP FIN 3 0.0016 (0.0004) C>S TCP FINHere is what I see when the dev application owner points the redirect at my lab virtual server, which has the clientssl profile that is identical to the known-good production setup.
1 1 0.0012 (0.0012) C>SV3.3(161) Handshake ClientHello Version 3.3 random[32]= 5b 89 7a b6 35 98 e1 be de 21 a4 60 62 ef 00 9e 93 95 37 7c 4f db dc 02 c1 c4 0d ec 53 fe fb 8e cipher suites TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 TLS_RSA_WITH_AES_128_CBC_SHA256 TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA TLS_RSA_WITH_AES_128_CBC_SHA TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA TLS_ECDH_RSA_WITH_AES_128_CBC_SHA TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 TLS_DHE_DSS_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA TLS_RSA_WITH_3DES_EDE_CBC_SHA TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA TLS_EMPTY_RENEGOTIATION_INFO_SCSV compression methods NULL ERROR: Length mismatch - Kevin_Stewart
The second one is truncated with the "Length mismatch" error, but can I assume the second one works?
In any case, what you're seeing here is the server side rejecting the client right after the client hello message. The only thing that generally happens in this message is the client sending a preferred TLS protocol, a list of supported ciphers, and any number of TLS extensions (ex. SNI, EC points, etc.).
There is something in the client's client hello message that the server does not like, and is therefore closing the connection. At this stage it can only be a disagreement in one of these things: TLS protocol, ciphers, TLS extensions. I don't actually see the TLS extensions in the capture though.
So to clarify, who is the client and who is the server in this capture?
- Zuke
Kevin, both situations result in a failure.
In both instances, the F5 is the server.
I asked the customer if he can change the ciphers on the application servers, since I'm not allowed to make a change to my F5 configuration. Here is the config of the parent clientssl profile showing the defined ciphers, which I believe don't match up with any ciphers in the client hello
ltm profile client-ssl /Common/default_cert_clientssl_profile { app-service none cert /Common/default.crt cert-key-chain { default { cert /Common/default.crt key /Common/default.key } } chain none ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:AES256-G CM-SHA384:AES256-SHA256:!SHA1:!SSLv3:!TLSv1:!TLSv1_1 defaults-from /Common/clientssl inherit-certkeychain true key /Common/default.key passphrase none renegotiation disabled }When I compare the ciphers allowed in the clientssl profile and what was sent in the Client Hello packet, it looks to me like there are no ciphers that both sides agree upon.
Client Hello:
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_RSA_WITH_AES_128_CBC_SHA256
- TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
- TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
- TLS_RSA_WITH_AES_128_CBC_SHA
- TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
- TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA
- TLS_DHE_DSS_WITH_AES_128_CBC_SHA
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
- TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
- TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
- TLS_RSA_WITH_3DES_EDE_CBC_SHA
- TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA
- TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
- TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
- TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
- TLS_EMPTY_RENEGOTIATION_INFO_SCSV
Client SSL profile:
- ECDHE-RSA-AES256-GCM-SHA384
- ECDHE-ECDSA-AES256-GCM-SHA384
- ECDHE-RSA-AES256-SHA384
- ECDHE-ECDSA-AES256-SHA384
- DHE-RSA-AES256-GCM-SHA384
- DHE-RSA-AES256-SHA256
- AES256-GCM-SHA384:AES256-SHA256
- !SHA1
- !SSLv3
- !TLSv1
- !TLSv1_1
- Zuke
Alright, so we got a new SSL certificate and my theory does not appear to be correct. The clientssl parent profile has been changed to the clientssl-insecure-compatible
Here is a copy of the SSLDUMP:
New TCP connection 22: 192.168.105.92(59853) <-> 192.168.28.140(443) 22 1 0.0012 (0.0012) C>SV3.3(161) Handshake ClientHello Version 3.3 random[32]= 5b 8e d9 0f 64 f5 32 7b 1c 0b fc 92 41 a8 82 7e cb f4 c2 0b 6a 7f f6 f9 30 0a f8 5c f6 85 17 bc cipher suites TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 TLS_RSA_WITH_AES_128_CBC_SHA256 TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA TLS_RSA_WITH_AES_128_CBC_SHA TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA TLS_ECDH_RSA_WITH_AES_128_CBC_SHA TLS_DHE_RSA_WITH_AES_128_CBC_SHA TLS_DHE_DSS_WITH_AES_128_CBC_SHA TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 TLS_DHE_DSS_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA TLS_RSA_WITH_3DES_EDE_CBC_SHA TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA TLS_EMPTY_RENEGOTIATION_INFO_SCSV compression methods NULL 22 2 0.0036 (0.0024) S>CV3.3(87) Handshake ServerHello Version 3.3 random[32]= 75 aa a3 00 16 c9 d3 98 56 56 25 51 f6 1c 62 bc 2b a1 99 ea 0b 79 9b d9 9b b8 df c8 86 e8 7c 24 session_id[32]= aa 00 6c db 7f c5 60 3f 93 c9 50 e3 e6 0d 08 c7 34 e2 db 94 60 26 d7 30 42 be 6a a8 b6 9d 58 d7 cipherSuite TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA compressionMethod NULL 22 3 0.0036 (0.0000) S>CV3.3(4491) Handshake Certificate Subject CN=itsdtest.example.com Issuer DC=com DC=example CN=InternalCA Serial 5a 00 00 80 b0 f0 2d 42 8b 48 43 0b bf 00 01 00 00 80 b0 Extensions Extension: X509v3 Key Usage Extension: X509v3 Subject Key Identifier Extension: X509v3 Subject Alternative Name Extension: X509v3 Authority Key Identifier Extension: X509v3 CRL Distribution Points Extension: Authority Information Access Extension: 1.3.6.1.4.1.311.21.7 Extension: X509v3 Extended Key Usage Extension: 1.3.6.1.4.1.311.21.10 Subject DC=com DC=example CN=InternalCA Issuer CN=InternalCAroot Serial 44 00 00 00 03 5e eb 3e 5c 75 84 80 b2 00 00 00 00 00 03 Extensions Extension: 1.3.6.1.4.1.311.21.1 Extension: 1.3.6.1.4.1.311.21.2 Extension: X509v3 Subject Key Identifier Extension: 1.3.6.1.4.1.311.20.2 Extension: X509v3 Key Usage Extension: X509v3 Basic Constraints Critical Extension: X509v3 Authority Key Identifier Extension: X509v3 CRL Distribution Points Extension: Authority Information Access Subject CN=InternalCAroot Issuer CN=InternalCAroot Serial 1c 53 ec 09 e5 3f 49 ab 46 69 30 03 1c 2c eb 61 Extensions Extension: X509v3 Key Usage Extension: X509v3 Basic Constraints Critical Extension: X509v3 Subject Key Identifier Extension: 1.3.6.1.4.1.311.21.1 22 4 0.0036 (0.0000) S>CV3.3(333) Handshake ServerKeyExchange 22 5 0.0036 (0.0000) S>CV3.3(4) Handshake ServerHelloDone 22 6 0.0042 (0.0006) C>SV3.3(2) Alert level fatal value certificate_unknown 22 0.0042 (0.0000) C>S TCP FIN 22 0.0043 (0.0000) S>C TCP FIN - Kevin_Stewart
Zuke, you may have answered this but I didn't see it. Can I assume this is a non-browser client? A browser would usually issue a certificate warning to the user and allow the user to bypass the error. A browser client would likely never send this:
22 6 0.0042 (0.0006) C>SV3.3(2) Alert level fatal value certificate_unknownwhich basically implies that the client (C>S) does not trust the itsdtest.example.com server certificate.
