SSL handshake errors

Have you checked this one out?

https://support.f5.com/kb/en-us/solutions/public/15000/200/sol15292.html?sr=49481718

Yes I have, this did not help. Sorry.

I think this issue is the following.

https://support.f5.com/kb/en-us/solutions/public/16000/100/sol16159.html

I recommend your bigip should be upgraded.

Upgraded to what? I am already running TMOS 12.0.

Naresh, it looks like you've been working in two threads: https://devcentral.f5.com/questions/ssl-handshake-errors

You indicated in the other thread that the server sends a reset AFTER the client sends an application data packet, and here you said that you don't see any handshake failures at all. That could indicate just a few things:

1. I know you're convinced that this is a client side issue, but you might actually see this pattern on the client side if the server side SSL was broken. Have you done SSL captures on the server side?

    

2. If both sides are handling SSL just fine, then the likely issue is inside the layer 7 communications. You may need to force an RSA key exchange and crack open the SSL traffic with ssldump and the -k option to see what that looks like.

    

Kevin,

Yes sorry. I got an email from Devcentral and the link in that landed me on the other thread that I didn't know existed and it had same title. Later I realized my mistake but anyway thanks for your response.

1. I have no serverside ssl enabled, ssl terminates at the F5.
2. I did ssldump with -k option with key but I can't see what is going on inside application_data, which I was hoping to see. Also tried -A but can't see any details.

Naresh

I meant SSL terminates at VIP (not fully awake yet I guess).

Okay, I guess I assumed from the other thread that you were doing SSL on the server side. So this actually makes things a bit easier. Let's try some additional tests:

1. Run an tcpdump, listening ONLY on the server side VLAN. Do you see ANY traffic going to the server when you test? If you do, do you see a reset coming from the server?

2. With the -k option you also need to:

   a. Provide the private key (the one you use in the client SSL profile)

   ssldump -k /path/to/private.key -AdNn -i [client-side VLAN] port 443 [and any additional filters]
   

   b. Force the client and BIG-IP to use an RSA key exchange. The simplest option here might be to just temporarily change the Cipher string in the client SSL profile to: !SSLv3:RSA+AES. This will allow ssldump to decrypt the traffic.

3. Run a client side capture like HTTPWatch or Fiddler and see if there's any HTTP traffic before it fails, and if so where it fails.

You actually have a few clients in this capture, so I'll strip out everything but the last one (569) and re-format for visibility:

New TCP connection 569: 70.198.140.97(7336) <-> x.x.x.x(443) 
569 1 0.1171 (0.1171) C>SV3.3(184) Handshake 
    ClientHello 
        Version 3.3 
        random[32]= 56 4a c5 04 f4 09 eb... 
        cipher suites 
            TLS_EMPTY_RENEGOTIATION_INFO_SCSV 
            TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 
            TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 
            TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA 
            TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA 
            TLS_ECDHE_ECDSA_WITH_RC4_128_SHA 
            TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA 
            TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 
            TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 
            TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 
            TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA 
            TLS_ECDHE_RSA_WITH_RC4_128_SHA 
            TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA 
            TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 
            TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 
            TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 
            TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 
            TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA 
            TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA 
            TLS_ECDH_ECDSA_WITH_RC4_128_SHA 
            TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA 
            TLS_ECDH_RSA_WITH_AES_128_CBC_SHA 
            TLS_ECDH_RSA_WITH_AES_256_CBC_SHA 
            TLS_ECDH_RSA_WITH_RC4_128_SHA 
            TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA 
            TLS_RSA_WITH_AES_256_CBC_SHA256 
            TLS_RSA_WITH_AES_128_CBC_SHA256 
            TLS_RSA_WITH_AES_128_CBC_SHA 
            TLS_RSA_WITH_RC4_128_SHA 
            TLS_RSA_WITH_RC4_128_MD5 
            TLS_RSA_WITH_AES_256_CBC_SHA 
            TLS_RSA_WITH_3DES_EDE_CBC_SHA 
            TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 
            TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 
            TLS_DHE_RSA_WITH_AES_128_CBC_SHA 
            TLS_DHE_RSA_WITH_AES_256_CBC_SHA 
            TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA 
            compression methods 
                NULL 

The client is issuing its request to start an SSL session. The ClientHello message contains the client's preferred protocol (Version 3.3 = TLSv1.2), a random number, and a list of supported cipher suites (in general order of preference)

569 2 0.1180 (0.0009) S>CV3.3(85) Handshake 
ServerHello 
        Version 3.3 
        random[32]= 6c 0f a3 fc 6e d1 8a... 
        session_id[32]= e8 27 5b f0 9c 4a...
        cipherSuite 
            TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 
            compressionMethod 
                NULL 

The server responds with a ServerHello that contains the selected protocol (Version 3.3), its own random number, and the selected cipher from the client's list (TLS_DHE_RSA_WITH_AES_256_CBC_SHA256)

569 3 0.1180 (0.0000) S>CV3.3(2488) Handshake 
    Certificate 
        Subject 
            C=US 
            ST=California L=Mountain View 
            O=abc, Inc. 
            OU=Network Operations 
            CN=*.abc.net Issuer 
            C=US O=thawte, Inc. 
            CN=thawte SHA256 SSL CA 
        Serial      49 a1 db 0d 32 5e a5 16 dd 0b 5c 71 eb ec f3 6a 
        Extensions 
            Extension: 
                X509v3 Subject Alternative Name Extension: 
                X509v3 Basic Constraints Extension: 
                X509v3 Certificate Policies Extension: 
                X509v3 Key Usage Critical Extension: 
                X509v3 Authority Key Identifier Extension: 
                X509v3 CRL Distribution Points Extension: 
                X509v3 Extended Key Usage Extension: 
                Authority Information Access 
        Subject 
            C=US O=thawte, Inc. 
            CN=thawte SHA256 SSL CA 
        Issuer 
            C=US O=thawte, Inc. 
            OU=Certification Services Division 
            OU=(c) 2008 thawte, Inc. - For authorized use only 
            CN=thawte Primary Root CA - G3 
        Serial      36 34 9e 18 c9 9c 26 69 b6 56 2e 6c e5 ad 71 32 
        Extensions 
            Extension: 
                Authority Information Access Extension: 
                X509v3 Basic Constraints Critical Extension: 
                X509v3 Certificate Policies Extension: 
                X509v3 CRL Distribution Points Extension: 
                X509v3 Key Usage Critical Extension: 
                X509v3 Subject Alternative Name Extension: 
                X509v3 Subject Key Identifier Extension: 
                X509v3 Authority Key Identifier 

The server immediately sends a Certificate message that should contain its own certificate and optionally any immediate issuing CAs.

569 4 0.1180 (0.0000) S>CV3.3(527) Handshake 
    ServerKeyExchange 
        params 
            DH_p[128]= b9 77 c2 e3 f9 1d 78 15...
            DH_g[1]= 02 
            DH_Ys[128]= 58 5d 21 35 30 dc 7f 35... 

The server follows the Certificate message with a ServerKeyExchange message, which generally only occurs when a Diffie-Hellman cipher is selected. Here the server is sending its DH parameters.

569 5 0.1180 (0.0000) S>CV3.3(4) Handshake 
    ServerHelloDone

The server then indicates it's done with a ServerHelloDone message.

569 0.6487 (0.5306) C>S TCP FIN 
569 0.6487 (0.0000) S>C TCP FIN 

What should be next is the client's ClientKeyExchange message, which for Diffie-Hellman is the client's DH parameters (or for RSA the encrypted pre-master secret). As the client is sending a FIN at this point, the most likely cause is that the client doesn't like something about the server's response. We can pretty much rule out cipher selection and protocol, as those were selected from the client's list. You may have scrubbed the data a little too much here, but one interesting thing I noticed is that the server's certificate chain starts with a CA (CN=thawte SHA256 SSL CA). Is that correct? At this point in the SSL handshake the client is going to attempt to validate the server's certificate. If the client cannot establish a trust chain with the server's certificate, or the server's certificate is otherwise invalid, the client agent may either prompt the user with an "untrusted certificate" warning, or in some circumstances just fail completely. My guess here, assuming the capture is mostly intact, is that the client is failing the SSL handshake because of the certificates presented. If you test access to the BIG-IP VIP with cURL,

curl -k https://x.x.x.x

(the -k tells cURL to ignore certificate trust issues) do you get any further?

Kevin,

Thanks for looking at it, sorry for the bad formatting. Yes "thawte SHA256 SSL CA" is fine. The clientssl profile that I use here is used in 100s of more VIPs and have no problem with certificate chain. I have tried to use the one with a bad chain and I get a different error - something to the effect "unknown_ca" in packet capture, which I am not seeing here. Since Handshake failed, no application_data started. I am still at a loss to understand why this (and others I sent) failed. One of them only has one line where client makes a new connection to server and fails right after, making no sense of what happened.

Naresh