Forum Discussion
Bhargav_9588
Nimbostratus
Oct 24, 2008handshake_failure
Hello,
After upgrading LTM to 9.3.1 we started noticing "handshake_failure" errors with java clients that are trying to connect to https://www.***.com/ (virtual server in LTM with Client SSL Profile and Client Certificate set to "request").
It use to work with out any issues before upgrade with same settings. After upgrade, when java client tries to connect to same webservice with https getting following error:
main, WRITE: TLSv1 Handshake, length = 32
main, received EOFException: error
main, handling exception: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
main, SEND TLSv1 ALERT: fatal, description = handshake_failure
main, WRITE: TLSv1 Alert, length = 18
main, called closeSocket()
Exception :javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
Authorization: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:801)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1089)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1116)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1100)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:402)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:170)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:857)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:230)
at gateway.login(gateway.java:64)
at createCollection.main(createCollection.java:31)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:782)
Any help is appreciated and thanks in advance.
--Thanks
7 Replies
- hoolio
Cirrostratus
It looks like an SSL handshake error, but the client program isn't giving details on the actual SSL error. It could be a cipher issue. Can you use openssl s_client to test? That should give you more debug information. Else, you could use ssldump on the BIG-IP to capture a trace of the problem.
You can use tcpdump to capture the encrypted traffic failure:
tcpdump -i0.0 -s0 -w/var/tmp/encrypted.dmp host 1.1.1.1
-i 0.0 captures on all switch interfaces
-w writes the output to a binary file
host 1.1.1.1 filters on traffic to/from 1.1.1.1
Make sure to start the tcpdump before the client establishes a connection to the VIP. SOL411 (Click here) has more detail on using tcpdump.
You can then use ssldump to decrypt the tcpdump file.
https://support.f5.com/kb/en-us/solutions/public/7000/800/sol7823.html
ssldump -AdneN -r/var/tmp/encrypted.dmp -k/path/to/server.key >/var/tmp/decrypted.txt
Where /var/tmp/encrypted.dmp is the tcpdump file, /path/to/server.key is the path to the server's (VIP) SSL key, and /var/tmp/decrypted.txt is the cleartext output file. You can run ssldump on the BIG-IP or a windows host. For details on the command, check the man page: man ssldump.
Also, there is a forum for Advanced Configuration (Click here) where you might get more relevant eyes on your post.
Aaron - Bhargav_9588
Nimbostratus
Thanks Aaron!
I have tried s_client and I see the following (handshake failure):
openssl s_client -connect www.xyz.com:443 -CAfile ../../IssuingCA.cer
CONNECTED(00000004)
depth=1 /emailAddress=email@xyz.com/C=**/ST=**/L=***/O=Corp./OU=Corporate/CN=Corp. CA
verify error:num=2:unable to get issuer certificate
issuer= /emailAddress=cisinfrastructure@aes.com/C=**/ST=**/L=***/O=The XYZ Corp./OU=Corporate/CN=Root CA
verify return:0
1618:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:226:
It works fine with no issues from browser. We started seeing issues with java clients after upgrading LTM.
I have not made success yet to decrypt tcpdump file using ssldump.
Thanks in advance. - Bhargav_9588
Nimbostratus
Just for information....Our CA is not a public CA and we use Windows Certificate Services - hoolio
Cirrostratus
Hi bhargav,
I'd suggest using ssldump to decrypt the trace. It should give you a more specific error.
Why do you have the client SSL profile set to request a client cert? Is the application expecting the client to send a client cert? If so, how is it getting the details of the client cert if you're decrypting the traffic on LTM?
You should be able to find more info on the error strings through any search engine as they're standard openssl error messages. For details on the SSL profile options, you can check the LTM config guide for your version. Here is a link to the latest 9.4.x guide (Click here)
Aaron - Bhargav_9588
Nimbostratus
We use SSL Client authentication and certificate is being stored in an http header using the following iRule:
when CLIENTSSL_CLIENTCERT {
set ssl_stuff [list cert cert_status]
set ssl_cert_status [X509::verify_cert_error_string [SSL::verify_result]]
lset ssl_stuff 0 [SSL::cert 0]
lset ssl_stuff 1 $ssl_cert_status
session add ssl [SSL::sessionid] $ssl_stuff 3600
}
when HTTP_REQUEST {
set ssl_stuff [session lookup ssl [SSL::sessionid]]
set the_cert [lindex $ssl_stuff 0]
set ssl_cert_status [lindex $ssl_stuff 1]
if { $ssl_cert_status eq "ok"}
{
if { $the_cert != ""}
{
HTTP::header insert SSLClientCert [ join [string trim [string map { "-----BEGIN CERTIFICATE-----" "" "-----END CERTIFICATE-----" ""} [X509::whole $the_cert ] ] ] "" ]
}
}
}
Thanks,
Bhargav - Bhargav_9588
Nimbostratus
Here is the decrypted string with ssldump:
New TCP connection 1: 10.1.129.4(57430) <-> 10.1.4.22(443)
1 1 1224858753.8851 (0.2500) C>S SSLv2 compatible client hello
Version 3.1
cipher suites
TLS_RSA_WITH_RC4_128_MD5
SSL2_CK_RC4
TLS_RSA_WITH_RC4_128_SHA
Unknown value 0x2f
Unknown value 0x35
Unknown value 0x33
Unknown value 0x39
Unknown value 0x32
Unknown value 0x38
TLS_RSA_WITH_3DES_EDE_CBC_SHA
SSL2_CK_3DES
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_DES_CBC_SHA
SSL2_CK_DES
TLS_DHE_RSA_WITH_DES_CBC_SHA
TLS_DHE_DSS_WITH_DES_CBC_SHA
TLS_RSA_EXPORT_WITH_RC4_40_MD5
SSL2_CK_RC4_EXPORT40
TLS_RSA_EXPORT_WITH_DES40_CBC_SHA
TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA
TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA
1 2 1224858753.8905 (0.0054) S>CV3.1(74) Handshake
ServerHello
Version 3.1
random[32]=
f1 62 06 51 60 fe 2b b9 b3 bc 01 e0 40 48 0c c4
bb 8e 97 4f 77 c6 27 01 f2 d1 90 fc 8a 85 16 e4
session_id[32]=
60 4c f0 c7 96 0a 2e d0 85 17 36 65 d4 99 93 ca
18 3b e4 1a 6e 47 d2 93 93 d9 03 b7 77 50 49 57
cipherSuite TLS_RSA_WITH_RC4_128_MD5
compressionMethod NULL
1 3 1224858753.8910 (0.0004) S>CV3.1(1730) Handshake
Certificate
Subject
C=US
ST=***
L=asb
O=Corporation
OU=Corporate IT
CN=www.xyz.com
Issuer
emailAddress=abc@xyz.com
C=US
ST=***
L=***
O=Corp.
OU=Corporate IT
CN=Corp. Issuing CA
Serial 68 9f 1e a4 00 00 00 00 00 5c
Extensions
Extension: X509v3 Subject Key Identifier
Extension: X509v3 Authority Key Identifier
Extension: X509v3 CRL Distribution Points
Extension: Authority Information Access
Extension: X509v3 Basic Constraints
Critical
Extension: X509v3 Key Usage
Extension: X509v3 Extended Key Usage
Extension: 1.3.6.1.4.1.311.20.2
1 4 1224858753.8910 (0.0000) S>CV3.1(170) Handshake
CertificateRequest
certificate_types rsa_sign
certificate_authority
emailAddress=abc@xyz.com
C=US
ST=***
L=***
O=Corp.
OU=Corporate IT
CN=Root CA
1 5 1224858753.8910 (0.0000) S>CV3.1(4) Handshake
ServerHelloDone
1 6 1224858753.9199 (0.0288) C>SV3.1(269) Handshake
Certificate
ClientKeyExchange
EncryptedPreMasterSecret[256]=
45 be b7 09 1b 7b bb 1a 80 e7 cc cd b2 5f 28 0e
cd e0 c9 40 44 33 34 c0 a0 a7 87 82 b6 14 c7 f9
3a 74 28 ce 0f de 13 ce ad 43 10 b9 15 07 2d 1e
68 ed 11 0b 90 66 ad b4 05 a8 1c 15 49 7f d9 c5
1b 00 a2 36 69 76 95 bb 5d 90 da 75 10 dc a8 87
20 1e 6c 33 83 be f7 25 b2 e1 4b fa 5e 05 d9 a5
b2 10 af 2b f2 e3 fe 64 da 3e 39 15 92 65 ba d6
40 19 87 96 77 8d b7 65 11 84 95 a3 48 f6 8c 5b
c2 18 9a 11 8b d7 41 79 75 b3 21 3c 5b b3 51 72
8f 04 c3 98 15 e9 19 db 15 16 01 73 e1 cf ec de
2a 5a 23 9a a9 ca 10 23 4f 74 ae a6 bc b7 0b ee
0b 9c a3 d6 7e fd 86 38 04 8e 58 96 66 82 25 a7
55 b0 46 43 81 17 4c f0 c6 17 08 c1 b0 d4 3c 74
43 ab d3 ea ef cb 4d 13 ee 5b 9f 7f c4 83 4c 0e
2a c6 e2 e6 1a 20 54 29 e7 82 d5 08 75 12 44 1b
6d b7 0b 2b c3 76 b1 1e 88 53 2f 37 2a 6b f8 f3
1 7 1224858753.9216 (0.0017) C>SV3.1(1) ChangeCipherSpec
1 8 1224858753.9235 (0.0018) C>SV3.1(32) Handshake
1 1224858753.9374 (0.0139) S>C TCP FIN
1 9 1224858753.9386 (0.0012) C>SV3.1(18) Alert
1 1224858753.9387 (0.0001) C>S TCP FIN - Steve_Scott_873Historic F5 AccountCould i suggest wireshark as an easier to use alternative to SSLDump.
Also, this is invaluable: http://en.wikipedia.org/wiki/Secure_Sockets_Layer (Don't laugh, Wiki is pretty good for this sort of info and a lot easier to read than an RFC!)
Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects