Forum Discussion
Babak_AA_246963
Altostratus
AltostratusBIG-IP Proxy SSL 12.1 Handshake Failure
I set up SSL Proxy in order to do client certificate authentication on my IIS web server on LTM 12.1 firmware. The setup is working fine on Firefox version 43, IE 10 and OpenSSL but it fails on Chrome 51, Firefox 47 and IE 11.
I've captured the packets. Clients use TLS1 or TLS1.2 using the same ciphersuite of TLS_RSA_WITH_AES_256_CBC_SHA (0x0035), the same process takes place for the passing and failing cases.
- Client Hello
- Server Hello, Certificate, Server Hello Done
- Client Key Exchange, Change Cipher Spec, Encrypted Handshake Message 4. 4.1 Either Server sends Change Cipher Spec and then Application Data gets transfered Or 4.2 The server sends Alert level: Fatal, Descrition: Handshake Failure
So I suspect the BIG-IP fails to decrypt the handshake sent by the client in some cases but I can't figure out why because there's nothing different between failing and passing tests.
ssldump using Firefox 47 (Fails):
New TCP connection 1: 192.168.100.125(55041) <-> 192.168.100.231(443)
1 1 0.0027 (0.0027) C>S Handshake
ClientHello
Version 3.3
cipher suites
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Unknown value 0xcca9
Unknown value 0xcca8
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA
compression methods
NULL
1 2 0.0033 (0.0005) S>C Handshake
ServerHello
Version 3.3
session_id[32]=
d9 0a 00 00 3e 11 22 ac e2 c2 00 f5 9a 41 35 53
43 6a 9e a5 e0 26 32 e4 f8 38 2e ca 72 3c fb 93
cipherSuite TLS_RSA_WITH_AES_256_CBC_SHA
compressionMethod NULL
Certificate
ServerHelloDone
1 3 0.0185 (0.0151) C>S Handshake
ClientKeyExchange
1 4 0.0185 (0.0000) C>S ChangeCipherSpec
1 5 0.0185 (0.0000) C>S Handshake
1 6 0.0196 (0.0011) S>C Alert
level fatal
value handshake_failure
1 0.0197 (0.0000) S>C TCP FIN
1 0.0205 (0.0008) C>S TCP FIN
New TCP connection 2: 192.168.100.125(55042) <-> 192.168.100.231(443)
2 1 0.0005 (0.0005) C>S Handshake
ClientHello
Version 3.2
cipher suites
Unknown value 0x5600
Unknown value 0xcca9
Unknown value 0xcca8
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA
compression methods
NULL
2 2 0.0010 (0.0005) S>C Handshake
ServerHello
Version 3.2
session_id[32]=
85 48 00 00 8f 2a ae 80 b8 d7 e9 e2 47 c0 15 4e
e8 af 69 6f 2d b9 b8 d6 ed d5 29 3c a3 a3 44 b3
cipherSuite TLS_RSA_WITH_AES_256_CBC_SHA
compressionMethod NULL
Certificate
ServerHelloDone
2 3 0.0145 (0.0134) C>S Handshake
ClientKeyExchange
2 4 0.0145 (0.0000) C>S ChangeCipherSpec
2 5 0.0145 (0.0000) C>S Handshake
2 6 0.0158 (0.0013) S>C Alert
level fatal
value handshake_failure
2 0.0158 (0.0000) S>C TCP FIN
2 0.0162 (0.0003) C>S TCP FIN
New TCP connection 3: 192.168.100.125(55043) <-> 192.168.100.231(443)
3 1 0.0005 (0.0005) C>S Handshake
ClientHello
Version 3.1
cipher suites
Unknown value 0x5600
Unknown value 0xcca9
Unknown value 0xcca8
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA
compression methods
NULL
3 2 0.0010 (0.0004) S>C Handshake
ServerHello
Version 3.1
session_id[32]=
aa 41 00 00 04 82 07 3f ed 35 96 49 e2 c5 ba 79
f8 39 5a f2 d2 41 19 33 8e 5b 05 5e 2f d1 ca 24
cipherSuite TLS_RSA_WITH_AES_256_CBC_SHA
compressionMethod NULL
Certificate
ServerHelloDone
3 3 0.0141 (0.0131) C>S Handshake
ClientKeyExchange
3 4 0.0141 (0.0000) C>S ChangeCipherSpec
3 5 0.0141 (0.0000) C>S Handshake
3 6 0.0155 (0.0013) S>C Alert
level fatal
value handshake_failure
3 0.0155 (0.0000) S>C TCP FIN
3 0.0165 (0.0009) C>S TCP FIN
ssldump using Firefox 43 (Passes):
New TCP connection 1: 192.168.100.125(55099) <-> 192.168.100.231(443)
1 1 0.0007 (0.0007) C>S Handshake
ClientHello
Version 3.3
cipher suites
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA
compression methods
NULL
1 2 0.0012 (0.0004) S>C Handshake
ServerHello
Version 3.3
session_id[32]=
0f 16 00 00 ec 24 3b 75 10 f0 53 c4 45 d3 df ef
97 91 f0 9a b8 fe c2 98 5d 15 fd 11 ed 2f 55 58
cipherSuite TLS_RSA_WITH_AES_256_CBC_SHA
compressionMethod NULL
Certificate
ServerHelloDone
1 3 0.0031 (0.0018) C>S Handshake
ClientKeyExchange
1 4 0.0031 (0.0000) C>S ChangeCipherSpec
1 5 0.0031 (0.0000) C>S Handshake
1 6 0.0053 (0.0022) S>C ChangeCipherSpec
1 7 0.0056 (0.0002) S>C Handshake
1 8 0.2922 (0.2865) C>S application_data
1 9 0.3330 (0.0408) S>C Handshake
1 10 0.3337 (0.0006) C>S Handshake
1 11 0.3368 (0.0031) S>C Handshake
1 12 0.3473 (0.0104) C>S Handshake
1 13 0.3473 (0.0000) C>S ChangeCipherSpec
1 14 0.3473 (0.0000) C>S Handshake
1 15 0.3500 (0.0026) S>C ChangeCipherSpec
1 16 0.3501 (0.0001) S>C Handshake
1 17 0.3512 (0.0011) S>C application_data
1 18 0.3779 (0.0266) C>S application_data
New TCP connection 2: 192.168.100.125(55102) <-> 192.168.100.231(443)
2 1 0.0008 (0.0008) C>S Handshake
ClientHello
Version 3.3
resume [32]=
b3 15 00 00 94 41 0f d7 0f ce 39 45 82 5e 53 85
b4 4f de 6d 1c f7 23 16 c6 8b bb d6 96 d9 53 c5
cipher suites
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_256_CBC_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA
compression methods
NULL
2 2 0.0011 (0.0003) S>C Handshake
ServerHello
Version 3.3
session_id[32]=
b3 15 00 00 94 41 0f d7 0f ce 39 45 82 5e 53 85
b4 4f de 6d 1c f7 23 16 c6 8b bb d6 96 d9 53 c5
cipherSuite TLS_RSA_WITH_AES_256_CBC_SHA
compressionMethod NULL
2 3 0.0012 (0.0000) S>C ChangeCipherSpec
2 4 0.0018 (0.0006) S>C Handshake
1 19 0.3804 (0.0025) S>C application_data
2 5 0.0025 (0.0006) C>S ChangeCipherSpec
2 6 0.0025 (0.0000) C>S Handshake
2 7 0.0033 (0.0007) C>S application_data
2 8 0.0057 (0.0023) S>C Handshake
2 9 0.0062 (0.0005) C>S Handshake
2 10 0.0072 (0.0010) S>C Handshake
2 11 0.0210 (0.0137) C>S Handshake
2 12 0.0210 (0.0000) C>S ChangeCipherSpec
2 13 0.0210 (0.0000) C>S Handshake
2 14 0.0246 (0.0035) S>C ChangeCipherSpec
2 15 0.0246 (0.0000) S>C Handshake
2 16 0.0250 (0.0003) S>C application_data
I figured out the issue I was facing. It had to do with TLS Extended Master Secret and the BIG-IP was failing to decrypt the handshake. The extended master secret changes the way pre-master secret is generated for TLS sessions and I suspect BIG-IP fails to detect its presence and calculates the pre-master secret as if extended master secret is not in place, anyways I've written my experience in a couple of blog posts in case someones willing to get into the details: TLS Extended Master Secret Breaking SSL Proxies.
Solution
As for the solution, until BIG-IP adds this feature (decrypting sessions where extended master secret is used) I disabled it on my web server (The threat it was mitigating was minimal in my case when the choice is between having a WAF or having extended master secret enabled, it basically prevents rogue CAs to create bogus certificates and use them to MITM live TLS sessions, more details in the blog post).
Disabling TLS Extended Master Secret in Windows Server/IIS:
For IIS you'd have to go into registry and under SCHANNEL configurations add the following key:
Under HKLM\System\CurrentControlSet\Control\SecurityProviders\Schannel:
Add DisableServerExtendedMasterSecret as REG_DWORD with the value of 1 (anything other than 0 works)
The setting applies immediately and you don't need to restart the server.
- Saravanan_M_K
Employee
Try setting the SSL logging level to "Debug" on the LTM and see what debug info it logs in the /var/log/ltm for the failure case. It may give more clue. After the testing, make sure to change the logging level back to the original value.
What is the version of IIS server/OS are you using?
Babak_AA_246963ltm logs say:
srv warning tmm[12511]: 01260009:4: Connection error: hud_ssl_handler:1224: codec alert(20)
srv info tmm[12511]: 01260013:6: SSL Handshake failed for TCP 192.168.100.215:443 -> 192.168.100.230:51271
srv info tmm[12511]: 01260013:6: SSL Handshake failed for TCP 192.168.100.230:51271 -> 192.168.100.215:443
I'm using windows server 2012 and IIS 8.
Let me point out again that old web browsers and openssl cli connects through the LTM/ASM correctly but modern browsers fail to do so and get a handshake error. In order to isolate the issue I made sure all my tests are under same protocol/cipher suite (TLS_RSA_WITH_AES_256_CBC_SHA).
I looked at packet captures from BIG-IP's point of view and web servers point of view. The cases where there's a handshake failure alert at BIG-IP, the webserver sees a FIN followed by a RST after client sends change cipher spec and it's the BIG-IP resetting the connection due to handshake failure (I suspect it fails to decrypt the handshake for some reason). But in the case where openssl cli or older browsers are used the BIG-IP happily decrypts the handshake and the connection goes through it.
Hannes_RappNimbostratus
Check if your client-auth certificate, client-ssl certificate and all certificates in CA intermediary certificate bundle are signed with modern SHA algorithms, or if there are any certs signed by the deprecated SHA1. Note that the very root certificate of your CA may still remain SHA1-signed, but any intermediary certificates above it must be SHA256-signed (or stronger). Any of this does not apply to SSL handshakes and SHA in a cipher suite is still OK.
Newer browsers are becoming more and more restrictive with SHA1 certificates and this could be one potential cause to your issue. This OpenSSL command can help you get started:
openssl x509 -text -in '/Path/To/Cert.crt' | grep "Signature Algorithm" Signature Algorithm: sha256WithRSAEncryption- Babak_AA_246963Apart from the root certificate which was SHA1, the rest of the certificates where using SHA256. but the fact that the BIG-IP sends a handshake failure and resets the connection has to do something with an issue on BIG-IP side rather than clients browser being unhappy, correct me if this assumption is incorrent.
- Hannes_RappYou may be right, but we cannot really tell, as in your own words, there are browsers with which the whole solution works. To further isolate the issue, and eliminate variables, perhaps disabling Client-Auth for a test would be worth it. If the SSL handshake then completes successfully, it can at least be confirmed that it's a SSL-MA problem, and maybe tweaking some settings will do the trick. After all, it's also possible you're looking to solve a problem caused by a bug, so I recommend to open a support case with F5.
Hannes_Rapp_162Nacreous
Check if your client-auth certificate, client-ssl certificate and all certificates in CA intermediary certificate bundle are signed with modern SHA algorithms, or if there are any certs signed by the deprecated SHA1. Note that the very root certificate of your CA may still remain SHA1-signed, but any intermediary certificates above it must be SHA256-signed (or stronger). Any of this does not apply to SSL handshakes and SHA in a cipher suite is still OK.
Newer browsers are becoming more and more restrictive with SHA1 certificates and this could be one potential cause to your issue. This OpenSSL command can help you get started:
openssl x509 -text -in '/Path/To/Cert.crt' | grep "Signature Algorithm" Signature Algorithm: sha256WithRSAEncryption- Babak_AA_246963Apart from the root certificate which was SHA1, the rest of the certificates where using SHA256. but the fact that the BIG-IP sends a handshake failure and resets the connection has to do something with an issue on BIG-IP side rather than clients browser being unhappy, correct me if this assumption is incorrent.
- Hannes_Rapp_162You may be right, but we cannot really tell, as in your own words, there are browsers with which the whole solution works. To further isolate the issue, and eliminate variables, perhaps disabling Client-Auth for a test would be worth it. If the SSL handshake then completes successfully, it can at least be confirmed that it's a SSL-MA problem, and maybe tweaking some settings will do the trick. After all, it's also possible you're looking to solve a problem caused by a bug, so I recommend to open a support case with F5.
JGCumulonimbus
You can remove ECDHE-RSA-AES256-CBC-SHA from F5's cipher suite in your client-side SSL profile and see if that makes a difference.
Mickael_SrutowsHello,
I have the same problem, you have a solution or not ? I have this problem since few days.
Thanks.
- Babak_AA_246963
I figured out the issue I was facing. It had to do with TLS Extended Master Secret and the BIG-IP was failing to decrypt the handshake. The extended master secret changes the way pre-master secret is generated for TLS sessions and I suspect BIG-IP fails to detect its presence and calculates the pre-master secret as if extended master secret is not in place, anyways I've written my experience in a couple of blog posts in case someones willing to get into the details: TLS Extended Master Secret Breaking SSL Proxies.
Solution
As for the solution, until BIG-IP adds this feature (decrypting sessions where extended master secret is used) I disabled it on my web server (The threat it was mitigating was minimal in my case when the choice is between having a WAF or having extended master secret enabled, it basically prevents rogue CAs to create bogus certificates and use them to MITM live TLS sessions, more details in the blog post).
Disabling TLS Extended Master Secret in Windows Server/IIS:
For IIS you'd have to go into registry and under SCHANNEL configurations add the following key:
Under HKLM\System\CurrentControlSet\Control\SecurityProviders\Schannel:
Add DisableServerExtendedMasterSecret as REG_DWORD with the value of 1 (anything other than 0 works)
The setting applies immediately and you don't need to restart the server.- mfkk531_168091
I'm experiencing similar issue after upgrade to v12.1.2 Final.
Any recommendation on how to fix this if the pool members are Apache servers ?Thanks!
- Troy_Murray_196
Cirrus
AskF5 Support Article "K34019109: SSL handshakes using TLS extended master secret extensions fail when Proxy SSL is enabled" with this information can be found at https://support.f5.com/csp/article/K34019109
MLNo more errors in logs with 12.1.3.
