Forum Discussion

Babak_AA_246963's avatar
Babak_AA_246963
Icon for Altostratus rankAltostratus
Jul 05, 2016

BIG-IP Proxy SSL 12.1 Handshake Failure

I set up SSL Proxy in order to do client certificate authentication on my IIS web server on LTM 12.1 firmware. The setup is working fine on Firefox version 43, IE 10 and OpenSSL but it fails on Chrom...
  • Babak_AA_246963's avatar
    Sep 23, 2016

    I figured out the issue I was facing. It had to do with TLS Extended Master Secret and the BIG-IP was failing to decrypt the handshake. The extended master secret changes the way pre-master secret is generated for TLS sessions and I suspect BIG-IP fails to detect its presence and calculates the pre-master secret as if extended master secret is not in place, anyways I've written my experience in a couple of blog posts in case someones willing to get into the details: TLS Extended Master Secret Breaking SSL Proxies.

     

    Solution

     

    As for the solution, until BIG-IP adds this feature (decrypting sessions where extended master secret is used) I disabled it on my web server (The threat it was mitigating was minimal in my case when the choice is between having a WAF or having extended master secret enabled, it basically prevents rogue CAs to create bogus certificates and use them to MITM live TLS sessions, more details in the blog post).

     

    Disabling TLS Extended Master Secret in Windows Server/IIS:

     

    For IIS you'd have to go into registry and under SCHANNEL configurations add the following key:

     

    Under HKLM\System\CurrentControlSet\Control\SecurityProviders\Schannel:

     

    Add DisableServerExtendedMasterSecret as REG_DWORD with the value of 1 (anything other than 0 works)

     

    The setting applies immediately and you don't need to restart the server.