ssl handshake failure with backend server

Hi, I am trying to SSL termination to backend server using client profile and server profile.

This is the server profile:

  • admin@(f5lab01-asm)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm profile server-ssl back-end-servers

  • ltm profile server-ssl back-end-servers {

    alert-timeout 10

    app-service none

    authenticate once

    authenticate-depth 9

    authenticate-name none

    ca-file none

    cache-size 262144

    cache-timeout 3600

    cert none

    chain none

    ciphers SSLv3:SSLv3+RC4-SHA

    crl-file none

    defaults-from serverssl

    expire-cert-response-control drop

    generic-alert enabled

    handshake-timeout 10

    key none

    mod-ssl-methods disabled

    mode enabled

    options none

    peer-cert-mode ignore

    proxy-ssl disabled

    proxy-ssl-passthrough disabled

    renegotiate-period indefinite

    renegotiate-size indefinite

    renegotiation disabled

    retain-certificate true

    secure-renegotiation require

    server-name none

    session-mirroring disabled

    session-ticket disabled

    sni-default false

    sni-require false

    ssl-forward-proxy disabled

    ssl-forward-proxy-bypass disabled

    ssl-sign-hash any

    strict-resume disabled

    unclean-shutdown enabled

    untrusted-cert-response-control drop

    }

the test with openssl

[admin@f5lab01-asm:Active:In Sync] ~ openssl s_client -host 192.168.0.1 -port 443 CONNECTED(00000003) 46963579710592:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:184:

no peer certificate available

No client certificate CA names sent

SSL handshake has read 0 bytes and written 305 bytes

New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE

The ssldump:

  • [admin@f5lab01-asm:Active:In Sync] ~ ssldump -Aed -k /config/filestore/files_d/Common_d/certificate_key_d/:Common:home.com.key_63567_1 -n -i internal host 192.168.0.1 New TCP connection 1: 192.168.0.63(36056) ↔ 192.168.0.1(443) 1 1 1447104036.1652 (0.0008) C>SV3.0(87) Handshake

    ClientHello

    Version 3.0

    random[32]=

    09 30 c3 e9 06 5d 07 f9 29 59 e2 3c 3d 84 bc 7c

    85 19 71 27 86 ec 58 c2 8e 30 77 47 f4 b9 40 ce

    cipher suites

    SSL_DHE_RSA_WITH_AES_256_CBC_SHA

    SSL_DHE_DSS_WITH_AES_256_CBC_SHA

    SSL_DH_anon_WITH_AES_256_CBC_SHA

    SSL_RSA_WITH_AES_256_CBC_SHA

    SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA

    SSL_DH_anon_WITH_3DES_EDE_CBC_SHA

    SSL_RSA_WITH_3DES_EDE_CBC_SHA

    SSL_DHE_RSA_WITH_AES_128_CBC_SHA

    SSL_DHE_DSS_WITH_AES_128_CBC_SHA

    SSL_DH_anon_WITH_AES_128_CBC_SHA

    SSL_RSA_WITH_AES_128_CBC_SHA

    SSL_DH_anon_WITH_RC4_128_MD5

    SSL_RSA_WITH_RC4_128_SHA

    SSL_RSA_WITH_RC4_128_MD5

    SSL_DHE_RSA_WITH_DES_CBC_SHA

    SSL_DH_anon_WITH_DES_CBC_SHA

    SSL_RSA_WITH_DES_CBC_SHA

    SSL_RSA_EXPORT1024_WITH_DES_CBC_SHA

    SSL_RSA_EXPORT1024_WITH_RC4_56_SHA

    SSL_RSA_EXPORT_WITH_DES40_CBC_SHA

    SSL_RSA_EXPORT_WITH_RC4_40_MD5

    Unknown value 0xff

    compression methods

    NULL

    1 1447104036.1659 (0.0007) S>C TCP FIN

    1 1447104036.1660 (0.0000) C>S TCP RST

Any ideas that we need to change?

I am using 11.6 HF6.

Regards

That’s a bit unreadable I’d recommend using some wiki tags so the info formats better.

That being said, what is 192.168.111.58? The backend or the virtual IP?

If it’s the backend, it would seem to me the backend isn’t talking SSL or requires a client certificate. You can add -prexit to your openssl command to see if it wants a client cert.

If it’s the virtual IP, then we’d have to know more about your client-ssl profile. You don’t get to the server-ssl profile until after client-ssl was successful.

Looks like the server doesn’t support any of the ciphers you are offering in your server SSL profile. What kind of server is it on the backend? Do you know what ciphers it supports? Have you tried using something more broad like DEFAULT or NATIVE for your cipher string to find out what it can negotiate?

Looks like the server doesn’t support any of the ciphers you are offering in your server SSL profile. What kind of server is it on the backend? Do you know what ciphers it supports? Have you tried using something more broad like DEFAULT or NATIVE for your cipher string to find out what it can negotiate?

My client ssl profile:

  • admin@(f5lab01-asm)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm profile client-ssl home.com ltm profile client-ssl home.com {

    alert-timeout 10

    allow-non-ssl disabled

    app-service none

    cache-size 262144

    cache-timeout 3600

    cert home.com.crt

    cert-key-chain {

    home {

    cert home.com.crt

    key home.com.key

    }

    }

    chain none

    ciphers DEFAULT

    defaults-from clientssl

    generic-alert enabled

    handshake-timeout 10

    inherit-certkeychain false

    key home.com.key

    max-renegotiations-per-minute 5

    mod-ssl-methods disabled

    mode enabled

    options { dont-insert-empty-fragments }

    passphrase none

    peer-no-renegotiate-timeout 10

    proxy-ssl disabled

    proxy-ssl-passthrough disabled

    renegotiate-max-record-delay indefinite

    renegotiate-period indefinite

    renegotiate-size indefinite

    renegotiation enabled

    secure-renegotiation require

    server-name none

    session-mirroring disabled

    session-ticket disabled

    sni-default false

    sni-require false

    ssl-sign-hash any

    strict-resume disabled

    unclean-shutdown enabled

    }

What about your server SSL profile. I think that’s where your issue lies.

The sever is IBM Webshere 6.1 when i am using Native this is the output: * New TCP connection 14: 192.168.0.63(42494) ↔ 192.168.0.1(443)

14 1 1447162756.9137 (0.0008) C>SV3.3(215) Handshake

ClientHello

Version 3.3

random[32]=

5e 59 b6 e6 73 f5 6f de ba 99 6f 06 1b fb 9e e9

21 d6 03 9c ad 8d e1 6d 75 15 0b ba 6e be 46 a7

cipher suites

Unknown value 0xc030

Unknown value 0xc02c

Unknown value 0xc028

Unknown value 0xc024

Unknown value 0xc014

Unknown value 0xc00a

Unknown value 0xa3

Unknown value 0x9f

TLS_DHE_DSS_WITH_AES_256_CBC_SHA256

TLS_DHE_RSA_WITH_AES_256_CBC_SHA

TLS_DHE_DSS_WITH_AES_256_CBC_SHA

Unknown value 0xa7

TLS_DH_anon_WITH_AES_256_CBC_SHA

Unknown value 0xc032

Unknown value 0xc02e

Unknown value 0xc02a

Unknown value 0xc026

Unknown value 0xc00f

Unknown value 0xc005

Unknown value 0x9d

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS_RSA_WITH_AES_256_CBC_SHA

Unknown value 0xc012

Unknown value 0xc008

TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA

TLS_DH_anon_WITH_3DES_EDE_CBC_SHA

Unknown value 0xc00d

Unknown value 0xc003

TLS_RSA_WITH_3DES_EDE_CBC_SHA

Unknown value 0xc02f

Unknown value 0xc02b

Unknown value 0xc027

Unknown value 0xc023

Unknown value 0xc013

Unknown value 0xc009

Unknown value 0xa2

Unknown value 0x9e

TLS_DHE_DSS_WITH_AES_128_CBC_SHA256

TLS_DHE_RSA_WITH_AES_128_CBC_SHA

TLS_DHE_DSS_WITH_AES_128_CBC_SHA

Unknown value 0xa6

TLS_DH_anon_WITH_AES_128_CBC_SHA

Unknown value 0xc031

Unknown value 0xc02d

Unknown value 0xc029

Unknown value 0xc025

Unknown value 0xc00e

Unknown value 0xc004

Unknown value 0x9c

TLS_RSA_WITH_AES_128_CBC_SHA256

TLS_RSA_WITH_AES_128_CBC_SHA

TLS_DH_anon_WITH_RC4_128_MD5

TLS_RSA_WITH_RC4_128_SHA

TLS_RSA_WITH_RC4_128_MD5

TLS_DHE_RSA_WITH_DES_CBC_SHA

TLS_DH_anon_WITH_DES_CBC_SHA

TLS_RSA_WITH_DES_CBC_SHA

TLS_RSA_EXPORT1024_WITH_DES_CBC_SHA

TLS_RSA_EXPORT1024_WITH_RC4_56_SHA

TLS_RSA_EXPORT_WITH_DES40_CBC_SHA

TLS_RSA_EXPORT_WITH_RC4_40_MD5

Unknown value 0xff

compression methods NULL

14 1447162756.9144 (0.0007) S>C TCP FIN

14 1447162756.9146 (0.0001) C>S TCP RST * Whene i try to Default: * New TCP connection 6: 192.168.0.63(13306) ↔ 192.168.0.1(443)

6 1 1447161846.9679 (0.0008) C>SV3.3(131) Handshake

ClientHello

Version 3.3

random[32]=

bb d4 c6 54 aa b6 c4 be be 54 4e a8 12 39 63 7d

12 9a c2 d0 fa 70 54 b6 cf 96 d6 cf b1 8f e8 22

cipher suites

Unknown value 0x9f

Unknown value 0x9e

TLS_DHE_RSA_WITH_AES_256_CBC_SHA

TLS_DHE_RSA_WITH_AES_128_CBC_SHA

TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA

Unknown value 0x9d

Unknown value 0x9c

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS_RSA_WITH_AES_256_CBC_SHA

TLS_RSA_WITH_AES_128_CBC_SHA256

TLS_RSA_WITH_AES_128_CBC_SHA

TLS_RSA_WITH_3DES_EDE_CBC_SHA

Unknown value 0xc030

Unknown value 0xc02f

Unknown value 0xc028

Unknown value 0xc014

Unknown value 0xc027

Unknown value 0xc013

Unknown value 0xc012

Unknown value 0xff

compression methods

NULL

6 1447161846.9688 (0.0008) S>C TCP FIN

6 1447161846.9689 (0.0000) C>S TCP RST Regards

The sever is IBM Webshere 6.1 when i am using Native this is the output: * New TCP connection 14: 192.168.0.63(42494) ↔ 192.168.0.1(443)

14 1 1447162756.9137 (0.0008) C>SV3.3(215) Handshake

ClientHello

Version 3.3

random[32]=

5e 59 b6 e6 73 f5 6f de ba 99 6f 06 1b fb 9e e9

21 d6 03 9c ad 8d e1 6d 75 15 0b ba 6e be 46 a7

cipher suites

Unknown value 0xc030

Unknown value 0xc02c

Unknown value 0xc028

Unknown value 0xc024

Unknown value 0xc014

Unknown value 0xc00a

Unknown value 0xa3

Unknown value 0x9f

TLS_DHE_DSS_WITH_AES_256_CBC_SHA256

TLS_DHE_RSA_WITH_AES_256_CBC_SHA

TLS_DHE_DSS_WITH_AES_256_CBC_SHA

Unknown value 0xa7

TLS_DH_anon_WITH_AES_256_CBC_SHA

Unknown value 0xc032

Unknown value 0xc02e

Unknown value 0xc02a

Unknown value 0xc026

Unknown value 0xc00f

Unknown value 0xc005

Unknown value 0x9d

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS_RSA_WITH_AES_256_CBC_SHA

Unknown value 0xc012

Unknown value 0xc008

TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA

TLS_DH_anon_WITH_3DES_EDE_CBC_SHA

Unknown value 0xc00d

Unknown value 0xc003

TLS_RSA_WITH_3DES_EDE_CBC_SHA

Unknown value 0xc02f

Unknown value 0xc02b

Unknown value 0xc027

Unknown value 0xc023

Unknown value 0xc013

Unknown value 0xc009

Unknown value 0xa2

Unknown value 0x9e

TLS_DHE_DSS_WITH_AES_128_CBC_SHA256

TLS_DHE_RSA_WITH_AES_128_CBC_SHA

TLS_DHE_DSS_WITH_AES_128_CBC_SHA

Unknown value 0xa6

TLS_DH_anon_WITH_AES_128_CBC_SHA

Unknown value 0xc031

Unknown value 0xc02d

Unknown value 0xc029

Unknown value 0xc025

Unknown value 0xc00e

Unknown value 0xc004

Unknown value 0x9c

TLS_RSA_WITH_AES_128_CBC_SHA256

TLS_RSA_WITH_AES_128_CBC_SHA

TLS_DH_anon_WITH_RC4_128_MD5

TLS_RSA_WITH_RC4_128_SHA

TLS_RSA_WITH_RC4_128_MD5

TLS_DHE_RSA_WITH_DES_CBC_SHA

TLS_DH_anon_WITH_DES_CBC_SHA

TLS_RSA_WITH_DES_CBC_SHA

TLS_RSA_EXPORT1024_WITH_DES_CBC_SHA

TLS_RSA_EXPORT1024_WITH_RC4_56_SHA

TLS_RSA_EXPORT_WITH_DES40_CBC_SHA

TLS_RSA_EXPORT_WITH_RC4_40_MD5

Unknown value 0xff

compression methods NULL

14 1447162756.9144 (0.0007) S>C TCP FIN

14 1447162756.9146 (0.0001) C>S TCP RST * Whene i try to Default: * New TCP connection 6: 192.168.0.63(13306) ↔ 192.168.0.1(443)

6 1 1447161846.9679 (0.0008) C>SV3.3(131) Handshake

ClientHello

Version 3.3

random[32]=

bb d4 c6 54 aa b6 c4 be be 54 4e a8 12 39 63 7d

12 9a c2 d0 fa 70 54 b6 cf 96 d6 cf b1 8f e8 22

cipher suites

Unknown value 0x9f

Unknown value 0x9e

TLS_DHE_RSA_WITH_AES_256_CBC_SHA

TLS_DHE_RSA_WITH_AES_128_CBC_SHA

TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA

Unknown value 0x9d

Unknown value 0x9c

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS_RSA_WITH_AES_256_CBC_SHA

TLS_RSA_WITH_AES_128_CBC_SHA256

TLS_RSA_WITH_AES_128_CBC_SHA

TLS_RSA_WITH_3DES_EDE_CBC_SHA

Unknown value 0xc030

Unknown value 0xc02f

Unknown value 0xc028

Unknown value 0xc014

Unknown value 0xc027

Unknown value 0xc013

Unknown value 0xc012

Unknown value 0xff

compression methods

NULL

6 1447161846.9688 (0.0008) S>C TCP FIN

6 1447161846.9689 (0.0000) C>S TCP RST Regards

This is my last change on ssl server profile

admin@(f5lab01-asm)(cfg-sync In Sync)(Active)(/Common)(tmos) list ltm profile server-ssl back-end-servers ltm profile server-ssl back-end-servers { alert-timeout 10

app-service none

authenticate once

authenticate-depth 9

authenticate-name none

ca-file none

cache-size 262144

cache-timeout 3600

cert none

chain none

***ciphers SSLv3:SSLv3+RC4-SHA:SSLv2:RSA+RC4:RSA+AES:RSA+DES:RSA+3DES***

crl-file none

defaults-from serverssl

expire-cert-response-control drop

generic-alert enabled

handshake-timeout 10

key none

mod-ssl-methods disabled

mode enabled

options none

peer-cert-mode ignore

proxy-ssl disabled

proxy-ssl-passthrough disabled

renegotiate-period indefinite

renegotiate-size indefinite

renegotiation disabled

retain-certificate true

**secure-renegotiation request**

server-name none

session-mirroring disabled

session-ticket disabled

sni-default false

sni-require false

ssl-forward-proxy disabled

ssl-forward-proxy-bypass disabled

ssl-sign-hash any

strict-resume disabled

unclean-shutdown enabled

untrusted-cert-response-control drop

}

Out of interest, does a default HTTPS monitor work on the backend pool member? Can you connect to the backend pool member directly without going via the F5 i.e. the SSL Handshake works then?

Out of interest, does a default HTTPS monitor work on the backend pool member? Can you connect to the backend pool member directly without going via the F5 i.e. the SSL Handshake works then?

Your server is FINing the SSL hanshake. It is either expecting a client cert or doesn’t understand TLSv1.2 client HELLO. Can you get the SSL configuration from your WebSphere admins?

Your server is FINing the SSL hanshake. It is either expecting a client cert or doesn’t understand TLSv1.2 client HELLO. Can you get the SSL configuration from your WebSphere admins?

I fixed the issue using only cipher TLSv1 and it worked. I used this command to validate which ciphers server accept. openssl s_client -host 192.168.0.1 -port 443 -tls1

I fixed the issue using only cipher TLSv1 and it worked. I used this command to validate which ciphers server accept. openssl s_client -host 192.168.0.1 -port 443 -tls1

That’s where I was going. I’m glad its working now.

That’s where I was going. I’m glad its working now.

For what it’s worth, the server is sending a FIN right after the client’s ClientHello. That would never indicate that the server requires a client certificate as the CertificateRequest message is AFTER the server’s ServerHello and Certificate messages. At most the reasons for the server not sending a ServerHello would be (in general order of precedence):

  1. No support for the client’s preferred protocol version or list of supported ciphers
  2. A missing server name indicator (SNI) extension, if the server requires it
  3. Some unknown or incorrect TLS extension