cancel
Showing results for 
Search instead for 
Did you mean: 
Login & Join the DevCentral Connects Group to watch the Recorded LiveStream (May 12) on Basic iControl Security - show notes included.

pool member sending Alert (Level: Fatal, Description: Unknown Certificate [46]) for a new SSL Cert of a VS

masajjad
Cirrus
Cirrus

Current wildcard cert (2048 bit key size) about to expired used for SSL offloading on HTTPS VS. New cert (4096 bit key size) was applied to the that VS and application breaks.

 

VS server also has SSL Server profile (factory default). The tcpdump shows F5 Self IP sends client hello. Pool member (Tomcat) sends certificate installed on backend to F5 Self IP. Next there is client key exchange from F5 Self IP. Followed by encrypted handshake message and application data exchange between them.

 

Next pool member sends client hello to VS. VS sends server hello and the new cert to pool member. But, pool member sends  Alert (Level: Fatal, Description: Unknown Certificate [46]) to VS. And communication ends with TCP FIN.

 

Can't wrap my head around. Any suggestion?

12 REPLIES 12

Lidev
MVP
MVP

Hi masajjad,

 

if you use SSL 2 ways between the Virtual Server and the Tomcat server. It's necessary that the Tomcat server can check if the certificate is signed by an authorized CA.

Have you checked if the Tomcat has the CA who signed the certificate in his trust store?

 

Regards

masajjad
Cirrus
Cirrus

Appreciate your reply Lidev. Earlier same client side cert was installed on Tomcat server. Now that wildcard is expiring Security does not want us to install new client cert on back end. We installed a new cert on back-end that was signed by our "local CA". F5 knows that local CA. From tcpdump we see can see that new cert being used to secured communication between F5 (self IP) and pool member (only one server in the back-end). Afterwards we also see VS (not self IP) is also sending the new cert as part of SSL handshake that is where application breaks. Hope it shades some light. Thanks.

masajjad
Cirrus
Cirrus

I came across this https://mta.openssl.org/pipermail/openssl-users/2017-April/005683.html

 

> Secure Sockets Layer

> TLSv1.2 Record Layer: Alert (Level: Fatal, Description: Certificate Unknown)

> Content Type: Alert (21)

> Version: TLS 1.2 (0x0303)

> Length: 2

> Alert Message

> Level: Fatal (2)

> Description: Certificate Unknown (46)

Client objects to the server chain. Either does not trust the MiTM root CA, or

is unhappy about its encoding (assuming tshark is not generating an FP warning).

 

And here is the tcpdump analysis that highlights our situation.

  1. Bank-end sends Locally signed Cert to F5 Self IP. F5 likes it and we see Client Key Exchange.
  2. VS sends new 4096 bit key size client cert (signed by Entrust that signs current one as well) to back-end
  3. Back-end flags Alert Cert Unknown.

 

0691T000008uHQPQA2.png

 

 

Lidev
MVP
MVP

Thank you for this clarification and the tcpdump.

Can you post you SSL profile configuration to help debugging ?

Also, disable Generic Alert in SSL profile, when the setting is disabled, the system sends more specific SSL alert messages.

0691T000008uHRXQA2.png

masajjad
Cirrus
Cirrus

Thanks for you time and effort to help me out. Current wildcard is expiring this Sunday. So, it's a bit of crisis.

 

Disabling Generic Alert in client SSL profile did not add much detail. Both cases I see only handshake error message between pool member and VS. Nothing much besides this error.

 

I had tail -f /var/log/ltm running

 

 

Jul 23 06:32:49 Ottciof5101 warning tmm1[21959]: 01260013:4: SSL Handshake failed for TCP 10.5.15.120:62226 -> 10.5.29.11:443

 

Jul 23 06:38:11 Ottciof5101 warning tmm1[21959]: 01260013:4: SSL Handshake failed for TCP 10.5.15.120:62242 -> 10.5.29.11:443

 

This being a wildcard cert it was used on multiple client SSL profile. I created a custom parent SSL profile pointed the new cert there. And all old SSL profiles were extended from that custom parent instead of factory default

 

(/Common)(tmos)

 

ltm profile client-ssl clientssl_Wildcard_COMPANY {

  app-service none

  cert-key-chain {

    Wildcard_COMPANY_2020_Entrust_Chain_Bundle_Dec_2030_0 {

      cert Wildcard_COMPANY_2020.com

      chain Entrust_Chain_Bundle_Dec_2030

      key Wildcard_COMPANY_2020.com

    }

  }

  defaults-from clientssl

  inherit-ca-certkeychain true

  inherit-certkeychain false

}

 

(cfg-sync In Sync)(Active)(/HR)(tmos)#

 

 

ltm profile client-ssl HR-Accero-SNI_cfmws.com-SHA2 {

  app-service none

  cert-key-chain {

    Wildcard_COMPANY_2020_Entrust_Chain_Bundle_Dec_2030_0 {

      cert /Common/Wildcard_COMPANY_2020.com

      chain /Common/Entrust_Chain_Bundle_Dec_2030

      key /Common/Wildcard_COMPANY_2020.com

    }

  }

  defaults-from /Common/clientssl_Wildcard_COMPANY

  generic-alert disabled

  inherit-ca-certkeychain true

  inherit-certkeychain true

  server-name *.abc.com

  sni-default true

}

ltm profile client-ssl HR-Accero-SNI_cfpsa.com_SHA2 {

  app-service none

  cert-key-chain {

    Wildcard_COMPANY_2020_Entrust_Chain_Bundle_Dec_2030_0 {

      cert /Common/Wildcard_COMPANY_2020.com

      chain /Common/Entrust_Chain_Bundle_Dec_2030

      key /Common/Wildcard_COMPANY_2020.com

    }

  }

  defaults-from /Common/clientssl_Wildcard_COMPANY

  generic-alert disabled

  inherit-ca-certkeychain true

  inherit-certkeychain true

  server-name *.xyz.com

}

 

 

Lidev
MVP
MVP

The configuration of Client SSL profiles looks good.

Can you provide the result of theses commands please ?

openssl s_client -connect <virtual_server>:<port> openssl s_client -connect <backend>:<port>

 

 

Excuse my delay. Pardon me... instead of replying back I was writing answer. Organized mess 🙂

 

@syslog:~$ openssl s_client -connect 10.5.29.11:443

CONNECTED(00000003)

depth=2 C = US, O = "Entrust, Inc.", OU = See www.entrust.net/legal-terms, OU = "(c) 2009 Entrust, Inc. - for authorized use only", CN = Entrust Root Certification Authority - G2

verify return:1

depth=1 C = US, O = "Entrust, Inc.", OU = See www.entrust.net/legal-terms, OU = "(c) 2012 Entrust, Inc. - for authorized use only", CN = Entrust Certification Authority - L1K

verify return:1

depth=0 C = XX, ST = XX, L = XX, O = X Y Z, XX = *.abc.com

verify return:1

---

Certificate chain

 0 s:/C=xxxxx/CN=*.abc.com

  i:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2012 Entrust, Inc. - for authorized use only/CN=Entrust Certification Authority - L1K

 1 s:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2012 Entrust, Inc. - for authorized use only/CN=Entrust Certification Authority - L1K

  i:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2

 2 s:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2

  i:/C=US/O=Entrust, Inc./OU=See www.entrust.net/legal-terms/OU=(c) 2009 Entrust, Inc. - for authorized use only/CN=Entrust Root Certification Authority - G2

---

Server certificate

 

 

Server public key is 4096 bit

 

  Verify return code: 0 (ok)

 

 syslog:~$ openssl s_client -connect 10.5.15.120:443

CONNECTED(00000003)

depth=0 C = XX, ST = XX, L = XX, O = xxxxxxx, OU = IT, CN = *.abc.com

verify error:num=20:unable to get local issuer certificate

verify return:1

depth=0 C = XX, ST = XX, L =XX, O = xxxxxxxx, OU = IT, CN = *.abc.com

verify error:num=21:unable to verify the first certificate

verify return:1

---

Certificate chain

 0 s:/C=xx/ST=xx/L=xx/O=xxxxxxxx/OU=IT/CN=*.abc.com

  i:/DC=com/DC=domain/CN=COLOCAL-CA

---

Server certificate

-----BEGIN CERTIFICATE-----

 

subject=/C=XXXXXXXX/OU=IT/CN=*.abc.com

issuer=/DC=com/DC=domain/CN=COLOCAL-CA

---

No client certificate CA names sent

 

 

Server public key is 4096 bit

 

 

  Verify return code: 21 (unable to verify the first certificate)

---

closed

 

 

I am taking the client doing SSL connection request does not have the Local CA cert installed.

if the test were performed from the BIG-IP, you find the root cause of your issue 😉

Double check the CA assign on your SSL Client and Server profile (you may have forgotten to include intermediate chain) .

if you perform TLS 2 ways, check also Trusted Certificate Authorities.

 

Hi Lidev,

 

Appreciate your continued feedback.

 

  1. Cert of "Local CA" that signed the cert for back-end was imported to F5 (from System File Management SSL Certificate List).
  2. From tcpdump we see back-end send that newly generated cert signed by the Local CA and F5 does client exchange. This suggests F5 knows about the Local CA that signed the cert for back-end
  3. Issue arises when F5 sends (Public CA signed) cert to back-end. We send public CA chain bundle for both and new cert from SSL profile.

 

 

Am I missing something from your reply?

 

Thanks again.

Hello,

It looks like an SSL configuration problem on the backend server side, enable SSL log debug on your F5 BIG-IP .

modify /sys db log.ssl.level value Debug

 don't forget to disable SSL debug logging after by typing the following command: (modify /sys db log.ssl.level value Warning)

 

Also start a SSL Dump to monitor all SSL trafifc (https://support.f5.com/csp/article/K10209)

 

With all this, you will have more information about SSL traffic and you may have more insight into the problem you are facing.

Hi Lidev,

 

Turns out there is some authentication component of the application that requires the frontend cert. We copied the external cert to backend and issue was fixed. We were chasing out own tail.

 

Thanks a lot for your time and effort.

Lidev
MVP
MVP

Hi masajjad,

I'm glad to hear it. please don't forget to mark the answer as "the best" in order to pass your post as resolved.

 

Regards