SSL connection error, client certificate validation issue - RSASSA-PSS signature algorithm support?
2-way SSL VIP with client certificate "required", we are getting SSL connection errors. The LTM log contains: SSL Handshake failed for TCP from 10.20.15.156:57757 to 172.20.104.11:443 The CA chain and leaf certs are all issued by Windows certificate svcs. I believe the problem is the use of the RSASSA-PSS signature algorithm in the certs. When I upload the CA chain and a client cert to the F5 and manually do an "openssl verify -purpose sslclient -verbose -CAfile ./ca.cer ./client.cer", I get error messages as follow: [root@lb:Active:Changes Pending] ~ openssl verify -purpose sslclient -verbose -CAfile ./ca.cer ./client.cer client.cer: /DC=local/DC=ad/OU=People/CN=Bucci, David/emailAddress=David.Bucci@ad.net error 7 at 0 depth lookup:certificate signature failure 10346:error:0D0C50A1:asn1 encoding routines:ASN1_item_verify:unknown message digest algorithm:a_verify.c:152: And my client cert does in fact have signature algorithm set to RSASSA-PSS. In order to prove to myself it's the RSASSA-PSS signature algorithm, I used a different CA chain/child cert, where the intermediate CA cert uses that signature algorithm, but the child cert uses vanilla sha1rsa, and in that case, the error statement changes to "error 7 at 1 depth" ... which I believe means the child cert was fine, but then when it traversed up 1 level to the intermediate CA cert, the problem occurred. When I upload the exact same ca.cer and client.cer to a Red Hat box and run the same command, the certificate verifies fine -- as do the other child/CA chain, where the child has sha1rsa, but the intermediate CA has the RSASSA-PSS. That RHEL box is running openssl version 1.0.1e-fips 11 Feb 2013, while the F5 is at 11.4.1, with openssl version 0.9.8y 5 Feb 2013. Does that seem like a cogent analysis? Are there any other steps it would make sense to take to further verify the root issue? And, if the analysis is correct, is there any way to get the F5 to accept RSASSA-PSS as a signature algorithm? thx!