Forum Discussion
Client SSL authentication error - hs msg overflow
I'm trying to set up 2-way SSL with a SSL bridging scenario using LTM 12.1.2. As of right now the "server" side of this is working, with the F5 sending a cert to the server and requiring an auth cert in return. For the client SSL however, I seem to be running into issues with the cert exchange. Here's an abbreviated version of the conversation I'm seeing:
Client Hello Server Hello, Certificate, Alert (Level: Fatal, Description: Handshake Failure)
The specific SSL error that I see reported in the LTM logs is "ssl_hs_tx:2661: hs msg overflow (40)"
I'm guessing that the certificate request is failing (since that would be the next step after sending the certificate), but I'm not sure why. So far I haven't been able to find many suggestions pertaining to this error specifically, but I have been following some of the general steps for handshake troubleshooting (like the ones in https://support.f5.com/csp/article/K15292), and have come up empty. The error says "overflow" but the only buffer I can think of that might be overflowing is the 32KB SSL buffer, but these requests haven't even hit 5KB yet based on what I'm seeing in tcpdumps.
I'm at a bit of a loss as to what I could be missing in the clientSSL profile, right now I'm requesting a client auth cert, I have the proper CA chain, and I'm advertising the default trusted bundle of CA's. Everything else is default.
If anyone has any thoughts on what the problem could be or what could be missing, I would appreciate any help. I can provide screenshots or other details if needed.
Looks like some problem in mutual auth config. Please check clientssl profile settings as per doc: https://support.f5.com/csp/article/K14783
If issue persists, provide clientssl profile details.
- Effrum
Nimbostratus
For anyone having a similar issue, it turns out the problem was using the default ca-bundle on the F5 for the "Advertised Certificate Authorities" portion of client auth. It appears that some servers require that this field NOT be NULL, but sending the default ca-bundle was too large for the SSL handshake. I created a new CA bundle using only the public cert of our internal CA and it worked.
Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com