Many times you cannot validate mismatch in Cer/Key/Chain combination from TCPdump or SSL DUMP.

To reduce the troubleshooting time you can try following process if oyu are not sure of if the cert+Key+chain are valid for each other or not.

its a very important step to isolate the issue.

You need to validate the cert key Chain pair MD5 validity on both sides Client Side SSL profile and server Side SSL profile first to validate the right cert key and chain pair are in use.

you need to gather the path of all the config items as follows before start executing the commands to get the MD5 hash to validate right cert key and chain pair are in use

I use WINSCP software to get these paths.

You can also use cd command to get these paths from the CLI.

Get client side certPath
===============
/config/filestore/files_d/Common_d/certificate_d/\:Common:TEST_server_cert2_66362_1

Get client side Key Path

/config/filestore/files_d/Common_d/certificate_key_d/\:Common:TEST_server_key2_66369_1

Get client side Chain Path
==========
/config/filestore/files_d/Common_d/certificate_d/\:Common:DIGICERT_BUNDLE_66373_5

Now once the path are discovered, you can replace paths in the following command as per your config

========1.===To obtain Cert MD5====

openssl x509 -in  /config/filestore/files_d/Common_d/certificate_d/\:Common:TEST_server_cert2_66362_1 -pubkey -noout | md5sum

========2.===To obtain Key MD5====

openssl rsa -in /config/filestore/files_d/Common_d/certificate_key_d/\:Common:TEST_server_key2_66369_1  -pubout | md5sum

========3.===Validating  Get client side chain & Get client side Cert are  matching ====

openssl verify -CAfile /config/filestore/files_d/Common_d/certificate_d/\:Common:DIGICERT_BUNDLE_66373_5  /config/filestore/files_d/Common_d/certificate_d/\:Common:TEST_server_cert2_66362_1 

Cert and Chain not matching


Do the same above steps for Server Side SSL profile Cer/Key/Chain combination validation

************************************Get server side*******************************************

root@(Test-Box1-Active)(cfg-sync In Sync)(Standby)(/Common)(tmos)# bash
[root@Test-Box1-Active:Standby:In Sync] ~ # openssl x509 -in  /config/filestore/files_d/Common_d/certificate_d/\:Common:TEST_server_cert2_66362_1 -pubkey -noout | md5sum
89897d536c761cdec4c0a2f02fc965657  -
[root@Test-Box1-Active:Standby:In Sync] ~ #
[root@Test-Box1-Active:Standby:In Sync] ~ #
[root@Test-Box1-Active:Standby:In Sync] ~ #
[root@Test-Box1-Active:Standby:In Sync] ~ # openssl rsa -in /config/filestore/files_d/Common_d/certificate_key_d/\:Common:TEST_server_key2_66369_1  -pubout | md5sum
writing RSA key
89897d536c761cdec4c0a2f02fc965657  -
[root@Test-Box1-Active:Standby:In Sync] ~ # openssl verify -CAfile /config/filestore/files_d/Common_d/certificate_d/\:Common:DIGICERT_BUNDLE_66373_5  /config/filestore/files_d/Common_d/certificate_d/\:Common:TEST_server_cert2_66362_1
/config/filestore/files_d/Common_d/certificate_d/:Common:TEST_server_cert2_66362_1: CN = TEST.cloud1.mydomain.com, C = CA, O = *****SCRUBBED******, OU = For Intranet Use Only, OU = Infrastructure Planning and Engineering
error 20 at 0 depth lookup:unable to get local issuer certificate
[root@Test-Box1-Active:Standby:In Sync] ~ #
[root@Test-Box1-Active:Standby:In Sync] ~ #
[root@Test-Box1-Active:Standby:In Sync] ~ #

=====================================================================

Server Cert Key Chain

Server Cert Path

/config/filestore/files_d/Common_d/certificate_d/\:Common:2024_TEST.cloud1.mydomain.com_67357_1

Server  key path
/config/filestore/files_d/Common_d/certificate_key_d/\:Common:2024_TEST.cloud1.mydomain.com_67373_1

Server chain Path

/config/filestore/files_d/Common_d/certificate_d/\:Common:2024_CHAIN_TEST.cloud1.mydomain.com_67364_1

========1.===To obtain Server-Side Cert MD5====

openssl x509 -in  /config/filestore/files_d/Common_d/certificate_d/\:Common:2024_TEST.cloud1.mydomain.com_67357_1 -pubkey -noout | md5sum

========2.===To obtain Server-Side Key MD5====

openssl rsa -in /config/filestore/files_d/Common_d/certificate_key_d/\:Common:2024_TEST.cloud1.mydomain.com_67373_1  -pubout | md5sum

========3.===Validating  Server-Side chain &  Cert are  matching ====

openssl verify -CAfile /config/filestore/files_d/Common_d/certificate_d/\:Common:2024_CHAIN_TEST.cloud1.mydomain.com_67364_1  /config/filestore/files_d/Common_d/certificate_d/\:Common:2024_TEST.cloud1.mydomain.com_67357_1

OutPut for Server-Side Cert Chain

[root@Test-Box1-Active:Standby:In Sync] ~ # openssl x509 -in  /config/filestore/files_d/Common_d/certificate_d/\:Common:2024_TEST.cloud1.mydomain.com_67357_1 -pubkey -noout | md5sum
99999960d64ce6bffef9c5fc2999aede  -
[root@Test-Box1-Active:Standby:In Sync] ~ # openssl rsa -in /config/filestore/files_d/Common_d/certificate_key_d/\:Common:2024_TEST.cloud1.mydomain.com_67373_1  -pubout | md5sum
writing RSA key
99999960d64ce6bffef9c5fc2999aede  -
[root@Test-Box1-Active:Standby:In Sync] ~ # 

[root@Test-Box1-Active:Standby:In Sync] ~ #
[root@Test-Box1-Active:Standby:In Sync] ~ #
[root@Test-Box1-Active:Standby:In Sync] ~ # openssl verify -CAfile /config/filestore/files_d/Common_d/certificate_d/\:Common:2024_CHAIN_TEST.cloud1.mydomain.com_67364_1  /config/filestore/files_d/Common_d/certificate_d/\:Common:2024_TEST.cloud1.mydomain.com_67357_1

Output

/config/filestore/files_d/Common_d/certificate_d/:Common:2024_TEST.cloud1.mydomain.com_67357_1: OK

=========================

If No error you will get OK in the output , ok in output means CERT+CHAIN are matching.

[root@Test-Box1-Active:Standby:In Sync] ~ #

openssl rsa -des -in /config/filestore/files_d/Common_d/certificate_key_d/\:Common:2024_TEST.cloud1.mydomain.com_67373_1 -out :Common:2024-CER-ENC-TEST.cloud1.mydomain.com.key



==============================================


Here is the article for more details
https://my.f5.com/manage/s/article/K13349

K13349: Verifying SSL certificate and key pairs from the command line (11.x - 16.x)

Description

The BIG-IP system uses SSL encryption for functions, such as load balancing Client and Server SSL virtual servers, and securing administrative connections. Occasionally, you may need to verify SSL certificate and key pairs by using the command line. You can verify whether a given SSL certificate and SSL key match, by comparing the public key information obtained from both. If the public key information for each is the same, then the SSL certificate and SSL private key are a matching pair.

Prerequisites

You must meet the following prerequisite to use this procedure:

• You have command line access to the BIG-IP system.

Procedures

Verifying SSL certificate and key pairs used by Client and Server SSL profiles

Impact of procedure: The following procedure should not have a negative impact on your system.

1. Log in to the BIG-IP command line.
2. Locate the SSL certificate and key pair used by the SSL profile using the following table:
    

   File	Location	Example
   certificate	/config/filestore/files_d/<partition>_d/certificate_d/	/config/filestore/files_d/Common_d/certificate_d/
   key	/config/filestore/files_d/<partition>_d/certificate_key_d/	/config/filestore/files_d/Common_d/certificate_key_d/

3. To obtain the public key information for the SSL certificate, use the following command syntax:Note: The command output is passed through md5sum to reduce the amount of text compared in the final step.For example, if the SSL certificate used by the SSL profile resides on the Common partition, and is named www.test.com.crt, use the following command:Output should display the md5 message digest, similar to the following example:
4. f8f154ba5cf31b9798b7671549be1bf0
5. openssl x509 -in /config/filestore/files_d/Common_d/certificate_d/\:Common\:www.test.com.crt_1 -pubkey -noout | md5sum
6. openssl x509 -in <path_to_cert>/<cert_name> -pubkey -noout | md5sum
7. To obtain the public key information for the SSL private key, use the following command syntax:Note: Ensure that you choose the correct key option (RSA, DSA, or EC) for the type of SSL key being verified; most SSL keys use RSA.For example, if the SSL key used by the SSL profile resides on the Common partition, is named www.test.com.key, and is the RSA type, use the following command:Note: For passphrase encrypted keys you will be prompted to enter the passphrase. Optionally, to prevent being prompted for the passphrase, you can include the -passin pass:<passphrase> option in the command using the following syntax:Output should display the md5 message digest, similar to the following example:
8. f8f154ba5cf31b9798b7671549be1bf0
9. Note: Output will not be echoed to STDOUT. The passphrase will be saved to a variable named REPLY
   read -s
   openssl <rsa|dsa|ec> -in <path_to_key>/<key_name> -passin pass:"${REPLY}" -pubout | md5sum && unset REPLY
10. openssl rsa -in /config/filestore/files_d/Common_d/certificate_key_d/\:Common\:www.test.com.key_1 -pubout | md5sum
11. openssl <rsa|dsa|ec> -in <path_to_key>/<key_name> -pubout | md5sum

• Compare the md5 message digests from steps 3 and 4 to ensure that they are the same.

Verifying the default SSL certificate and key pair used by the Configuration utility

Impact of procedure: The following procedure should not have a negative impact on your system.

1. Log in to the BIG-IP command line.
2. To obtain the public key information for the SSL certificate, use the following command syntax:Note: The command output is passed through md5sum to reduce the amount of text compared in the final step.For example:Output should display the md5 message digest, similar to the following example:
3. 95500b0e49e155da5cc6eee0cdb99911
4. openssl x509 -in /config/httpd/conf/ssl.crt/server.crt -pubkey -noout | md5sum
5. openssl x509 -in <path_to_cert>/<cert_name> -pubkey -noout | md5sum
6. To obtain the public key information for the SSL private key, use the following command syntax:openssl rsa -in <path_to_key>/<key_name> -pubout | md5sumopenssl rsa -in /config/httpd/conf/ssl.key/server.key -pubout | md5sumNote: Output will not be echoed to STDOUT. The passphrase will be saved to a variable named REPLY
   read -s
   openssl rsa -in <path_to_key>/<key_name> -passin pass:"${REPLY}" -pubout | md5sum && unset REPLY95500b0e49e155da5cc6eee0cdb99911
7. Output should display the md5 message digest, similar to the following example:
8. Note: For passphrase encrypted keys you will be prompted to enter the passphrase. Optionally, to prevent being prompted for the passphrase, you can include the -passin pass:<passphrase> option in the command using the following syntax:
9. For example:
10. Compare the md5 message digests from steps 2 and 3 to ensure that they are the same.

Rate if it helps.

HTH

F5 Design Engineer

Tcpdump can be very helpful in capturing and analyzing the traffic between the client, F5, and backend Nginx node. You can use the following command to capture the traffic:

tcpdump -i <interface> -s0 -w /var/tmp/capture.pcap host <client_ip> and host <nginx_ip> and port <nginx_port>

Once you have the capture file, you can transfer it to your local machine and analyze it using Wireshark or another packet analysis tool.

Look for the SSL handshake packets and examine the Certificate message to see if the client certificate is being sent.

Nginx Configuration:

Ensure that your Nginx server is configured to request and accept client certificates. This typically involves setting the ssl_verify_client directive to on or optional and specifying the trusted CA certificates with ssl_client_certificate.

Check the Nginx logs to see if there are any messages related to client certificate validation. The logs can provide clues about whether the certificate is being received and if there are any issues with it.

Finally

You can also use tmsh commands on the F5 to get more insights into the SSL profile and connections. For example:

tmsh show ltm profile client-ssl <profile_name>

as zamroni777 says tcpdump should be enough. it helps knowing what to look for, the article gives an example:

https://medium.com/@mintdev/debug-mtls-using-wireshark-b805cf47a1ea

also notice the TLS1.3 section, in that case you might have to look into decrypting traffic.

yes, tcpdump will show details of the ssl session setup.

if there is ips/ids/ngfw between f5 and pool member, they might do ssl proxy that removes the f5's client cert.