Forum Discussion
MikeRobinson_64
Nimbostratus
Dec 01, 2009auth_result not called for some client certificates
Hi everyone,
I am trying to resolve an issue with an irule that is utilized to ask for client certificates for certain folders. The irule works great except for certain client certificates. I have noticed from the statistics in the irule editor that when testing with these certificates the session does not appear to pass through the auth_result section. As a result the user gets to see the ever popular "page cannot be displayed" message. I need to get this resolved for a larger test. The irule is pasted below. Thanks in advance for your help!
Mike
when RULE_INIT {
set tmm_auth_subscription "*"
}
when CLIENT_ACCEPTED {
set needcert 0
set gotcert 0
set badcert 0
set tmm_auth_ssl_ocsp_sid 0
set tmm_auth_ssl_ocsp_done 0
}
when CLIENTSSL_HANDSHAKE {
set certcnt [SSL::cert count]
if { $certcnt > 0 } {
set gotcert 1
HTTP::release
}
}
when CLIENTSSL_CLIENTCERT {
set tmm_auth_ssl_ocsp_done 0
if { $needcert == 1 and $tmm_auth_ssl_ocsp_sid == 0} {
set tmm_auth_ssl_ocsp_sid [AUTH::start pam ssl_ocsp_prod]
AUTH::subscribe $tmm_auth_ssl_ocsp_sid
set varcert [SSL::cert 0]
AUTH::cert_credential $tmm_auth_ssl_ocsp_sid $varcert
AUTH::cert_issuer_credential $tmm_auth_ssl_ocsp_sid [SSL::cert issuer 0]
AUTH::authenticate $tmm_auth_ssl_ocsp_sid
set id [SSL::sessionid]
set ssl_array [list blah1 blah2]
lset ssl_array 0 [X509::verify_cert_error_string [SSL::verify_result]]
SSL::handshake hold
}
}
when AUTH_RESULT {
array set auth_response_data [AUTH::response_data]
set auth_status [AUTH::status]
set ocsp_status [lindex [array get auth_response_data ocsp:response:status] 1]
if {$tmm_auth_ssl_ocsp_sid eq [AUTH::last_event_session_id]} {
if {[AUTH::status] == 2} {
reject
}
elseif {[AUTH::status] != 0} {
SSL::handshake resume
lset ssl_array 1 "auth_failure"
session add ssl $id $ssl_array 21600
log local0.debug "ocsp_status = $ocsp_status"
log local0.debug "status = autherror"
log local0.debug "SSLClientCertIssuer [X509::issuer $varcert]"
log local0.debug "SSLClientCertNotValidBefore [X509::not_valid_before $varcert]"
log local0.debug "SSLClientCertNotValidAfter [X509::not_valid_after $varcert]"
log local0.debug "SSLClientCertSubject [X509::subject $varcert]"
}
else {
SSL::handshake resume
lset ssl_array 1 "auth_success"
session add ssl $id $ssl_array 21600
}
}
}
when HTTP_REQUEST {
switch -glob [string tolower [HTTP::uri]] {
"/folder1/*" -
"/folder2/*" {
set id [clientside {SSL::sessionid}]
set ssl_array1 [session lookup ssl $id]
set ssl_data0 [lindex $ssl_array1 0]
set ssl_data1 [lindex $ssl_array1 1]
if {$gotcert == 0} {
if { [SSL::cert count] == 0} {
HTTP::collect
set needcert 1
SSL::authenticate always
SSL::authenticate depth 9
SSL::cert mode require
SSL::session invalidate
SSL::renegotiate
}
}
else {
pool my_pool
}
}
}
}
when LB_SELECTED {
if {$needcert == 1} {
set ssl_array1 [session lookup ssl $id]
set ssl_data0 [lindex $ssl_array1 0]
set ssl_data1 [lindex $ssl_array1 1]
if { $ssl_data0 contains "expired" } {
set cert_status "expired"
}
elseif { $ssl_data1 contains "auth_failure" } {
set cert_status "revoked"
}
elseif { $ssl_data0 contains "ok" } {
set cert_status "success"
}
else {
set cert_status "other"
}
HTTP::header replace "SSLCLientCertStatus" $cert_status
HTTP::header replace "SSLClientCertVersion" [X509::version $varcert]
HTTP::header replace "SSLClientCertSerialNumber" [X509::serial_number $varcert]
HTTP::header replace "SSLClientCertIssuer" [X509::issuer $varcert]
HTTP::header replace "SSLClientCertNotValidBefore" [X509::not_valid_before $varcert]
HTTP::header replace "SSLClientCertNotValidAfter" [X509::not_valid_after $varcert]
HTTP::header replace "SSLClientCertSubject" [X509::subject $varcert]
}
}
13 Replies
- hoolio
Cirrostratus
Can you post logs from a success and failure? Is there anything noticeably different about the client certs which fail?
Aaron - MikeRobinson_64
Nimbostratus
Thanks for the reply Aaron! I would but the logs do not show anything to work with. I do not have successes log anything and real failures (revoked or expired) show entries for the items I am logging. This particular failure shows nothing. I have added log statements previously to capture the sid value and to show the cert variable with success but I cannot figure out how to handle an error (one that does not produce and auth_result) out of auth::authenticate. The interesting thing to note is that this same card and client certificate works on other PCs. On the user's PC is does not work with the irule but it works with other sites that require certificates. I need to get this resolved as I believe this issue occurred with other users during our large scale test we previously ran. One last piece of information, successes seem to have a one to one ratio on the CLIENTSSL_CLIENTCERT and AUTH_RESULT statistics but this type of failure shows 2 times through the CLIENTSSL_CLIENTCERT section and 0 through the AUTH_RESULT section. Thanks again!
Mike - hoolio
Cirrostratus
Hi Mike,
I'm having a hard time following the exact details of the failure without seeing any logs of a failure.
Is it possible that the client is not sending the client cert first in a set of client and intermediate certs that might be in their client cert bundle? Can you log the subjects for [SSL::cert 0] and higher certs that may be there with something like this:for { set i 0 } { $i < [SSL::cert count] } { incr i } { log local0. "Cert $i subject: [X509::subject [SSL::cert $i]]" }
Aaron - MikeRobinson_64
Nimbostratus
I had them try it twice and here are the results (X's have been added to protect PII):
Dec 2 09:54:39 tmm tmm[1864]: Rule irule_CAC_Required : Cert 0 subject: CN=XXXXXXXXXXXX,OU=XXXXXXXXXX,OU=PKI,OU=DoD,O=U.S. Government,C=US
Dec 2 09:54:39 tmm tmm[1864]: Rule irule_CAC_Required : Cert 1 subject: CN=DOD CA-20,OU=PKI,OU=DoD,O=U.S. Government,C=US
Dec 2 09:54:39 tmm tmm[1864]: Rule irule_CAC_Required : Cert 2 subject: CN=DoD Root CA 2,OU=PKI,OU=DoD,O=U.S. Government,C=US
Dec 2 09:54:39 tmm tmm[1864]: Rule irule_CAC_Required : Cert 3 subject: CN=DoD Interoperability Root CA 1,OU=PKI,OU=DoD,O=U.S. Government,C=US
Dec 2 09:54:39 tmm tmm[1864]: Rule irule_CAC_Required : Cert 4 subject: OU=Entrust,OU=FBCA,O=U.S. Government,C=US
Dec 2 09:57:11 tmm tmm[1864]: Rule irule_CAC_Required : Cert 0 subject: CN=XXXXXXXXXXXX,OU=XXXXXXXXXX,OU=PKI,OU=DoD,O=U.S. Government,C=US
Dec 2 09:57:11 tmm tmm[1864]: Rule irule_CAC_Required : Cert 1 subject: CN=DOD CA-20,OU=PKI,OU=DoD,O=U.S. Government,C=US
Dec 2 09:57:11 tmm tmm[1864]: Rule irule_CAC_Required : Cert 2 subject: CN=DoD Root CA 2,OU=PKI,OU=DoD,O=U.S. Government,C=US
Dec 2 09:57:11 tmm tmm[1864]: Rule irule_CAC_Required : Cert 3 subject: CN=DoD Interoperability Root CA 1,OU=PKI,OU=DoD,O=U.S. Government,C=US
Dec 2 09:57:11 tmm tmm[1864]: Rule irule_CAC_Required : Cert 4 subject: OU=Entrust,OU=FBCA,O=U.S. Government,C=US
It looks like the certificates are coming through...
Mike - hoolio
Cirrostratus
Hi Mike,
Sorry, I was just trying to get you to check if, on failures, the first cert in the set of certs the client sends is not the actual client cert. I'm guessing from your anonymized logs, that it is the actual client cert sent first. So that wasn't the problem.
Can you post anonymized logs from the full iRule for a failure?
Thanks,
Aaron - MikeRobinson_64
Nimbostratus
Hi Aaron,
Sorry if I am being a bit dense here but are you looking for something other than the LTM log? Since I do not know what to log for this failure there is nothing in the LTM log. If you need a different log let me know as I am just not sure what you are looking for.
Mike - hoolio
Cirrostratus
Hi Mike,
Sorry for not being clearer--I was thinking you already had a fair amount of debug logging in the iRule. I was confusing it with another post. Could you add debug logging to each major code block in the existing iRule and post the /var/log/ltm output from a failure. Here's an example of the kind of info that would be helpful:when CLIENT_ACCEPTED { set needcert 0 set gotcert 0 set badcert 0 set tmm_auth_ssl_ocsp_sid 0 set tmm_auth_ssl_ocsp_done 0 log local0. "[IP::client_addr]:[TCP::client_port]: New TCP connection" } when CLIENTSSL_HANDSHAKE { log local0. "[IP::client_addr]:[TCP::client_port]: Cert count: [SSL::cert count]" set certcnt [SSL::cert count] if { $certcnt > 0 } { log local0. "[IP::client_addr]:[TCP::client_port]: Cert0 subject: [X509::subject [SSL::cert 0]]. Releasing HTTP." set gotcert 1 HTTP::release } } when CLIENTSSL_CLIENTCERT { set tmm_auth_ssl_ocsp_done 0 if { $needcert == 1 and $tmm_auth_ssl_ocsp_sid == 0} { set tmm_auth_ssl_ocsp_sid [AUTH::start pam ssl_ocsp_prod] AUTH::subscribe $tmm_auth_ssl_ocsp_sid set varcert [SSL::cert 0] AUTH::cert_credential $tmm_auth_ssl_ocsp_sid $varcert AUTH::cert_issuer_credential $tmm_auth_ssl_ocsp_sid [SSL::cert issuer 0] AUTH::authenticate $tmm_auth_ssl_ocsp_sid set id [SSL::sessionid] set ssl_array [list blah1 blah2] lset ssl_array 0 [X509::verify_cert_error_string [SSL::verify_result]] log local0. "[IP::client_addr]:[TCP::client_port]: SessionID: [SSL::sessionid],\ SSL verify result: [X509::verify_cert_error_string [SSL::verify_result]]. Holding SSL handshake." SSL::handshake hold } } ...
Aaron - MikeRobinson_64
Nimbostratus
Ok, got it. Here are the scrubbed results (shows one good attempt and two bad attempts):
Dec 3 10:28:16 tmm tmm[1864]: Rule irule_CAC_Required : BAD.BAD.BAD.BAD:29757: New TCP connection
Dec 3 10:28:17 tmm tmm[1864]: Rule irule_CAC_Required : BAD.BAD.BAD.BAD:29757: Cert count: 0
Dec 3 10:28:29 tmm tmm[1864]: Rule irule_CAC_Required : BAD.BAD.BAD.BAD:29757: SessionID: db87c438a301cd28544bab28d0ded00e525d8725a9
05020f04ab0c072ab2ea5c, SSL verify result: unable to get local issuer certificate. Holding SSL handshake.
Dec 3 10:28:29 tmm tmm[1864]: Rule irule_CAC_Required : BAD.BAD.BAD.BAD:29757: SessionID: db87c438a301cd28544bab28d0ded00e525d8725a9
05020f04ab0c072ab2ea5c, SSL verify result: unable to get local issuer certificate. Holding SSL handshake.
Dec 3 10:29:05 tmm tmm[1864]: Rule irule_CAC_Required : GOOD.GOOD.GOOD.GOOD:30711: New TCP connection
Dec 3 10:29:05 tmm tmm[1864]: Rule irule_CAC_Required : GOOD.GOOD.GOOD.GOOD:30711: Cert count: 0
Dec 3 10:29:06 tmm tmm[1864]: Rule irule_CAC_Required : GOOD.GOOD.GOOD.GOOD:30711: SessionID: db87c438a301aa6f544bab28d0dedff4525d8725a
905020f04ab0c072ab2ea26, SSL verify result: ok. Holding SSL handshake.
Dec 3 10:29:06 tmm tmm[1864]: Rule irule_CAC_Required : GOOD.GOOD.GOOD.GOOD:30711: Cert count: 2
Dec 3 10:29:06 tmm tmm[1864]: Rule irule_CAC_Required : GOOD.GOOD.GOOD.GOOD:30711: Cert0 subject: CN=XXXXXX.XXXXXXX.XXXXXX.XXXXXXXX,OU=
USA,OU=PKI,OU=DoD,O=U.S. Government,C=US. Releasing HTTP.
Dec 3 10:30:24 tmm tmm[1864]: Rule irule_CAC_Required : BAD.BAD.BAD.BAD:32433: New TCP connection
Dec 3 10:30:24 tmm tmm[1864]: Rule irule_CAC_Required : BAD.BAD.BAD.BAD:32433: Cert count: 0
Dec 3 10:30:35 tmm tmm[1864]: Rule irule_CAC_Required : BAD.BAD.BAD.BAD:32433: SessionID: db87c438a301ccd5544bab28d0dedf38525d8725a9
05020f04ab0c072ab2ead3, SSL verify result: unable to get local issuer certificate. Holding SSL handshake.
Any idea on what to do with the error presented?
Mike - hoolio
Cirrostratus
Hi Mike,
Great, that helps. The "unable to get local issuer certificate" error indicates that the client cert can't be chained back to any cert in the trusted CA cert bundle configured on the client SSL profile.
If you take a look at the failing cert's issuer details, does it match the working client certs? You can use openssl from the LTM command line to test the cert:
openssl verify
http://www.openssl.org/docs/apps/verify.html
Here is another method for checking client certs:
http://www.cyberciti.biz/faq/test-ssl-certificates-diagnosis-ssl-certificate/
Aaron - MikeRobinson_64
Nimbostratus
Hi Aaron,
Thanks for the info! The issuer does match what is in the bundle and the certificate is able to be used on other PCs without issue. I am trying to figure out what is special about this one machine and certificate combo. We also have other users in the field that have the same issuer and do not have any problems.
Mike
Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects
