Forum Discussion
hui_37443
Nimbostratus
NimbostratusOCSP responder trouble shooting
One of our test server is having trouble to initiate an OCSP check. The authentication status value is always misleading 1. However, from the tcp trace, we have noticed that there has been no traffic heading to OCSP server.
On the shell console, we can run "curl -k https://ocsp.server" and get the default html response back. The corresponding tcp dump shows dns enquiry is maded and communication between Big-IP and OCSP Server happens after that. However in the runtime, nothing suggests that there is any effort to make the contact. I am running out of ideas here. The core functionality resides in the call "AUTH::authenticate" and therefore trace statements in iRule are not much useful.
Any suggestions that can help me to reveal the true nature of the issue?
hui_37443The version is 9.4.5.
hoolioCirrostratus
You could try copying the default OCSP auth iRule and add debug logging to see which code paths are being executed.
It would also be good to consider upgrading to 9.4.8 as there have been some significant improvements / fixes since 9.4.5.
Aaron- hoolioThere are also two related fixes available in a hotfix for 9.4.8 for OCSP functionality that may help you:
CR125264 - HTTP::respond should be allowed in CLIENTSSL_HANDSHAKE
CR126501 - OCSP AUTH iRules need to detect server down vs. bad cert
CR126501 in particular may help you troubleshoot. You could contact F5 Support to get details on these issues.
Aaron - hui_37443Here is the iRule we are using. It works on a dev box, which talks to the same OCSP server. Therefore, I believe it does the job. The trace log suggests "doing OCSP" & "authentication status of round 1 $tmm_auth_status" happens in same second.
One major difference on the infrastructure is that the test box uses DNS lookup to resolve the IP, while the dev box uses the local hosts. Not sure whether that posts any difference.
when CLIENT_ACCEPTED {
set tmm_auth_ssl_ocsp_sid [AUTH::start pam default_ssl_ocsp]
}
when CLIENTSSL_CLIENTCERT {
set ssl_cert [SSL::cert 0]
set id [SSL::sessionid]
set hash [b64encode [sha1 $ssl_cert]]
set ssl_array [list blah1 blah2 $hash]
log local0.debug "client certificate base64 encoded: [b64encode $ssl_cert]"
lset ssl_array 0 [X509::verify_cert_error_string [SSL::verify_result]]
if {[matchclass 1 equals $::cbz_ocsp_check]} {
AUTH::cert_credential $tmm_auth_ssl_ocsp_sid $ssl_cert
AUTH::cert_issuer_credential $tmm_auth_ssl_ocsp_sid [SSL::cert issuer 0]
AUTH::subscribe $tmm_auth_ssl_ocsp_sid
AUTH::authenticate $tmm_auth_ssl_ocsp_sid
log local0.debug "doing OCSP"
SSL::handshake hold
} else {
log local0.debug "ocsp bypassed"
lset ssl_array 1 "ocsp_bypassed"
session add ssl "$id any pool" $ssl_array 21600
}
}
when AUTH_RESULT {
if {[info exists tmm_auth_ssl_ocsp_sid] and $tmm_auth_ssl_ocsp_sid eq [AUTH::last_event_session_id]} {
set tmm_auth_status [AUTH::status]
log local0.debug "authentication status of round 1 $tmm_auth_status"
if {$tmm_auth_status eq 0} {
Success
log local0.debug "OCSP verfication succeeded"
SSL::handshake resume
lset ssl_array 1 "auth_success"
} elseif {$tmm_auth_status eq 1} {
Failure
log local0.debug "OCSP verfication failed"
SSL::handshake resume
lset ssl_array 1 "auth_failure"
} elseif {$tmm_auth_status eq -1} {
Error
log local0.debug "OCSP verfication error"
lset ssl_array 1 "auth_failure"
}
else {
Not authorised
log $tmm_auth_status
log local0.debug "OCSP verfication not authorised"
reject
}
session add ssl "$id any pool" $ssl_array 21600
}
}
when HTTP_REQUEST priority 100 {
log local0. "Inside ocsp"
if {not([matchclass [string tolower [HTTP::path]] contains $::cbz_data_static])} {
if {[matchclass 1 equals $::cbz_ocsp_check]} {
set id [SSL::sessionid]
set ssl_array1 [session lookup ssl "$id any pool"]
set ssl_data0 [lindex $ssl_array1 0]
set ssl_data1 [lindex $ssl_array1 1]
if { $ssl_data0 contains "expired" } {
log local0.debug "Certificate expired"
HTTP::path "/common/common.web/html/CertificateExpired.html"
} elseif { $ssl_data1 contains "auth_failure" } {
log local0.debug "Authentication failure"
HTTP::path "/Common/Common.Web/html/OcspFailed.html"
} elseif { $ssl_data0 contains "ok" } {
HTTP::header insert SSLCLientCertStatus $ssl_data1
} else {
log local0.debug "unknow error"
HTTP::path "/common/common.web/html/CertificateError.html"
}
} else {
log local0.debug "Generate ocsp bypass header"
HTTP::header insert OcspBypassed 1
}
}
} - hui_37443Turned out to be the certificate name case issue. There has been no sign to show F5 having trouble to pick up the signing cert. One of the colleagues had similar experience before so we tried and the problem is gone. In general this trial & error approach is not very efficient. It would be great that F5 can provide some facilities to reveal the problems inside the builtin modules.
Cheers, - hoolioHi Hui,
I ran into a similar issue while testing an OCSP iRule in that I wasn't able to get any details on why the OCSP lookup failed. F5 development came up with a fix as described in CR126501:
The solution is to return additional information via AUTH::response_data, which is what we already did for LDAP. The DevCentral Wiki page has more information on that command. For OCSP, the key to use is "ocsp:response:status". For the particular case that this customer is interested in, where the server is unreachable, the error message we return is "Error (Could not connect to server)". It is important to note the second paragraph on the DevCentral page. It states: "AUTH::subscribe must first be called to register interest in query results prior to calling AUTH::authenticate. As a convenience when using the builtin system auth rules, these rules will call AUTH::subscribe if the variable tmm_auth_subscription is set. Instead of calling AUTH::subscribe directly, we recommend setting tmm_auth_subscription to "*" when using the builtin system auth rules in the interest of forward-compatibility." If AUTH::response_data is returning no data at all, it's probably because the AUTH::subscribe steps above were not taken.
This fix is available in a hotfix for 9.4.8.
Aaron 
dimka___104021may be offtopic, but still couple words about 9.4.8:
If VS has client ssl profile with REQUEST auth mode enabled and client doesn't have a client certificate TLSv1 in IE6.0 is not working. You just get "page cannot be displayed".
SSL debug shows, that fatal error "unexpected message" is occured. But REQUEST should always grant access to resource, regarding of presence of client certificate.
In older versions e.g. 9.4.7, everything works like it should.
I've case @support, but they're mumbling "we don't have any workaround for this issue."- hoolioHi Dimka,
Can you try adding SSL::session invalidate before calling SSL::renegoatiate and see if IE6 clients then work? One issue I was seeing with some versions of IE is that they wouldn't use a new SSL session ID when LTM was re-negotiating the SSL handshake to request the client cert. Calling SSL::session invalidate seemed to force IE to select a new SSL session ID and then the handshake completed.
Aaron