Problems with using Kerberos Authentication

Jason, I don't think you're going to find what that error number means, so here are some tips and observations from the field on setting up APM client side Kerberos authentication.

1. First and foremost, in my opinion the absolute BEST way to troubleshoot a Kerberos problem is with WireShark. It has the filters built right in to show you exactly what the Kerberos payloads look like. You'll just need to plant it somewhere in the line of fire, preferably in this case the client or the KDC.

2. You can enable debug logging for client side Kerberos with a TMSH command:

   tmsh modify sys db log.rba.level value debug
   

   But make sure you turn this off when you're done.

   tmsh modify sys db log.rba.level value notice
   

   This log data will show up in /var/log/apm

3. On any Windows machine, the default security posture for IE is to only pass Kerberos credentials to a site in its Local Intranet sites list

   Tools - Internet Options - Security tab - Local Intranet - Sites button - Advanced
   

   You'll want to add this site URL to that list. To do the same thing in Firefox (on a domain-joined Windows machine), type about:config in the address bar and look for:

   network.negotiate-auth.trusted-uris
   

4. Time is critical to Kerberos. You usually have a little bit of fudge room, but it's always best for all parties to be in step. That means the client, KDC, and BIG-IP should all be synchronized.

5. Kerberos also relies very heavily on DNS. Make sure the hostname/SPN of the APM VIP has both an A and PTR record in AD DNS. You cannot use IP addresses with Kerberos.

6. Admittedly some of the documentation out there for setting this stuff up is, at times, vague. Assuming you're using a Windows 2003 or 2008 AD, you'll want to create your keytab file with this syntax:

   ktpass /princ HTTP/vip.example.com@EXAMPLE.COM /mapuser example\krbsvc /ptype KRB5_NT_PRINCIPAL /pass password /out c:\file.keytab
   

   The most important parts being the SPN format - try to use an uppercase SPN service (HTTP/), lowercase server name (vip.example.com), and uppercase realm (@EXAMPLE.COM). You'll also want to use the KRB5_NT_PRINCIPAL ptype. Some of the other ptypes will work, but this is more semantically correct. And finally, though this isn't a hard and fast rule, I've had difficulties in the past when the username was not lowercase.

7. After this command runs, you'll notice in the AD service account's properties that a) the account's User Logon Name is set to the SPN value, and B) a servicePrincipalName attribute has also been created for the account (with this same SPN value). Make sure that is true in your case.

8. Unless you need it for something else, try removing the DNS settings from the BIG-IP and test. I know that may sound odd, but I've seen that cause a problem too. You may also want to fire up WireShark and look for any anomalous DNS traffic.

9. Between tests you'll want to clear any residual Kerberos caches. For APM:

   bigstart restart rba
   

   For Windows (the client):

   klist purge     
   

10. This one is a long shot, but it's also possible that password changes and other account activities can update the KVNO number and put it out of sync with the keytab. To verify, from the KDC:

    ldifde -f c:\spn_out.txt -d "DC=example.com,DC=com" -l *,msDS-KeyVersionNumber -r "(serviceprincipalname=HTTP/vip*)" -p subtree
    

    Look for the "msDS-KeyVersionNumber" string in the output file

    Then from the BIG-IP:

    klist -ekt 
    

    If the KVNO numbers don't match, you'll want to completely delete and recreate the AD service account and re-export a keytab.

11. If all else fails, please respond back with the Kerberos error information you see in the WireShark capture.