If I may add, what you're describing is client side Kerberos - clients passing Kerberos tickets to the F5, not server side Kerberos - the F5 passing Kerberos tickets to the application. This distinction is important because the configurations are different. Server side Kerberos, or Kerberos SSO, performs KCD and KPT. Client side Kerberos does not.
So as you've gathered from the referenced guide, the client side Kerberos configuration requires a 401 agent that tells the client to go get a Kerberos ticket, and a Kerberos Auth agent that validates that ticket against a AAA server and local keytab. It's important to also understand that the real work here is done by the client. For example, if the client is inside the same domain, it will go to its local KDC and request a ticket for the HTTP servicePrincipalName of the VIP (the SPN in the keytab). That ticket is a request token that is encrypted with the service's private key, and then encrypted again with the client's private key. The client decrypts the outer "shell" and passes the rest to the service. The keytab in the AAA contains the service's private key. So if the AAA can successfully decrypt and validate (read) the ticket, client side Kerberos is considered successful. Now, a client from another trusted domain must take a few additional steps. Ultimately it must request access to the resource's KDC from its KDC, which will then issue it a ticket for the service.
The first things I would do to troubleshoot this are:
-
Capture the client side Kerberos traffic - look at what a local client is doing, and then what a cross-domain client is doing. Using a tool like Wireshark, you can usually see the Kerberos error messages in the traffic.
-
Enable debug logging for client side Kerberos - aside from setting debug on the APM "access" option in the logging configuration, also do this in the shell:
tmsh modify sys db log.rba.level value debug
Just remember to disable these after it's all working. I would also review this forum thread as it goes into some pretty deep detail on the subject:
https://devcentral.f5.com/questions/kerberos-and-ntlm-authentication-using-apm
Please report back what you find.