Kerberos Authentication from Multiple Forests

If I may add, what you're describing is client side Kerberos - clients passing Kerberos tickets to the F5, not server side Kerberos - the F5 passing Kerberos tickets to the application. This distinction is important because the configurations are different. Server side Kerberos, or Kerberos SSO, performs KCD and KPT. Client side Kerberos does not.

So as you've gathered from the referenced guide, the client side Kerberos configuration requires a 401 agent that tells the client to go get a Kerberos ticket, and a Kerberos Auth agent that validates that ticket against a AAA server and local keytab. It's important to also understand that the real work here is done by the client. For example, if the client is inside the same domain, it will go to its local KDC and request a ticket for the HTTP servicePrincipalName of the VIP (the SPN in the keytab). That ticket is a request token that is encrypted with the service's private key, and then encrypted again with the client's private key. The client decrypts the outer "shell" and passes the rest to the service. The keytab in the AAA contains the service's private key. So if the AAA can successfully decrypt and validate (read) the ticket, client side Kerberos is considered successful. Now, a client from another trusted domain must take a few additional steps. Ultimately it must request access to the resource's KDC from its KDC, which will then issue it a ticket for the service.

The first things I would do to troubleshoot this are:

1. Capture the client side Kerberos traffic - look at what a local client is doing, and then what a cross-domain client is doing. Using a tool like Wireshark, you can usually see the Kerberos error messages in the traffic.

    

2. Enable debug logging for client side Kerberos - aside from setting debug on the APM "access" option in the logging configuration, also do this in the shell:

    

   tmsh modify sys db log.rba.level value debug

    

Just remember to disable these after it's all working. I would also review this forum thread as it goes into some pretty deep detail on the subject:

https://devcentral.f5.com/questions/kerberos-and-ntlm-authentication-using-apm

Please report back what you find.

What is the subject name in the request directly before the KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN message (what is the SPN the client is asking for)? For cross-domain authentication, let's say a user in domA wants to access a resource in domB. The user should already have a TGT for access to domA. When the user attempts to access a resource in domB, it'll get a 401 error message requesting authentication, and then go immediately to its own domain. Through some validation/lookup process, either the client or the domain will know that the requested SPN is for a service in another trusted domain. The client will then request, from domA, a TGT for the domB KDC. It'll then use this TGT to make a TGS request for the resource in domB.

I can create a 2nd account in the 2nd forest and add the same SPN there so it can be found, but do I need to make some form of combined keytab file for the F5 to be able to work with both accounts?

Completely unnecessary. It could be that neither party knows what the SPN goes to. Try running a network capture between the client and the two KDCs and filter on Kerberos (port 88) and DNS (port 53) traffic.

No 401 message, not even one from an AD-bound server, would indicate anything about a specific domain or forest. The F5 in the Forest B environment doesn't care where you get the ticket from as long it can decrypt it based on the keytab. The client and/or KDC in Forest B have to figure that out. Further, the client shouldn't be sending a TGS request to Forest A. It should be sending a krbtgt request to Forest A for access to Forest B's KDC, to then send a TGS request to Forest B.

Is there a global catalog between these forests? Have you determined any DNS failures?

I'm not 100% certain on this. Been awhile since I've tried it. But I'd imagine adding an alternate UPN suffix at the forest level of ONE of the forests should work. Make sure the GC has time to find this information before testing. So in Forest B, add an alternate UPN suffix for "company.com".

Setting up a service account in each Forest and building multiple keytabs that are then combined turned out to be the best working solution here. Thanks for the feedback guys