It appears you are trying to achieve the server side Kerberos SSO. From the error it appears DC is responding back with 401 response. Please check if you have correctly followed all the steps as per below
https://devcentral.f5.com/s/articles/apm-cookbook-single-sign-on-sso-using-kerberos
- create new service account
- set SPN and set it's properties
- reverse DNS
- create Kerberos SSO profile using service account credentials
- Setup VPE policy using SSO credentials mapping.
If you are still getting error analyse the wireshark capture and APM logs again. Try to get help from your active directory team also to troublesoot