Invalid Session ID. Your session may have expired - during kerberos auth

I have an IdP setup that is doing client side kerberos auth before then sending on a SAML token to an SP. I have an intermittant issue were sometimes using the kerberos auth It seems to fail and send...