Forum Discussion
CirrostratusCan't change AD password through APM
Hi!
I'm having some problems with password change for expired AD passwords through APM.
Running on APM+LTM 11.2.1 build 797.
Using "AD Auth" in the access policies to authenticate users.
The session log shows
AD module: Can't contact KDC 127.7.0.3 for domain xxxxxx.xx
AD module: (null) '' failed: (1331653729)
AD module: (): (1331653729)
AD module: LEAVE Function chgUserPassword
AD agent: Auth (logon attempt:1): failed to change password for 'xxxxxxxxx'
I found this old post describing a similiar issue, but it didn't help much in this case, so I'm making a new post, hoping that perhaps someone else has run into this issue.
The errors says that KDC can't be contacted, but running a packet capture for the DC shows that APM can communicate with them.
The bad thing I've noticed is that after a user has tried to change password and gotten this error, AD auth seems to fail completly for all users and all access policies for about 10 to 15 minutes.
I've verified this by keeping tcpdump running against the DCs. After the issue has occurred, 0 packets are captured for the DCs when users try to authenticate.
The only way I've been able to force it to work again is to recreate the AAA server and reapply all access policies.
I have a test environment running the same TMOS version and basically the same access policies with the same DCs, same AAA server config etc. In that env, password change works.
The only difference I've so far been able to see is that the test environment communicates over UDP to the DCs while the production env (where passwd change doesn't work) _only_ communicate over TCP.
To troubleshoot this further, is there any way to switch APM to use UDP instead of TCP for active directory?
Or perhaps vice versa, to force it to use TCP, so I can switch the test env to TCP and see if the problem occurs there as well?
Thanks.
/Andreas
13 Replies
- Kevin_Stewart
Employee
Are you certain that the production environment doesn't start with UDP and then switch to TCP? That is the usual case for AD/Kerberos authentication. Also, how do you have your AD AAA configured? Are you specifying the AD by IP address or by domain name? 
AndOs
Thanks for replying.
In test env. I can see that it start off by using UDP and then switches to TCP and then back again when APM is authenticating and changing password.
In prod I'm not seeing that, only tcp over port 88
I'm running the same tcpdump command in both test and prod.
tcpdump -s 0 -vnni 0.0:nnn -w active_directory.pcap '(host 10.0.155.10 or host 10.1.155.10 or host 10.10.155.10 or host 10.11.155.10)'
10.0.155.10, 10.1.155.10, 10.10.155.10, 10.11.155.10 are our DCs.
I've setup the AAA to use a pool and specified the DCs with their ip addresses.
Basically:
Domain Name: domain.com
Server connection: use pool
Domain Controller Pool Name: /Common/domain.com_ActiveDirectory_AAA_pool
Domain controllers: 10.0.155.10, 10.1.155.10, 10.10.155.10, 10.11.155.10
Server pool monitor: active_directory_kerberos_monitor
Admin name: BigIp_AD
Admin password: ******
Timeout: 15 sec
Monitor active_directory_kerberos_monitor does a tcp_half_open to port 88.
As a shot in the dark I removed the monitor from the AAA just to see if the monitor affected the ability to use UDP, but it didn't change anything.
/Andreas- Kevin_StewartPlease try the following two things:
1. In your AD AAA, remove the pool and monitor and leave only the domain name and admin user/pass. DNS should be configured so that BIG-IP can resolve this domain name.
2. Compare the /etc/krb5.conf file between test and prod. Do you see any significant differences? - AndOs1. In your AD AAA, remove the pool and monitor and leave only the domain name and admin user/pass. DNS should be configured so that BIG-IP can resolve this domain name.
Use "Server Connection: Direct" instead of Pool?
I switched the AAA to Direct and entered one of our DCs instead of using a pool and password change started to work!
Packet capture now shows that it's talking UDP to DCs just like the test env.
It's still strange though that test uses a pool and everything works there.
2. Compare the /etc/krb5.conf file between test and prod. Do you see any significant differences?
There's no difference that I can see, except that we have one other domain in test env.
I see in /etc/krb5.com that it defines a few log files, but I can't find them under /var/log.
It it possible to turn on those logs?
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log/Andreas
clarksmith_1314Nimbostratus
Hello Friend, I read you issue this is not big issue I send a blog you can read this blog and I hope that your issue will be sure solve. This blog is very informative and all steps are very secure. For more details read this blog : http://activedirectoryselfservice.blogspot.com/2013/06/what-to-do-when-you-cannot-reset.html
EmBee_57573what I heard is that you have to set it to only one DC that should have the emulator role. You should not use a pool. that does not work.
please let me know if it helped.
nash_65851We had this same error on our BigIP's.
The F5 support guys couldn't quite figure out the reason (I think it is a bug), but we found that making sure each DC in the pool had a different priority group resolved the issue.
The problem that we saw was that the APM would get a ticket from one DC and would then try and access a different DC. When we set up separate priority groups, the communications would stay with the one DC.
darren_19980Is there a reason why Kerberos is used for authentication? Why not try AD Query? I used it on my access policy and it works fine. The users will be prompted to change their password when it has expired. You will have to define your User Primary GrpID as well, typically, it's 513.
- nash_65851
And that is the really strange thing that I gave up trying to understand. We are using the AD Query and haven't even set up any Kerberos for anything.
- Kevin_Stewart
AD auth\query use Kerberos on the back side. I have observed that the AD AAA uses DC pool members like a SNAT pool, where one address is exhausted before moving to the next one. If this causes a problem on the DC, the AAA will mark the IP as offline for several minutes. A "best practice" solution is to select Direct in the AD AAA configuration, add the Domain Name in uppercase, and the admin account/password. AD will natively load balance itself and return the best DC for the job in an SRV query response.
