on 12-May-2015 06:16
Introduction
There have been a ton of requests on the boards for a simplified client side NTLM configuration, so based on Michael Koyfman’s excellent Leveraging BIG-IP APM for seamless client NTLM Authentication, I’ve put together this article to show the very basic requirements for setting up APM client side NTLM authentication.
So why APM client side NTLM? Like Kerberos, NTLM can provide seamless (i.e.. transparent) logon for local clients. It’s arguably not as secure as Kerberos as authentication protocols go, but if you’ve ever worked with Kerberos, you’ll likely appreciate that NTLM is simpler and a bit more flexible. APM’s client side NTLM authentication is also considerably different than the other client side methods that generally include visual policy authentication agents and a AAA configuration. This difference allows client side NTLM to be enabled and disabled per request as needed by Microsoft Exchange and Secure Web Gateway access features. APM’s client side NTLM is controlled by a feature called ECA, or External Client Authentication. Let’s jump right in and look at how ECA is used to configure NTLM. There have been some changes to the way client side NTLM functions between APM versions, so this article will focus on 11.6.
Configuration
Step 1: Create a virtual server
It may seem counter-intuitive to create the virtual server first, but you’ll need it in step 5 below. Nothing fancy here, just create a standard virtual server and assign an HTTP profile and pool. Of course if you’re offloading SSL (which you should be) also include a client SSL profile.
Step 2: Configure system DNS
APM needs to be able to resolve the Active Directory domain controller(s), so you’ll need to specify the DNS for the domain. In the BIG-IP management GUI, navigate to System -> Configuration -> Device -> DNS. In the DNS Lookup Server List section, add the IP address of any AD DNS server(s).
You can do the same in the TM shell as follows:
tmsh modify sys dns name-servers replace-all-with { [IP address of DC DNS server] }
Step 3: Create an Active Directory NTLM machine account
This step creates a computer account in the Active Directory that APM will use to validate the NTLM tokens from clients. This machine account will be used in a pass-through authentication capacity. Please see the following Microsoft article for more information on NTLM Pass-Through Authentication. In the BIG-IP management GUI, navigate to Access Policy -> Access Profiles -> NTLM -> Machine Account. Click the Create button. At a minimum you need an arbitrary profile object name, the name of the computer account to create (it must not already exists), the domain FQDN (the DCs will be resolved via DNS SRV queries), and the credentials of a user with privileges to create computer accounts. To specify a single domain controller, enter that DC’s FQDN into the Domain Controller FQDN field. Click the Join button to proceed.
You can do the same in the TM shell as follows:
tmsh create apm ntlm machine-account [profile object name] domain-fqdn [domain FQDN] machine-account-name [new machine account name] administrator-name [admin name] administrator-password ["admin password"]
If the command is successful you’ll see a new computer account in the Active Directory.
Step 4: Create an NTLM Auth Configuration
Now let’s add that computer account object to an NTLM Auth configuration. In the BIG-IP management GUI, navigate to Access Policy -> Access Profiles -> NTLM -> NTLM Auth Configuration. Click the Create button. Give it an arbitrary object profile name and specify the previously-created machine account name. Add the FQDN for a domain controller to the Domain Controller FQDN List field.
Note: You must add at least one domain controller here. If you don’t specify anything, users will be allowed access without any NTLM token validation. In some APM versions the Domain Controller FQDN List field displays as mandatory and in others optional. In all cases it is absolutely required.
Click the finished button to proceed. You can do the same in the TM shell as follows:
tmsh create apm ntlm ntlm-auth [profile object name] machine-account-name [machine account object name]
Step 5: Enable the ECA profile for a specific virtual server
Unlike the other APM client side authentication methods, there’s no GUI option to enable APM client side NTLM. To do that you need to apply the ECA profile to a virtual via the TM shell.
tmsh modify ltm virtual [virtual server name] profiles add { eca }
Step 6: Create an iRule to enable client side NTLM
This is where all of the magic happens. Create this very simple iRule and apply it to your virtual server:
when HTTP_REQUEST { ECA::enable ECA::select select_ntlm:/Common/my-ntlm-machine-auth }
That’s it. You need to enable ECA and then specify the name of the NTLM Auth configuration object (created in step 4 above). I would also point out here that client side NTLM authentication is a bit different from Kerberos in that ECA is generally going to issue a 401 Unauthorized NTLM challenge on every new request. If this proves to add too much overhead, the following modification to the above iRule will allow NTLM to be processed once at the beginning of the session. The APM session cookie is used thereafter to maintain the session.
when HTTP_REQUEST { if { [ACCESS::session data get session.ntlm.last.result] eq 1 } { ECA::disable } else { ECA::enable ECA::select select_ntlm:/Common/my-ntlm-machine-auth } }
Step 7: Create an access profile
Create a standard access profile and configure the visual policy like this:
There’s nothing to configure inside the NTLM Auth Result agent so it’s just there to validate the returned session.ntlm.last.result session variable. Add this access profile to the virtual server.
Step 8: Modify the client browser to support NTLM authentication
By default modern browsers do not, for obvious security reasons, send credentials to any web site that asks for them, so you have to explicitly define what sites the browser can send credentials to. For Microsoft Internet Explorer that’s as simple as adding the site to the Local Intranet Sites list. In Internet Explorer, go to Tools -> Internet Options. Open the Security tab, select Local intranet and then click the Sites button. Now click the Advanced button. Add the APM virtual server’s URL here. You can specify the exact URL here (ex. https://ntlm-test.domain.com) or a wildcard (ex. *.domain.com) to cover everything under a given domain.
This should also cover Chrome and Opera running in Windows. For Firefox, navigate to the config URL at about:config and type “trusted” in the Search field. You’ll see a short list of keys with the word “trusted” in them. Double click the “network.automatic-ntlm-auth.trusted-uris” key and add either the full FQDN of a specific APM virtual server (ex. ntlm-test.domain.com) or for an entire domain just the domain component itself (ex. .domain.com).
Testing and Troubleshooting
With your browser(s) configured, go ahead and test. If you get prompted for username and password, NTLM authentication has failed. Probably one of the best first things to do is to open a console (SSH) shell and tail the APM and LTM logs. In many cases the error will be screaming at you from one of these logs.
# cd /var/log # tail –f ltm apm
Here are some possible causes for client side NTLM authentication to fail:
Let’s now see what that client side NTLM authentication looks like on the wire. For this part I’m going to fire up WireShark and filter on “ntlmssp”
The ECA profile is responsible for generating the 401 Unauthorized response to the client’s initial request. In that 401 response is the WWW-Authenticate header and NTLM challenge. If there’s something wrong you may also see error messages manifest in the Wireshark captures.
Cross-domain Considerations
Client side APM NTLM authentication natively works across domains if the domains are in a forest trust, external two-way trust, or external one-way outgoing trust. For non-trusting domains, since the NTLM Auth config profile is assigned first, there’s no native way to switch between profiles based on something in the client’s NTLM challenge response. If you can identify the clients by some other characteristic, a unique IP subnet for example, that might be an option.
Expanding Possibilities
Now that we have APM client side NTLM humming along, let’s look at some other things you can do with it. A successful client side NTLM authentication will produce a few interesting session values:
These session variables can be used and evaluated inside the visual policy and in iRules. For example, let’s say you want to do an AD query with the username to see if the user is in a specific AD group:
The ECA profile also creates three other values that can be used in iRules:
Since the ECA profile is also enabled at will, there are some other things you can do with it. One example would be to enable and disable client side NTLM based on the requested URI:
when HTTP_REQUEST { if { [HTTP::uri] starts_with “/protected_uri” } { ECA::enable ECA::select select_ntlm:/Common/my-ntlm-machine-auth } else { ECA::disable } }
And finally you have two ECA events, ECA_REQUEST_ALLOWED and ECA_REQUEST_DENIED that can be used to trigger specific actions on successful authentication or denied logon attempts, respectively. Well that’s it for now. If you have any questions please post them below and I’ll try to answer. Otherwise stay tuned for more BIG-IP APM articles.
Thanks.
Hi, I am trying to do an AD Query after an NTLM Auth Result, as described above but it does not work and I get the following error:
AD module: query with '(sAMAccountName=myusername)' failed: empty password detected (-1)
What am I missing? I don't want to use AD Auth because the auth is NTLM only
Thanks
NTLM is a challenge-response protocol in which the client proves it knows the password by encrypting something with it. The client doesn't actually send the password to the server (APM in this case), so indeed APM wouldn't have the user's password to use for other things like an AD query. In this case you'd have to use a static "service" account to perform AD or LDAP queries.
is still still valid in 11.6/12.x ? Is ECA still necessary ?
With FF, i get a popup for auth. Can't get rid of it. If the username is wrong, it won't continue to the APM policy => can't do logon page fallback.
[Common] 10.10.10.10:53884 Authentication with configuration (/Common/NTLM-Auth-Conf-acces-ts-ced-RDG) result: MUUSERNAME@MYDOMAIN (MYPCHOSTNAME): Fail (STATUS_PASSWORD_MUST_CHANGE)
Any idea to trap a password change and let the user access when it's append?
401-based auth methods like NTLM do not support password change functionality. I am pretty certain that any product that's using NTLM auth cannot allow user's access if their AD flag is set to MUST_CHANGE_PASSWORD.
From my point of view, when facing this, the user/pass is correct, but the user only need to change is password. If user access the backend RDP server, then the user will be force to change it. Michael, do you have any idea how I can let a user access when facing STATUS_PASSWORD_MUST_CHANGE? Probably an iRules.
Hi Kevin,
I tried this setup in my Lab but i am really struggling with APM to respond with a 401 Unauthorized response. ECA profile enabled in Vs Machine account working correctly irule gets triggered but whenever i try logging an ECA status i get TCL error which confirms that APM is not generating a 401 unauth response.
TCL error: /Common/NTLM-AUTH-ECA-IRULE - plugin_tcl_command_execute: Error sending plugin message. invoked from within "ECA::status"
Checked browser settings (IE), machine is domain joined and i logged in using domain user etc Just running out of ideas. I just get Your session could not be established and in APM logs i see entries of:
Msg: variable "session.ntlm.last.result" was not found in the local cache for session "e5f4c05e"
Any tip will be greatly appreciated. I can share full config if that helps, please let me know. Regards, Ali
OK Just started seeing below TCL error,
Apr 6 16:12:57 BigIp-01 err tmm[16353]: 01220001:3: TCL error: /Common/NTLM-AUTH-ECA-IRULE - Operation not supported (line 1) invoked from within "ECA::select select_ntlm:/Common/NTLM_Auth_Config"
Apr 6 16:12:57 BigIp-01 err tmm1[16353]: 01220001:3: TCL error: /Common/NTLM-AUTH-ECA-IRULE - Operation not supported (line 2) invoked from within "ECA::select select_ntlm:/Common/NTLM_Auth_Config"
Well!! I had to upgrade to 11.6 HF8 in the end from 11.6.0 Base version. All working with following iRule.
when HTTP_REQUEST { if { [ACCESS::session data get session.ntlm.last.result] eq 1 } { ECA::disable } else { ECA::enable ECA::select select_ntlm:/Common/NTLM_Auth_Config } }
If i dont check for session.ntlm.last.result APM keeps generating 401 and the pop-up.
Thanks to the contributor.
What are you looking for with explict forward proxy? If you're looking to do NTLM auth to explicit proxy, then it's already built into the product: https://support.f5.com/kb/en-us/products/big-ip_apm/manuals/product/apm-secure-web-gateway-implement...
I dont have SWG licensed, the F5 will be a ssl forward proxy and decrypt https traffic and sends it through a Bluecoat device, the Authentication of the client will be done by F5 and credentials passed to Bluecoat via HTTP header. So will the above mentioned solution work for this setup?
It's a bit misleading, but you don't need to have SWG licensed in order for NTLM-authenticated Explicit proxy to work - that can be done just via APM. You'll need to insert a X-User header in the request to indicate username to Bluecoat upstream via Per-Request Policy VPE agent.
Still select SWG-Explicit. As long as you have APM licensed, all of the features except URL Filtering database and Analytics scanning will work.
But it mentions the same steps under the 11.6.1 doc https://support.f5.com/kb/en-us/products/big-ip_apm/manuals/product/apm-secure-web-gateway-implement...
Right. Steps are the same, but they are available to you in 12.1 or higher without SWG license
If this is used with SAML/SSO I found that the iRule code to conditionally disable ECA resulting in the SSO failing. The user gets an error page back when APM stops processing the request. The error in the log is:
err tmm1[19392]: 014d0002:3: 25408ee6: SSOv2 POST Authn Request has no body
err tmm1[19392]: 014d0002:3: 25408ee6: SSOv2 Error(12) Extracting SAML Data from Request
Hello everyone, first of all thanks to Kevin and Michael for the post and the continuous support. I've tried this setup on my lab and works perfectly fine after applying the iRule. My only question here is, Is there any way to present the F5 Logon Page form for those users whose computer is not joined to domain after they fail the NTLM auth? At this moment the user is presented with a Basic authentication if they are using a non-domain joined, although if i put my domain credential on it, it does nor shows the webtop but an error (unreachable site). If i remove the irule, the NTLM does not kick in for domain computers and all the users are presented with the Form Logon.
My Access flow is the following: Start>>NTLM auth 1 Success--->Resource assignment //// 2 Fallback ------>Logon>AD Auth> Resource assigment.
Is this possible or NTLM automatically discard the idea of working with non-domain computers? I saw before with Exchange this approach and work pretty well.
Thanks
Javier, the issue is not with APM, but with the way modern browsers work. Think about the protocol flow:
When "integrated authentication" is involved, a browser will attempt to provide authentication, usually starting with the strongest method (as defined in the server's Authenticate header), and then work down from there, ending with Basic. In the span of time between the server's 401 Authenticate response, and the browser prompting you for Basic credentials (because it can't send an NTLM token or Kerberos ticket), there's no communication with the server. There's no message that gets sent to the server between these events, so no opportunity for the server to respond with a different type of authentication request.
Most admins will insert an IP or URL check into the flow before deciding to issue the 401. That's what the referenced iRule is doing.
As long as you build the VPE(based on my article) to detect a domain-joined machine and steer policy accordingly to NTLM or forms-based authentication, it should work. The key is how to identify domain-joined machine beforehand. You can use machine cert check or windows machine check to determine domain membership.
But to be clear, the principal is the same. You need to know in advance that the client can present integrated credentials. You can do that with a simple IP subnet check, or look at machine certs, windows machine domain information (via the registry), etc.
Hi gents,
Thanks for the info, it was really useful. My doubt was if there is any way to enforce the "Negociate" authentication that happens on OWA for example. If the user cannot work/authenticate with NTLM use either Basic or OWA Form. I understood the "limitation" (if it can be call that) of NTLM not been able to auth non corporate users, but i was expecting the fallback path, which contains Logon agent, to be followed. Although since the irule always enforce NTLM the logon will not work, so i will follow Michael's approach of steering them. In my case i will ask the internal users to use a slightly different fqdn and use LTM policy to control the access to different internal VS with different Access policies. The idea of installing EDGE components only to review the registry requires too much effort to my customer.
As always, i appreciate your amazing support and happy new year - soon :)-
Hi everyone, -So, after reviewing we will go with pre-check before ntlm. I am interested in the option suggested by Michael above, about reviewing the windows registry, checking if the machine is join to a corporate domain and if so NTLM, else Logon Page. i am going to assume that i will use/install Edge components for this and then "Windows Registry"-Client Side-, "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon"."DefaultDomainName"="mydomain".
Now, since i cannot apply from the very beginning Michael's irule (it will enforce NTLm right away) i will need to instruct the variable when it should be triggered ( APM irule Event -ntlmon- right after domain check successful). So, the irule should look like the one below and since the non corporate users will not pass the windows registry they will follow the fallback path to logon page. My first concern is that i've never created an event with APM and i do not know if the device will accept the substitution of http request with "ntlmon"
when ntlmon {
ECA::enable
ECA::select select_ntlm:/Common/NTLM-auth }
So, my question is, is this feasible, does it make sense?. of course i will try on my environment but it will take some time and if somebody tried before and worked perfect i will keep pushing, else, i will appreciate a plain "no".
Again, thank you so much for the info
ntlmon isn't a thing.
I really have to ask why F5 has not taken this up to fully support this. It seems we are all left to invent a solution and most of us are struggling (at least I continue to struggle with it). Maybe an iApp? At least create an official implementation guide showing examples and detailing the specifics.. Example is the 'elusive' ECA, which cannot be configured via the GUI, it must be done via CLI for the virtual server.
I have struggled with this for a couple if years and am finally on version 12 and have it somewhat working. I am getting errors in my logs stating "01480001:4: No held transaction to sink." that nobody can seem to explain. I'm guessing it is dropping the message. For Chrome users it makes it unusable-- something is wrong. For IE users it seems to work for them.
My users are finally happy to be able to use SAML/SSO with NTLM without having it prompt the user for credentials if it can validate what they already have. But there are loose ends that are lingering on and need to be cleaned up. The problem is what exactly is wrong and what needs to be done to fix it.
Thanks to everyone who has contributed to this and other posting regarding NTLM authentication. I am grateful and appreciate the time you have taken to share and the time you probably have spent making it work in your own environments.
Brad, this is just from my own personal experience, but it seems to me from most previous posts on the matter, that there's a general misconception about how browsers perform Windows integration authentication. The issue that Javier was trying to solve was just that, the ability to do NTLM for one group of users and logon page for another. But browsers don't work that way, and there's nothing a server (not an F5, not Apache, not NGINX) can do about this. Once the browser gets the Authorization with Negotiate header, the server has to just wait for a response. That's why you use things like registry checks (for domain membership), or IP subnet matches to preemptively filter users into different authentication schemes. ECA is needed only when you have to enable/disable NTLM dynamically. As for struggles with NTLM in general, I don't think F5 admins/users are alone here. Windows integrated authentication has been a pain to configure for as long as I can remember.
Good morning everyone,
Kevin, hi. I've tried what you suggested and even if it makes sense it does not work as expected (more than probably because even after all I've learn about NTLM thanks to this post i do not completely understand how it works). The irule used is below.
So, i will explain what i did/tried and what i suspect is failing. I have one policy that checks the Registry looking for the specific domain and if so the next step will be an Irule Event in VPE, "ntlm_on" (Now it does exists 🙂 ) After that I created an irule that starts with http_request -> "ECA disable" and if one specific APM session variable, "session.ntlm.required", exists and the content equal "yes", then ECA enable, else keep it disabled.
Besides that the same irule has ACCESS_POLICY_AGENT_EVENT that checks for ntlm_on. If it detects on it will set session.ntlm.required = yes and technically will start ECA thanks to HTTP_Request section above...It does not. Reviewing the logs i can see that the ntlm_on event is detected and the irule set the session.ntlm.required to yes. I believe that since is the same HTTP transaction and since NTLM technically happens at LTM level, not APM, it cannot be triggered again for the same request, therefore it may not be able to be activated in the middle of the policy verification. I am wrong? By the way, NTLM is working fine on my lab. Tested with the same computer and http_request ECA::Enable irule.
when HTTP_REQUEST {
ECA::disable
set ntlm_required no
if the user is a corporate user the policy will require NTLM auth. session.ntlm.required is set below
set ntlm_user [ACCESS::session data get "session.ntlm.required"]
if { $ntlm_user == "yes" } {
log local0. "NTLM_Enabled"
ECA::enable
ECA::select select_ntlm:/Common/NTLM_Auth
}
elseif { $ntlm_user == "no" } {
log local0. "NTLM_Disabled"
ECA::disable
}
}
when ACCESS_POLICY_AGENT_EVENT
{
if NTLM is required by the policy a new session variable will be created and used in the HTTP_Request event
if { [ACCESS::policy agent_id] eq "ntlm_on" }
{
log local0. "ntlm_required yes set"
ACCESS::session data set session.ntlm.required "yes"
}
elseif { [ACCESS::policy agent_id] eq "ntlm_off" }
{
log local0. "ntlm_required off set"
ACCESS::session data set session.ntlm.required "no"
}
}
After some additional testing, I believe that a Windows Registry check would require something closer to what Michael Koyfman has described. So for clarity, it basically boils down to something like this:
Simplified iRule:
when RULE_INIT {
set static::ntlm_config "/Common/F5NTLM"
}
when ACCESS_SESSION_STARTED {
ACCESS::session data set "session.ntlm.last.retries" 0
}
when HTTP_REQUEST {
switch -glob -- [string tolower [HTTP::uri]] {
"/ntlm/auth" {
if { [HTTP::cookie value MRHSession] ne "" } {
set sid [HTTP::cookie value MRHSession]
}
catch {
set sid [ACCESS::session sid]
}
set referer [HTTP::header value Referer]
set x_session_id [HTTP::header value X-Session-Id]
if { [string length $x_session_id] != 0 } {
set sid $x_session_id
}
set retries [ACCESS::session data get -sid $sid "session.ntlm.last.retries"]
set auth_result [ACCESS::session data get -sid $sid "session.ntlm.last.result"]
if { ($auth_result == 1) || (($retries == 2) && ($auth_result != 1)) } {
ECA::disable
HTTP::redirect $referer
} else {
ECA::enable
ECA::select select_ntlm:$static::ntlm_config
}
unset x_session_id
unset referer
}
default {
ECA::disable
}
}
}
when ECA_REQUEST_ALLOWED {
ACCESS::session data set session.ntlm.last.username "[ECA::username]"
ACCESS::session data set session.ntlm.last.domainname "[ECA::domainname]"
ACCESS::session data set session.ntlm.last.machinename "[ECA::client_machine_name]"
ACCESS::session data set session.ntlm.last.status "[ECA::status]"
ACCESS::session data set session.ntlm.last.result 1
ACCESS::disable
HTTP::header insert X-Session-Id $sid
use virtual [ virtual name ]
}
The set of ACCESS::session set commands in the ECA_REQUEST_ALLOWED event are not expressly required, but useful if you need to access NTLM user information from elsewhere. So basically, once the Windows Registry check determines this is a domain client, control is passed to an external logon page that redirects to itself at the /ntlm/auth URI. The HTTP_REQUEST event in the iRule is triggered and enables/disables ECA based on where you are in the policy evaluation. Once NTLM/ECA succeeds, the ECA_ALLOWED_REQUEST event disables access evaluation and injects the X-Session-ID header.
It's worth noting that if you simply performed an IP subnet check in an iRule, you wouldn't need to do all of this.
Hi Kevin, I followed the procedure described above, and at the end when trying to open an HTTPS website, the Web Browser doesn't go beyond the logon page [captive portal], and the Website saying 'page can't be displayed', with the URL redirected to:
https://f5proxylogon.mydomain.com/F5Networks-SSO-Req?SSO_ORIG_URI=aHR0cDovL3d3dy5nb29nbGUuY28udWsv
The APM logs are being flooded by the following two messages:
Apr 24 15:29:49 Proxy-F5 err eca[5939]: 0162000e:3: Invalid metadata (select_ntlm:)
Apr 24 15:29:49 Proxy-F5 err eca[5939]: 0162000e:3: Invalid argument ()
The messages didn't stopped until I deleted the NTLM access profile completely from the APM [and the iRule as well]
Any idea why?
You should have one SWG-Transparent access policy with the NTLM auth and captive portal URL defined. This access policy should be applied to both VIPs (transparent proxy and captive portal).
Yes, there is one single and common APM policy [SWG Transparent] applied to all of the three VS: HTTP [port 80], HTTPS [port 443] & Captive portal [port 443].
This policy has got the NTLM Authentication config [step 4] and the Captive Portal URL embedded on it, and the flow is as on step 7.
You know what was at the end, even deleting the iRule for NTLM and the NTLM policy the two messages mentioned above were flooding the logs for APM.
What needs to be done is to delete the {eca} profile added on step 5 from that particular Virtual Server, and the logs disappear, bringing things back to normal:
tmsh modify ltm virtual [virtual server name] profiles delete { eca }