Forum Discussion
Why does XMLHTTPRequests from IE 10/11 fail intermittently against F5 LTM+APM?
We’re having problems with an application one of our dev teams is deploying.
F5 Big-IP v11.4.1 HF2
XMLHTTPRequests of various types (generally POSTs) from Internet Explorer (10 & 11) will fail intermittently but with good reproducability. When we watch the traffic on the wire, we see that IE sends the request headers, the VS sends an ACK followed immediately by an ACK/RST before the client sends the POST body.
The LTM VS in question is balancing a pool of two members, has an APM access policy attached to it for single sign-on, and is also assigned a rule controlling exemptions to the access policy (e.g. for public web services).
The IE XHR issue is definitely linked to the access policy somehow. When that policy is disabled in our test environment, the symptoms no longer present. When the VS is bypassed and the browser works directly with the application server, the symptoms no longer present.
The aforementioned iRule for exemptions follows this basic logic: if the URI requested matches this path, then ACCESS::disable. Yesterday we found that if we changed ACCESS::disable to WEBSSO::disable, that the IE XHR issue no longer presented. This made no sense to me as ACCESS::disable should encompass WEBSSO::disable. It turns out to be precluded as a workaround for us because most of this team’s endpoints need to be accessed by both a user’s browser (where an APM session can be expected to be active) and by other application servers (which will not have an APM session). Since WEBSSO::disable still requires that active APM session, it cannot be an effective workaround.
At this point, we are out of ideas. There’s very obviously something between Internet Explorer and our F5 Big-IP APM access policy that doesn’t play nice. We really need help figuring out what needs to change between the client-side JavaScript and the LTM/APM to get this working correctly.
7 Replies
- Kevin_Stewart
Employee
Could you clarify a few things?
we see that IE sends the request headers, the VS sends an ACK followed immediately by an ACK/RST before the client sends the POST body.
Does the XHR client send a single HTTP POST request? And if so do you see a full 3-way TCP handshake, and then the HTTP request headers? Is the POST payload not in that first HTTP request?
It turns out to be precluded as a workaround for us because most of this team’s endpoints need to be accessed by both a user’s browser (where an APM session can be expected to be active) and by other application servers (which will not have an APM session)
So you're saying some requests will be coming from agents that haven't authenticated through the access policy? When the XHR requests happen, do you always see an MRHSession cookie in that request? If XHR agents are accessing the APM VIP without an initial browser session, have you made allowances for those agents to handle the initial 302 redirect to /my.policy?
- IheartF5_45022
Nacreous
If you run tcpdump on your LTM using
tcpdump -i0.0:nnn -s0 -w /var/tmp/rst.cap host x.x.x.xwhere x.x.x.x is your vs address, then view rst.cap in Wireshark, you'll be able to see extra info on the RST packet that will tell you the cause of the RST.
- Eric_Kolb_27656
Nimbostratus
We've got a pcap from the endpoint perspective, but RSTs aren't annotated with reasons. Should a server-side pcap prove any different?
- Kevin_Stewart
Employee
A reset in a TCP capture isn't going to tell you a lot anyway. Because we're talking about layer 7 communications, you need to know what's happening at the application layer. You need to be able to see the HTTP requests, what's in those requests (URIs, headers, cookies, payload), what's in the responses, and who exactly is generating the error. The fact that WEBSSO::disable solves the problem almost indicates that the backend server is sending the error, so you need to be looking at both sides of the proxy.
Unless I've misunderstood, you've indicated that multiple client types access this APM virtual server: browser clients (that authenticate via APM), XHR agents inside the browser clients, and clients that are not browser-based and that do not authenticate. Is this correct? And if so, how do the third set of clients pass through the access policy? And where do the errors occur, and to whom?
- Kevin_Stewart
Employee
Okay, so a few more questions:
-
What is your SSO doing? What type of authentication?
-
Is it that the 3rd group doesn't need SSO either? How do they authenticate to the backend servers?
-
Can you look at the HTTP requests coming from the XHR agents and see if a) the MRHSession cookie exists when it fails, and b) if there's an Authorization header in the request?
-
- Kevin_Stewart
Employee
Do you think the fact that they're using JWT bearer tokens in the Authorization header might be playing foul with APM's expectations that Authorization should be used differently in an NTLM environment?
I'm suspecting that the Authorization header being sent by the client isn't being overwritten, or potentially is in the wrong way. The WEBSSO profile effectively inserts an NTLM Authorization header into the request stream on the way to the server. If the client is sending its own Authorization header, the WEBSSO may not be overwriting that value.
When we see it fail, it's worth noting that the request isn't technically completed. It's happening during a POST after the headers and before the POST body
Are the POST payloads very large in this case? If you watch the HTTP traffic on both sides of the F5, do you see the POST hitting the server and the server sending an error, or does it fail at the F5 without ever going to the server?
- Kevin_Stewart
Employee
If you can see both sides of the traffic, watch what the client sends, particularly the Authorization header, and see what shows up on the server side. I'm thinking that APM isn't overwriting the client's Authorization header.
Help guide the future of your DevCentral Community!
What tools do you use to collaborate? (1min - anonymous)Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
