We’re trying to implement OwnCloud in our environment, and I’ve developed a few iRules to handle some of the work , but we’ve run into an issue when trying to download files. The connection keeps getting reset and the file doesn’t come down. I think it gets part of the file (a little bit more each time you try), but continuously fails with the message http_process_state_prepend - Invalid action:0x109010.
my iRule disables the ACCESS profile (ACCESS::disable) when there is no currently valid session for certain url’s that don’t need authentication, and uses a Client Initiated forms SSO setup on the root path for the login page.
I know it’s not a lot of information, but I would appreciate any light that could be shined on this issue 
Can you show your iRule here?
Here’s the code:
when HTTP_REQUEST {
set DEBUG 2
set uri [HTTP::uri]
if {$DEBUG>1} { log local0. "" }
if {$DEBUG>1} { log local0. "----------------------------------------------" }
if {$DEBUG } { log local0. "Path: '[HTTP::path]'" }
if {$DEBUG>2} { log local0. "Request: [HTTP::request]" }
switch -glob [HTTP::header value "User-Agent"] {
"*mirall/*" -
"*csyncoC/*" -
"*neon/*" {
Disable APM if not already authenticated
if { not ( [HTTP::cookie exists MRHSession] && [ACCESS::policy result -sid [HTTP::cookie value MRHSession]] equals "allow" ) } {
ACCESS::disable
}
return
}
default {
Check the PATH for authentication exclusions
switch -glob [string tolower [HTTP::path]] {
"/remote.php/webdav/*" -
"*/status.php*" -
"*/public.php*" -
"*/core/img/*" -
"/index.php/apps/files_sharing/publicpreview.png" -
"*/*.css" -
"*/*.js" -
"/client/*" {
Disable APM if not already authenticated
if { not ( [HTTP::cookie exists MRHSession] && [ACCESS::policy result -sid [HTTP::cookie value MRHSession]] equals "allow" ) } {
ACCESS::disable
}
return
}
"/" {
Set the proper sso
WEBSSO::select [set foo /Common/CLIENT_INIT_FORMS_SSO]
return
}
default {}
}
}
}
}
I would probably recommend, if you haven’t already, adding additional logging throughout the code to see where the error is happening.
I did add some logging on the HTTP_RESPONSE, and logged the URI and HTTP status code, and the error message come up after that
: Status: 200 http_process_state_prepend - Invalid action:0x109010 (Server side: …)
I also tried adding logging for the actual payload data, which seemed to have what looked like correct data in it. Here’s the extra code I added:
when HTTP_RESPONSE {
if {$DEBUG} { log local0. "Uri: $uri" }
if {$DEBUG} { log local0. "Status: [HTTP::status]" }
Was returning here because collecting the payload messes up the response to the client (connection reset error).
return
Trigger collection for up to 1MB of data
if {[HTTP::header exists "Content-Length"] && [HTTP::header "Content-Length"] <= 1048576}{
set content_length [HTTP::header "Content-Length"]
} else {
set content_length 1048576
}
if {$DEBUG} { log local0. "Content-Length: $content_length" }
Check if $content_length is not set to 0
if { $content_length > 0} {
HTTP::collect $content_length
}
}
when HTTP_RESPONSE_DATA {
do stuff with the payload
set payload [HTTP::payload]
if {$DEBUG} { log local0. "Response Payload: '$payload'" }
}
Another thought from looking at similar error messages through devcentral and the SOL’s, it seems that non-compliant HTTP responses may cause the issue, and I’m wondering if that’s what’s going on here. A colleague was saying he tries downloading a 156MB file, and after 155MB, it failed, so he was left with a mostly complete file… This makes me think it might be a content-length mismatch or something that fails at the end of the download…
Is that possible?
Is that possible?
It is possible, but so far we’re not sure if this is an LTM or APM related issue. Can you test downloading without the access policy enabled?
You didn’t have an HTTP_RESPONSE event before, so I don’t think there’s any reason to believe the error was happening in the response. Can you add a bunch of logging inside the HTTP_REQUEST event, before and after different evaluations?
That’s true. I do get the error regardless of whether the HTTP_RESPONSE handler is in there.
For logging, I have logging on each step (each switch section, and before/after the commands throughout the iRule, and at beginning and end of HTTP_REQUEST. The error occurs after all that each time. And if I have the response in there, it occurs after the HTTP_RESPONSE completes (I also added an HTTP_RESPONSE_RELEASE handler and it occurs after that completes as well)
The URL that I’m going to in order to test this is one of those in which the ACCESS:DISABLE gets executed (a “/public.php” uri). It also happens when the access policy does not get disabled.
Given that, I’m inclined to think it’s likely LTM rather than APM.
Given that, I’m inclined to think it’s likely LTM rather than APM.
I would tend to agree. So any chance you can limit this to an LTM-only environment to test the download?
Just tried it going straight through the LTM without APM, and all things the same, it gave me the same result.
However, I decided to remove the HTTP profile completely from the VIP, I can download a file without seeing the error in the logs or getting an error prompt in the browser… Tried modifying different settings in the HTTP profile (which inherits from the default http profile with no changes), but can’t seem to figure out a way to get it working. If it’s on, it fails.
Okay, well then let’s do this. Add the following:
when CLIENT_ACCEPTED {
TCP::collect
}
when CLIENT_DATA {
log local0. [TCP::payload]
TCP::release
TCP::collect
}
Going on the assumption that something is attempting to talk through the VIP with a non-HTTP protocol, you should be able to see this in its raw form directly before the error. You’ll also need to disable client side SSL, if possible, for the payload to be readable.
I’m getting an error now when I try to access the site
TCL error: no connection established TCP::collect needs an established connection! (line 3) invoked from within "TCP::collect"
That seemed to work better. Response looks like this:
GET [[DOWNLOAD URI]] HTTP/1.1 Host: [[HOST]] User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate DNT: 1 Referer: [[REFERER]] Cookie: [[COOKIES]] X-ClickOnceSupport: ( .NET CLR 3.5.30729; .NET4.0E) Connection: keep-alive
Should I be seeing more data?
Forgot about the server side…
when CLIENT_ACCEPTED {
TCP::collect
}
when CLIENT_DATA {
log local0. [TCP::payload]
TCP::release
}
when SERVER_CONNECTED {
TCP::collect
}
when SERVER_DATA {
log local0. [TCP::payload]
TCP::release
}
The TCP::collect after the TCP::release should allow you to buffer more data. Not sure why that’s not working here. We’re looking for whatever happens right before the error.
Ok, I added that code, and tried to run it both with and without the HTTP profile associated to the VIP.
-
Without the HTTP profile, I see the request come though, and the SERVER_DATA gets the file (png image in this case) and sends back the payload without fail
-
With the HTTP profile, I see the same request and it looks like the data is being returned, but when I compare the two logs, there’s a large chunk of data missing from this one.
I modified the iRule to count the number of times SERVER_DATA and CLIENT_DATA were called, and found that when NOT using the HTTP profile, I got 435, but with the HTTP profile I only got 347.
I also tried using a simple text file for the download so I can look at the response from SERVER_DATA, and had a similar experience.
With HTTP profile, I got the “http_process_state_prepend - Invalid action:0x109010” error after 17 events.
Without HTTP profile, I got all 26 events and the file downloaded sucessfully.
Is this a bare bones HTTP profile? Do you have additional iRules logic applied? Do you have ASM or AAM (web accelerator) profiles applied?
Yea, the VIP is basic LTM. Removed APM and has no AAM profile and we don’t use ASM. Here’s my configs. The HTTP profile is just inherited from the default http profile. The rule is that logging rule and is the only one I’m using.
ltm virtual FILES_TEST {
destination 192.168.99.100:http
ip-protocol tcp
mask 255.255.255.255
pool FILES_POOL
profiles {
HTTP { }
tcp { }
}
rules {
AAAA_TCP_CHECKS
}
source 0.0.0.0/0
source-address-translation {
type automap
}
translate-port disabled
vs-index 336
}
ltm profile http HTTP {
app-service none
defaults-from http
proxy-type reverse
}
The thing is, all arrows seem to be pointing to the application. This error is generally triggered when the HTTP filter sees something that it doesn’t expect, like a Content-Length header that indicates a payload size smaller than the actual payload. Take a look at this SOL for more information on the error:
I understand that the conditions of the article probably won’t be exactly the same as your environment, but the idea should be the same. For this sort of thing you have at least two options:
-
Fire up WireShark, Fiddler, or HTTPWatch and capture traffic with and without the HTTP profile, then compare the two. Are the content lengths correct in the responses? Are there excessive HTTP 100 responses?
-
Open a support case - they’ll likely want to see a qkview and similar client side captures.
Thanks! I had read that article and wondered if the issue was similar to that. I’ll open a ticket. Thanks!