01-Sep-2022 02:03
Hi there,
actually my ASM Policy is blocking a file upload for one application with the error message:
HTTP protocol compliance failed
Chunks number exceeds request chunks limit: 1000
I raised the chunks limit blindly from 1000 to 1500 with no success. Where I can see the actually number of chunks without capturing the traffic? After disabling the funktion "Unparsable request content" Upload went through without a problem. But from the notice I would stick this on?
Note that disabling this check can result in losing many enforcement features in the ASM.
Sametime I get the following syslogs:
ASM out of memory error: event code X242 Exceeded maximum memory assigned for XML/JSON processing
Cannot allocate 27415074 more bytes for XML parser. current memory size 837505174 (in bytes)
As you can see I raised the available memory for XML request from 450MB (default) to nearly the double.
01-Sep-2022 03:24
01-Sep-2022 03:28
Hi HarunT,
yep checked that already. It feels that the additinal required memory is raising over time. So I changed the value already to nearly the double and everytime the message appears it wants about 20-30mb more and raising.
02-Sep-2022 08:33
What is the size of the file in question?
02-Sep-2022 13:57
Two files with bearly under 10MB. XML Profile and every other setting limits I found are at least high enough or any.
06-Sep-2022 08:34
Very odd. The error indicates that it's trying to handle ~840 Meg. ASM logs show anything?
07-Sep-2022 01:04
Hi, yes seems a bit strange to me. No other logs than posted above. Since this error did not occure the last days, logs were pruned. Since we are still on the 14.x branche with this device, I consider switchting to 15.x ... maybe this gets solved magicaly
20-Sep-2022 04:11
Maybe anyone got ad idea?
It happens 2-3times a day and we are already at 1GB but keep getting
BD_XML|ERR |Sep 20 12:11:39.672|26884|xml_validation.cpp:0244|Cannot allocate 22474662 more bytes for XML parser. current memory size 1002457583 (in bytes)
21-Sep-2022 12:05
Hey @P_Kueppers - are you still seeing the same issue, or were you able to resolve it?
27-Sep-2022 07:43
Nope. Same issue every few days and still on 14.x but I cant imagine that this is a bug and will be fixed with 15.x
I think the messages doesnt come from the application I suspect rather a other one. How can I track this? Some fancy iRule to log this big XML requests? I can imagine that this is coming from a total different application where Im not informed about xml/soap requests.
27-Sep-2022 08:15
Tagging @AubreyKingF5 again for his technical expertise...
27-Sep-2022 13:48 - edited 28-Sep-2022 06:09
Thanks, hopefully he can help me out... maybe i need to deploy something like this on all vservers to find that black sheep
when HTTP_REQUEST {
#Check if the request is a POST, with a content type of text and size over 10MB
if {[HTTP::has_responded]} {
return
}
elseif {[HTTP::method] eq "POST"}{
if {[HTTP::header value "Content-Type"] contains "xml" or [HTTP::header value "Content-Type"] contains "json"}{
if {[HTTP::header value "Content-Length"] >= 10000000}{
log local0. "This is the HTTP Path: [HTTP::path]"
log local0. "Client [IP::client_addr] This is the HTTP Host [HTTP::host]"
log local0. "Client [IP::client_addr] accessed [virtual]"
log local0. "Query string of URI: [HTTP::uri] is [URI::query [HTTP::uri]]"
log local0. "HTTP Content Length Header = [HTTP::header value "content-length"]"
log local0. "HTTP Content Type Header = [HTTP::header value "content-type"]"
}
}
}
}
27-Sep-2022 16:46
Not a bad plan. Can you show me the tmsh out for the http profile? I take it you're not doing anything fancy like HTTP2?
28-Sep-2022 00:05
Pretty default I would say
We have some http/2 servers active:
ltm profile http2 Standard_http2_profile {
app-service none
concurrent-streams-per-connection 100
connection-idle-timeout 60
defaults-from http2
}
ltm profile http2 http2 {
activation-modes { alpn }
app-service none
concurrent-streams-per-connection 10
connection-idle-timeout 300
enforce-tls-requirements enabled
frame-size 2048
header-table-size 4096
include-content-length disabled
insert-header disabled
insert-header-name X-HTTP2
receive-window 32
write-size 16384
}
But most is http/1.2
ltm profile http SecureWEB_http {
app-service none
defaults-from http
enforcement {
known-methods { CONNECT DELETE GET HEAD LOCK OPTIONS POST PROPFIND PUT UNLOCK }
}
header-insert X-Forwarded-Proto:https
hsts {
maximum-age 31536000
mode enabled
}
insert-xforwarded-for enabled
proxy-type reverse
redirect-rewrite all
server-agent-name LB
}
ltm profile http http {
accept-xff disabled
app-service none
basic-auth-realm none
encrypt-cookies none
enforcement {
known-methods { CONNECT DELETE GET HEAD LOCK OPTIONS POST PROPFIND PUT TRACE UNLOCK }
max-header-count 64
max-header-size 32768
max-requests 0
pipeline allow
truncated-redirects disabled
unknown-method allow
}
fallback-host none
fallback-status-codes none
header-erase none
header-insert none
hsts {
include-subdomains enabled
maximum-age 16070400
mode disabled
preload disabled
}
insert-xforwarded-for disabled
lws-separator none
lws-width 80
oneconnect-status-reuse "200 206"
oneconnect-transformations enabled
proxy-type reverse
redirect-rewrite none
request-chunking preserve
response-chunking selective
response-headers-permitted none
server-agent-name LB
sflow {
poll-interval 0
poll-interval-global no
sampling-rate 0
sampling-rate-global no
}
via-request preserve
via-response preserve
xff-alternative-names none
}
30-Sep-2022 13:11
Okay I maybe give up. Should I raise a support ticket? I put the iRule Logging on every vserver I think of such uploads but there were only one match in the near of a event log but that cant be a problem:
Sep 30 14:59:22 .dmz.local info tmm1[21243]: Rule /Common/URL_Logging <HTTP_REQUEST>: HTTP Content Length Header = 11019932
Sep 30 14:59:22 .dmz.local info tmm1[21243]: Rule /Common/URL_Logging <HTTP_REQUEST>: HTTP Content Type Header = application/soap+xml; charset=utf-8
BD_XML|ERR |Sep 30 15:03:51.849|26888|xml_validation.cpp:0244|Cannot allocate 19503077 more bytes for XML parser. current memory size 1008875463 (in bytes)
And 4 min later;
Broadcast message from systemd-journald@.dmz.local (Fri 2022-09-30 15:03:52 CEST):
perl[26640]: 01310003:0: ASM out of memory error: event code X242 Exceeded maximum memory assigned for XML/JSON processing
2022 Sep 30 15:03:52 .dmz.local perl[26640]: 01310003:0: ASM out of memory error: event code X242 Exceeded maximum memory assigned for XML/JSON processing