Pending rule event HTTP_REQUEST aborted

spark has explained it in the past discussion topic.

 

 

Limit Client Connections with Table

 

https://devcentral.f5.com/community/group/asg/50/aft/2156882/showtab/groupforums

The reference above related to suspending commands in events other than HTTP_REQUEST, but the gist is the same - if any event with a suspending command in it is being executed at the same time that a RST is received on the TCP connection, you get the a message logged. It seems that sometime browsers randomly send a request, and then immediately send a RST. If HTTP_REQUEST has suspending commands in it (such as table or after), and it's still being executed when RST received you get the Pending message.

I noted a few browsers doing this - not just limited to 1 - a few versions of IE and also Firefox (I didn't check many). I asked how to supress the message as it was causing wailing and gnashing of teeth, and got the following response from F5 support;

I had a look at your iRule, and the 2 commands that suspend are "after" and "table". However, there are several instances of them both, and it appears more appropriate to filter this out via Syslog instead - especially since you already have filters in there. 

I tested with a quick iRule

 tmsh list ltm rule logme
ltm rule logme {
    when HTTP_REQUEST {
        log local0. "Pending rule event HTTP_REQUEST aborted for 204.184.155.187:56082->101.167.164.43:80 (listener: vs_mcms_www.skynews.com.au_http)"
        log local0. "Another line"
    }
}

and masked it successfully with a filter as such.

syslog include "filter f_local0 {
    facility(local0) and not match (\"Pending rule event HTTP_REQUEST aborted for\"); };"

So this should also work for you. Please do let me know if otherwise. 

Note, you cannot edit /etc/syslog-ng/syslog-ng.conf manually - and you shouldn't either since it will get overwritten especially with upgrades. 

Instead, we're going to modify your base config which will in turn include the changes in syslog's config as well.

I noticed you already have custom filters, so you just need to add the new one to your bigip_base.conf. If you do it via `bpsh` you'll overwrite rather than append.



The object in question in bigip_base.conf is 

syslog {
   include "
filter f_local6_httpd_ssl_acc {
   facility(local6)
   and match(\"\\[ssl_acc\\]\") and not match(\"\\] 10.0.0.\"); }; filter f_local6_httpd_ssl_req {
   facility(local6)
   and match(\"\\[ssl_req\\]\") and not match(\"\\] 10.0.0.\"); }; filter f_remote_loghost {
level(info..emerg)
and not (match(\"10.0.0.\") and level(info)); }; destination d_remote_loghost { udp(\"172.18.164.134\" port(514)); udp(\"172.18.164.18\" port(514)); }; log { source(local); filter(f_remote_loghost); destination(d_remote_loghost); }; "
}



1) Create a .ucs archive just to be on the safe side.


2) Edit your bigip_base.conf, and replace the above syslog object with this one

syslog {
   include "
filter f_local6_httpd_ssl_acc {
   facility(local6)
   and match(\"\\[ssl_acc\\]\") and not match(\"\\] 10.0.0.\"); }; filter f_local6_httpd_ssl_req {
   facility(local6)
   and match(\"\\[ssl_req\\]\") and not match(\"\\] 10.0.0.\"); }; filter f_remote_loghost {
level(info..emerg)
and not (match(\"10.0.0.\") and level(info)); }; destination d_remote_loghost { udp(\"172.18.164.134\" port(514)); udp(\"172.18.164.18\" port(514)); }; log { source(local); filter(f_remote_loghost); destination(d_remote_loghost); }; filter f_local0 {
    facility(local0) and not match (\"Pending rule event HTTP_REQUEST aborted for\"); }; "
}


Notice it's the same but just the addition of the iRule filter on the bottom. Save and quit.

3) Verify the data that will be included in the syslog-ng.conf file

     bigpipe syslog include

        SYSLOG - Include Data:

        filter f_local6_httpd_ssl_acc {
           facility(local6)
           and match("\[ssl_acc\]") and not match("\] 10.0.0."); };
        filter f_local6_httpd_ssl_req {
           facility(local6)
           and match("\[ssl_req\]") and not match("\] 10.0.0."); };
        filter f_remote_loghost {
        level(info..emerg)
        and not (match("10.0.0.") and level(info));
        };
        destination d_remote_loghost {
        udp("172.18.164.134" port(514));
        udp("172.18.164.18" port(514));
        };
        log {
        source(local);
        filter(f_remote_loghost);
        destination(d_remote_loghost);
        };
        filter f_local0 {
            facility(local0) and not match ("Pending rule event HTTP_REQUEST aborted for");
        };

4) Load the configs from disk to mem, and then restart syslog

     bigpipe load
     bigstart restart syslog-ng


When done, please tail the logs again to verify that the aborts are no longer showing. 

You may also inject a test as such,

     logger -p local0.info "Pending rule event HTTP_REQUEST aborted for 204.184.155.187:56082->101.167.164.43:80 (listener: vs_mcms_www.skynews.com.au_http)"

    Compared to, say, a typo the filter will not catch

     logger -p local0.info "Pending rule eventttt HTTP_REQUEST aborted for 204.184.155.187:56082->101.167.164.43:80 (listener: vs_mcms_www.skynews.com.au_http)"

Please let me know if that does the trick for you. 

I will be implementing shortly and will let you know if any issues.

Hi, forgot to add my notes from troubleshooting the same log messages few months back. Myself and the customer managed simulate the behaviour, aborts were caused by http pause. When connection was reset, it caused browser to double send the data, in our case 2 items got added to shopping basket.

 

You might not be experiencing the same issue, but it's worth taking a look if web code contains any arguments that would pause the conversation while waiting for user/back end server input.

 

What do you mean "aborts were caused by http pause"?

 

All the instance I managed to capture, the RST was sent by the browser microseconds after the request and the request never got sent to the backend. The RST was not caused by anything serverside. There must be a few different causes of the message.

 

Apologies, it's halt/pause/sleep/wait option form within html and not http. The request to halt connection before continuing can come from any of the scripts used by site. In my case; we replicated issue by creating html file with php "sleep()" switch before resuming communication to load further content. This was performed by customer, I am not a web developer and don't know enough to give you exact details. The information was provided by customer while troubleshooting.

 

As for RST; please do bear in mind that a stateful device forges a RST packet to clear state table for inactive session. Further more, capture performed on the F5, client or server? Can you also see RST on the server? Can you see RST on the client?