Forum Discussion

MR_RJ's avatar
MR_RJ
Icon for Cirrus rankCirrus
Nov 21, 2011

Logging when there are multiple threads from one host

Hi,

 

 

I got a question from the development team that I doubt I can solve with a LTM but why not ask here, maybe anyone got a idea if it's possible at all.

 

 

We have a application (web service) on port 443. The SSL is terminated at the LTM/BIGIP (v10.2.x).

 

They wonder if I can somehow log if there is multiple threads from a single source IP.

 

Not sure how this is possible, how do i count threads? is it equal to two simultaneous connections from one single IP? or is it possible to have several threads within 1 current/open connection?

 

If I should log two or more simultaneous connections from a single IP I guess it's possible with a iRule, check if the source/client IP is in some table and then log it to syslog.

 

 

But... still not sure if that will do. Anyone in here that got similar situations that you have solved?

 

 

//Robert

 

application delivery
dev
devops
iRules
  • hooleylist's avatar
    hooleylist
    Icon for Cirrostratus rankCirrostratus
    Nov 21, 2011
    Hi Robert,

     

     

    It would be relatively simple to track when a client reuses a TCP connection for multiple HTTP requests using HTTP::request_num > 1:

     

     

    http://devcentral.f5.com/wiki/iRules.http__request_num.ashx

     

     

    If you wanted to track when clients open more than one TCP connection concurrently, you could increment a subtable with a key of the client IP and then decrement it in CLIENT_CLOSED.

     

     

    But I'm curious as to why you want to do this. Is it for custom clients? All current browsers open more than one TCP connection when loading the elements of a web page. And all current browsers reuse TCP connections for multiple requests. So if you did implement this type of logging, it would be triggered constantly.

     

     

    Can you elaborate on why you want to do this?

     

     

    Aaron
  • MR_RJ's avatar
    MR_RJ
    Icon for Cirrus rankCirrus
    Dec 01, 2011
    Hi Hoolio,

     

     

    Thanks for the reply. Didnt check the subscripte checkbox to didnt notice the answer until now :)

     

    The background for the need of this is that we have web services that got a large range of customers/systems (not end users as internet explorer etc).

     

    We had a bit of a issue not long ago when one of the major customers had changed how their application was handling the communication towards one of these web services. They opened a lot of threads within the session and used more resources that we could handle. We have made changes in several areas to be able to handle this but we still would like to keep track on other customers if they also make changes that could have negative impact for the whole system.

     

     

    So, it would be great to log source IPs that use more then x threads within a session.

     

    I will read through the suggestion you made and try to build something around it. Things that I see that can be a bit tricky is the logging part. Statistics profile doesn't seem to be able to handle dynamic data so logging "source IP: 1.1.1.1 . This list must be dynamically built since I don't know the source IPs in advance.

     

     

    Thanks

     

    Robert

     

  • MR_RJ's avatar
    MR_RJ
    Icon for Cirrus rankCirrus
    Dec 01, 2011
    Hi again Aaron,

     

     

    when HTTP_REQUEST {

     

    if { [HTTP::request_num] > 1 } {

     

    log local0. "SourceIP [IP::remote_addr] - SYSTEMNAME_Threads_high [HTTP::request_num]"

     

    }

     

    }

     

     

    Gave me:

     

     

    Dec 1 12:19:48 local/tmm info tmm[5376]: Rule SYSTEMNAME_TestEnv_Threads_High_Log : SourceIP 127.40.160.226 - SYSTEMNAME_Threads_high 2

     

    Dec 1 12:19:50 local/tmm info tmm[5376]: Rule SYSTEMNAME_TestEnv_Threads_High_Log : SourceIP 127.40.160.226 - SYSTEMNAME_Threads_high 3

     

     

    So I guess that part works well!

     

    Would be nice to get it in it's own file or as I hoped for, a dynamically built list with sourceIPs and how many times the comparing number of threads has been crossed.

     

     

    Must be possible somehow =)

     

     

    //Robert

     

  • MR_RJ's avatar
    MR_RJ
    Icon for Cirrus rankCirrus
    Dec 01, 2011
     
when RULE_INIT {
  array set ::customer_sourceip { }
}

when HTTP_REQUEST { 

if { [HTTP::request_num] > 1 } then { 
  if { ([info exists ::customer_sourceip([IP::client_addr])]) } { 
        incr ::customer_sourceip([IP::client_addr])
        log local0. "SourceIP - [IP::remote_addr] - _Threads_high [HTTP::request_num]"  
}
   else { set ::customer_sourceip([IP::client_addr]) 1 }
        log local0. "SourceIP - [IP::remote_addr] - _Threads_high [HTTP::request_num]"  
}

if { [HTTP::uri] starts_with "/ROB" } {
    HTTP::respond 200 content [array get ::customer_sourceip]

set td_html ""

 foreach {SRC_IPS} [array get ::customer_sourceip] {
  append td_html "$SRC_IPS$SRC_IPS"

  }
  append td_html ""
 HTTP::respond 200 content $td_html

 }
}

I have some issues with the html page...
Found this post that you are were active in:
http://devcentral.f5.com/Community/GroupDetails/tabid/1082223/asg/50/aft/1172718/showtab/groupforums/Default.aspx

Seems to be the same problem in 10.2.1 :(
//Robert
  • nitass's avatar
    nitass
    Icon for Employee rankEmployee
    Dec 02, 2011
    I have some issues with the html page...

     

    Found this post that you are were active in:

     

    http://devcentral.f5.com/Community/GroupDetails/tabid/1082223/asg/50/aft/1172718/showtab/groupforums/Default.aspxdo you mean foreach? why do you have only one variable i.e. SRC_IPS in foreach statement??
  • MR_RJ's avatar
    MR_RJ
    Icon for Cirrus rankCirrus
    Dec 05, 2011
    when RULE_INIT {
  array set ::customer_sourceip { }
}

when HTTP_REQUEST { 

if { [HTTP::request_num] > 20 } then { 
  if { ([info exists ::customer_sourceip([IP::client_addr])]) } { 
        incr ::customer_sourceip([IP::client_addr])
        log local0. "SourceIP - [IP::remote_addr] - SYSNAME_Threads_high [HTTP::request_num]"  
}
   else { set ::customer_sourceip([IP::client_addr]) 1 }
        log local0. "SourceIP - [IP::remote_addr] - SYSNAME_Threads_high [HTTP::request_num]"  
}

if { [HTTP::uri] starts_with "/ROB" } {
    HTTP::respond 200 content [array get ::customer_sourceip]

set td_html ""

 foreach {SRC_IPS CNT_HITS} [array get ::customer_sourceip] {
  append td_html "$SRC_IPS$CNT_HITS"

  }
  append td_html ""
log local0. "ROOOOOOOOOOOOOOOB $td_html"
 HTTP::respond 200 content $td_html

 }
}

    Sorry, here is the updated one. I did some changes when it didnt work as expected, that's why I removed the other variable in the forearch statement. But since there is a bug causing the forearch with array to not work as expected, I dont know how to do a work around. Saw something about Tables, going to take a look at it shortly.

    The output right now is for example: (IP IP, and on next row, COUNT COUNT)

    127.0.0.1 127.0.0.1

    135 135

    In the other article they referred to a know issue ID, but I haven't been able to find out if it's been fixed in later releases.

    -----

    Michael: About the log thingy, I tried it but since there is no syntax checking on the log separating... the syntax looked good to me (copy paste basically) it didnt work when I tried to restart syslog. And the command gives a clear warning not to use it without F5 support assistant.

    Thanks!

    //R
  • nitass's avatar
    nitass
    Icon for Employee rankEmployee
    Dec 05, 2011
    Bug 226119 is fixed in 10.2.1 HF1. i tested in 10.2.3 and it seems to be okay.

     

     

    Bug 226119 - Tcl foreach command not indexing pair correctly (Formerly CR 140814)
  • MR_RJ's avatar
    MR_RJ
    Icon for Cirrus rankCirrus
    Dec 05, 2011
    Ah nice, might be the new ID causing me not to get any hits in my search.

     

    ...can always upgrade our test environment but production is 1-2 months away easily.

     

    Trying to read up some on tables at the moment but my skills isn't really within coding, but I do enjoy it :)

     

     

    Thanks

     

    Robert