writing an irule to log all traffic

Actually, I would expect the filter names in syslog-ng.conf need to be unique. Can you try something like this?

filter f_local0_http_A_logger {
   facility(local0) and level(info) and match("http_A_logger");
};
filter f_local0_http_B_logger {
   facility(local0) and level(info) and match("http_B_logger");
};

Aaron

Nice catch. Good to hear you got it working.

 

 

Aaron

Can anybody provide the steps to configure / integrate F5 LTM & Firepass with Splunk.

 

 

Please allow me to take this question one step further. I have a need to log whenever someone uses the FTP APPEND command. My thought was that I do a TCP::collect in the client_accepted, then a switch statement in the client_data event. I cannot seem to get the order of the TCP::collect and TCP::release right as when I connect with the iRUle, I do not get anything past the connection. It is as if the iRule is waiting on a server response. I know that tcp::collect and release are slightly different than the http counterparts, but does anyone have a hint on how I can monitor basic data without affecting the flow of data between the client and server?

Hi Thomas,

 

 

I believe the problem is that the client waits for the server to send a message first. So there isn't any client data to collect initially. Spark described an option to use the skip_bytes flag on TCP::collect to handle this sort of scenario:

 

 

http://devcentral.f5.com/Forums/tabid/1082223/asg/50/showtab/groupforums/aff/5/aft/24911/afv/topic/Default.aspx25028

 

 

However, there might be a simpler option if all you want to do is look for APPEND in the request payloads. You might be better off using a blank stream profile and iRule which applies the stream filter only on requests and logs in the STREAM_MATCHED event. You could try enabling the stream filter using STREAM::enable in CLIENT_ACCEPTED and then disabling it in LB_SELECTED or SERVER_CONNECTED.

 

 

http://devcentral.f5.com/wiki/default.aspx/iRules/stream

 

 

Aaron

Shoudl something like this work combining the two rules?

i guess i would need one to log http traffic under http_mx_log and one for tcp under tcp_mx_log?

 when HTTP_REQUEST {
    set the URL here, log it on the response
   set url [HTTP::header Host][HTTP::uri]
   set vip [IP::local_addr]:[TCP::local_port]
}

when HTTP_RESPONSE {
   set client [IP::client_addr]:[TCP::client_port]
   set node [IP::server_addr]:[TCP::server_port]
   set nodeResp [HTTP::status]

     local0.* /var/log/ltm
filter f_local0 {
facility(local0) and level(info..emerg) and not match("http_mxa_log") and not match("http_mxb_log");
};

destination d_ltm {
file("/var/log/ltm" create_dirs(yes));
};

log {
source(local);
filter(f_local0);
destination(d_ltm);
}

[root@iris:Active] config  b virtual bar list
virtual bar {
   snat automap
   pool foo
   destination 172.28.17.33:http
   ip protocol tcp
   rules myrule
}
[root@iris:Active] config  b rule myrule list
rule myrule {
   when CLIENT_ACCEPTED {
        log local0. "[IP::client_addr]:[TCP::client_port]"
}
}

[root@iris:Active] config  b syslog include
SYSLOG - Include Data:

filter f_local0 {
   facility(local0) and
   not match("myrule");
};
log {
   source(s_syslog_pipe);
   filter(f_local0);
   filter(f_no_audit);
   destination(d_ltm);
};

filter f_myrule {
   match("myrule");
};
destination d_myrule {
   file("/var/log/myrule" create_dirs(yes));
};
log {
   source(s_syslog_pipe);
   filter(f_myrule);
   destination(d_myrule);
};

[root@iris:Active] config  cat /var/log/ltm

[root@iris:Active] config  cat /var/log/myrule
Oct 18 22:19:40 local/tmm info tmm[4601]: Rule myrule : 192.168.206.102:53447
Oct 18 22:19:42 local/tmm info tmm[4601]: Rule myrule : 192.168.206.102:53449
Oct 18 22:19:45 local/tmm info tmm[4601]: Rule myrule : 192.168.206.102:53450
Oct 18 22:20:10 local/iris notice b[28110]: 012e0045:5: AUDIT - user root - rule myrule list

hope this helps.

I now have another question is it possible to log traffic based on the cookie value of the traffic going to a particular host?

 

 

e.g. we are using an external provider for a search which we want tomonitor response times too and the http requests to that external source use a particular cookie value if that could just be logged then that would give us the info we need?

I now have another question is it possible to log traffic based on the cookie value of the traffic going to a particular host?yes, it's possible. you may check if cookie exists and then log response time for the request/response.

 

 

HTTP::cookie

 

http://devcentral.f5.com/wiki/iRules.HTTP__cookie.ashx

 

 

Log Tcp And Http Request Response Info by Aaron

 

http://devcentral.f5.com/wiki/iRules.LogTcpAndHttpRequestResponseInfo.ashx

 

 

hope this helps.

So something like this

when CLIENT_ACCEPTED {
 Get time for start of TCP connection in milleseconds
if { [HTTP::cookie exists "testcookie"]  } {
set tcp_start_time [clock clicks -milliseconds]
 
 Log the start of a new TCP connection
log "New TCP connection from [IP::client_addr]:[TCP::client_port] to [IP::local_addr]:[TCP::local_port]"
}
when HTTP_REQUEST {
 Get time for start of HTTP request
set http_request_time [clock clicks -milliseconds]
 
 Log the start of a new HTTP request
set LogString "Client [IP::client_addr]:[TCP::client_port] -> [HTTP::host][HTTP::uri]"
log local0. "$LogString (request)"
}
 
when HTTP_RESPONSE {
 Received the response headers from the server.  Log the pool name, IP and port, status and time delta
log local0. "$LogString (response) - pool info: [LB::server] - status: [HTTP::status] (request/response delta: [expr [clock clicks -milliseconds] - $http_request_time]ms)"
}
when CLIENT_CLOSED {
 Log the end time of the TCP connection
log "Closed TCP connection from [IP::client_addr]:[TCP::client_port] to [IP::local_addr]:[TCP::local_port] (open for: [expr [clock clicks -milliseconds] - $tcp_start_time]ms)"
}

Will this just log the requests that contain that cookie value and give the response times? and does this just write the info to the standard log file?

Thanks for your help

Pav