# writing an irule to log all traffic

**URL:** <https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311>\
**Category:** Technical Forums\
**Tags:** irules, dev, devops, application-delivery\
**Created:** [July 6, 2006, 6:13pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311 "2006-07-06T18:13:57Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tracy\_Butler\_90](https://avatars.discourse-cdn.com/v4/letter/t/bc79bd/32.png) [@Tracy\_Butler\_90](https://community.f5.com/u/Tracy_Butler_90)\
**Post date:** [July 6, 2006, 6:13pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/1 "2006-07-06T18:13:57Z")

</div>

Need assistance with writing an irule to log all traffic flow. Support suggested that this should be done versus making changes to the syslog-ng file. I’ve tried making changes to syslog-ng file with no luck. Please let me know if this is worth pursuing or should I go back to the syslog-ng file.

I’m looking to log source and destination IP addresses along with the corresponding ports.

Thanks

---

<div class="post-metadata">

**Author:** ![hoolio](https://avatars.discourse-cdn.com/v4/letter/h/f6c823/32.png) [@hoolio](https://community.f5.com/u/hoolio)\
**Post date:** [July 6, 2006, 11:26pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/2 "2006-07-06T23:26:32Z")

</div>

Hi,

You can use iRules to log the requests and syslog-ng to parse them. Here are some example rules and syslog-ng changes:

=======================================================

1. HTTP logger rule:

```tcl
when HTTP_REQUEST {
    set the URL here, log it on the response
   set url [HTTP::header Host][HTTP::uri]
   set vip [IP::local_addr]:[TCP::local_port]
}
when HTTP_RESPONSE {
   set client [IP::client_addr]:[TCP::client_port]
   set node [IP::server_addr]:[TCP::server_port]
   set nodeResp [HTTP::status]
    log connection info
   log local0.info "Client: $client -> VIP:$vip$url -> Node: $node with response $nodeResp"
}

```

=======================================================

2. TCP logger rule:

```tcl
when CLIENT_ACCEPTED {  
   set vip [IP::local_addr]:[TCP::local_port]
}
when SERVER_CONNECTED {  
   set client "[IP::client_addr]:[TCP::client_port]"
   set node "[IP::server_addr]:[TCP::server_port]"
}  
when CLIENT_CLOSED {  
    log connection info
   log local0.info "Client $client -> VIP: $vip -> Node: $node"  
}

```

=======================================================

3. UDP logger rule:

```tcl
when CLIENT_ACCEPTED {  
   set vip [IP::local_addr]:[UDP::local_port]
}
when SERVER_CONNECTED {  
   set client "[IP::client_addr]:[UDP::client_port]"
   set node "[IP::server_addr]:[UDP::server_port]"
}  
when CLIENT_CLOSED {  
    log connection info
   log local0.info "Client $client -> VIP: $vip -> Node: $node"  
}

```

=======================================================

Associate the TCP, UDP and HTTP rules with the respective virtual servers that you want to log connections for. You can enable a rule for a virtual server under the Resources tab for each virtual server. You will need to make sure that the rule matches the type for each virtual server. For example, you can use the TCP or HTTP rules on an HTTP virtual server. However, you cannot associate a UDP rule unless there is a UDP profile associated with the virtual server.

These rules will log to syslog-ng’s local0 facility with the following format:

Mar 1 08:34:01 tmm tmm[730]: Rule HTTP\_logger : Client: 192.168.42.26:4746 VIP:172.25.2.12:80 to server: 172.25.2.233:80 for 172.25.2.12/ with response 200

You can then configure syslog-ng to parse local0.info entries that contain “logger” and send them to a remote syslog server by making the following changes to the /etc/syslog-ng/syslog-ng.conf file.

=======================================================

1. Add: local0.info filter, destination and log statements:

```tcl
local0.info send logger entries to remote syslog server
filter f_local0.info {
   facility(local0) and level(info) and match("logger");
};
 destination can be a hostname or IP address
destination d_logger {
   tcp("syslog.myhost.com" port (5000));
};
log {
   source(local);
   filter(f_local0.info);
   destination(d_logger);
};

```

2. Add: and not match(“logger”) to local0.\* to exclude the logger entries from being written to file

```tcl
local0.* /var/log/ltm
filter f_local0 {
   facility(local0) and level(info..emerg) and not match("logger");
};
destination d_ltm {
   file("/var/log/ltm" create_dirs(yes));
};
log {
   source(local);
   filter(f_local0);
   destination(d_ltm);
};

```

For more complete documentation on syslog-ng, you can refer to their site:

[http://www.balabit.com/products/support/syslog-ng/](http://www.balabit.com/products/support/syslog-ng/)

Or here:

[http://www.iso.port.ac.uk/docs/downloaded/syslog-ng.html/book1.html](http://www.iso.port.ac.uk/docs/downloaded/syslog-ng.html/book1.html)

Aaron

---

<div class="post-metadata">

**Author:** ![Tracy\_Butler\_90](https://avatars.discourse-cdn.com/v4/letter/t/bc79bd/32.png) [@Tracy\_Butler\_90](https://community.f5.com/u/Tracy_Butler_90)\
**Post date:** [July 11, 2006, 1:10pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/3 "2006-07-11T13:10:12Z")

</div>

Thanks for the info. I’m now receiving the logging that I needed. I’ve also discovered that when I’m sending this to a remote syslog server, it’s not using the management interface. How do you designate which interface to use when making the connection to a remote syslog server?

---

<div class="post-metadata">

**Author:** ![JRahm](https://d1p9zq3aats0t8.cloudfront.net/user_avatar/community.f5.com/jrahm/32/3327_2.png) [@JRahm](https://community.f5.com/u/JRahm)\
**Post date:** [July 13, 2006, 1:33pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/4 "2006-07-13T13:33:30Z")

</div>

I don’t think I can keep up, he’s on fire!

---

<div class="post-metadata">

**Author:** ![hoolio](https://avatars.discourse-cdn.com/v4/letter/h/f6c823/32.png) [@hoolio](https://community.f5.com/u/hoolio)\
**Post date:** [July 13, 2006, 1:35pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/5 "2006-07-13T13:35:30Z")

</div>

Hah… I have a long way to go to catch up to you guys. This forum is a great resource though and I get a lot from the posts here.

---

<div class="post-metadata">

**Author:** ![Randy\_Johnson\_1](https://avatars.discourse-cdn.com/v4/letter/r/f07891/32.png) [@Randy\_Johnson\_1](https://community.f5.com/u/Randy_Johnson_1)\
**Post date:** [October 17, 2006, 6:03pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/6 "2006-10-17T18:03:07Z")

</div>

Group -

Thanks so much for this, it’s about 95% of what I need

Is there any way to ‘timestamp’ this sort of connection info?

I’ve been requested to determine how much time is spent ‘inside’ the F5 for certain http(s) requests.

Thanks !

---

<div class="post-metadata">

**Author:** ![hoolio](https://avatars.discourse-cdn.com/v4/letter/h/f6c823/32.png) [@hoolio](https://community.f5.com/u/hoolio)\
**Post date:** [October 18, 2006, 9:07am UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/7 "2006-10-18T09:07:15Z")

</div>

Here is an example of how you can use clock to get deltas between different points in the rule execution:

```tcl
when CLIENT_ACCEPTED {
   set tcp_start_time [clock clicks -milliseconds]
}
when HTTP_REQUEST {
   set http_request_time [clock clicks -milliseconds]
}
when HTTP_RESPONSE {
   set http_response_time [clock clicks -milliseconds]
}
when CLIENT_CLOSED {
   set tcp_end_time [clock clicks -milliseconds]
   log local0. "HTTP request/response difference: $http_response_time - $http_request_time = [expr $http_response_time - $http_request_time]"
   log local0. "Total connection time: $tcp_end_time - $tcp_start_time = [expr ($tcp_end_time - $tcp_start_time)]"
}

```

Apparently, there was an issue with high CPU usage when using the clock command in versions prior to 9.2. I did some searching but couldn’t find any relevant CR’s. I would upgrade to 9.2.3+ to use the clock function and would make sure to test this rule during a maintenance window if you’re applying it to every connection through the BIG-IP.

Aaron

---

<div class="post-metadata">

**Author:** ![Randy\_Johnson\_1](https://avatars.discourse-cdn.com/v4/letter/r/f07891/32.png) [@Randy\_Johnson\_1](https://community.f5.com/u/Randy_Johnson_1)\
**Post date:** [October 19, 2006, 7:39pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/8 "2006-10-19T19:39:08Z")

</div>

Thanks, hoolio !

As always, this brings up another question.

Using this works great, but I’m getting some puzzling results from my testing.

Frequently, the BigIP says it took less time (between HTTP\_REQUEST, and HTTP\_RESPONSE) than IIS says it took to complete the request (as taken from the ‘TimeTaken’ field in the IIS logs.

Perhaps I don’t fully understand the HTTP\_RESPONSE - Could IIS still be sending data, and the F5 records the moment that it reads the header data, and not when the request is ‘complete’ ?

If so, is there a way to capture the completion of the request from the F5 perspective ?

Management suspicion is that the F5 is adding a high amount of overhead / latency to HTTP traffic, and I’m trying to refute this.

Thanks !

---

<div class="post-metadata">

**Author:** ![hoolio](https://avatars.discourse-cdn.com/v4/letter/h/f6c823/32.png) [@hoolio](https://community.f5.com/u/hoolio)\
**Post date:** [October 20, 2006, 9:49am UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/9 "2006-10-20T09:49:27Z")

</div>

I suppose you could use the HTTP\_RESPONSE\_DATA event to trigger the end time for the HTTP request/response delta, but that would require using HTTP::collect to trigger the HTTP\_RESPONSE\_DATA event. HTTP::collect buffers the HTTP response content. I’m not sure how much load this would add. I would guess that this might increase the latency enough to impact the accuracy of the time measurements.

Can anyone else comment on the best way to measure the delta between the HTTP request being received and when the BIG-IP sends the response back to the client?

Thanks,

Aaron

---

<div class="post-metadata">

**Author:** ![tungsten\_112959](https://avatars.discourse-cdn.com/v4/letter/t/77aa72/32.png) [@tungsten\_112959](https://community.f5.com/u/tungsten_112959)\
**Post date:** [April 26, 2007, 7:11am UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/10 "2007-04-26T07:11:25Z")

</div>

Hi Adrian, some question

If I created 2 logger for logging two different virtual pools, how can I perform logging to 2 different files rather than logging them into the ltm log file?

below is what I have done, but only loggerA can be logged, can we also do the same to loggerB?

- I created 2 HTTP logger iRule with name “http\_A\_logger” and “http\_B\_logger”

- change the following in the /etc/syslog-ng/syslog-ng.conf file. How can we also do for loggerB which logs to /var/Blogger? Thanks in advance.

local0.info /var/log/Alogger

filter f\_local0.info {

facility(local0) and level(info) and match(“http\_A\_logger”);

};

destination d\_Alogger {

file(“/var/log/Alogger” create\_dirs(yes));

};

log {

source(local);

filter(f\_local0.info);

destination(d\_Alogger);

};

local0.\* /var/log/ltm

filter f\_local0 {

facility(local0) and level(info..emerg) and not match(“http\_A\_logger”);

};

destination d\_ltm {

file(“/var/log/ltm” create\_dirs(yes));

};

log {

source(local);

filter(f\_local0);

destination(d\_ltm);

};

---

<div class="post-metadata">

**Author:** ![hoolio](https://avatars.discourse-cdn.com/v4/letter/h/f6c823/32.png) [@hoolio](https://community.f5.com/u/hoolio)\
**Post date:** [April 26, 2007, 7:45am UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/11 "2007-04-26T07:45:20Z")

</div>

You should be able to add another set of statements (filter, destination and log) for “Blogger” events:

```tcl
local0.info /var/log/Blogger
filter f_local0.info {
   facility(local0) and level(info) and match("http_B_logger");
};
destination d_Blogger {
   file("/var/log/Blogger" create_dirs(yes));
};
log {
   source(local);
   filter(f_local0.info);
   destination(d_Blogger);
};

```

I haven’t tested this, but I think it should work with what you have already.

Aaron

---

<div class="post-metadata">

**Author:** ![hoolio](https://avatars.discourse-cdn.com/v4/letter/h/f6c823/32.png) [@hoolio](https://community.f5.com/u/hoolio)\
**Post date:** [April 26, 2007, 4:40pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/12 "2007-04-26T16:40:42Z")

</div>

Actually, I would expect the filter names in syslog-ng.conf need to be unique. Can you try something like this?

```tcl
filter f_local0_http_A_logger {
   facility(local0) and level(info) and match("http_A_logger");
};
filter f_local0_http_B_logger {
   facility(local0) and level(info) and match("http_B_logger");
};

```

Aaron

---

<div class="post-metadata">

**Author:** ![hoolio](https://avatars.discourse-cdn.com/v4/letter/h/f6c823/32.png) [@hoolio](https://community.f5.com/u/hoolio)\
**Post date:** [April 27, 2007, 7:13am UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/13 "2007-04-27T07:13:52Z")

</div>

Nice catch. Good to hear you got it working.

Aaron

---

<div class="post-metadata">

**Author:** ![kris\_52344](https://avatars.discourse-cdn.com/v4/letter/k/45deac/32.png) [@kris\_52344](https://community.f5.com/u/kris_52344)\
**Post date:** [February 3, 2010, 6:04am UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/14 "2010-02-03T06:04:38Z")

</div>

Can anybody provide the steps to configure / integrate F5 LTM & Firepass with Splunk.

---

<div class="post-metadata">

**Author:** ![Thomas\_Schaefer](https://avatars.discourse-cdn.com/v4/letter/t/8e8cbc/32.png) [@Thomas\_Schaefer](https://community.f5.com/u/Thomas_Schaefer)\
**Post date:** [August 9, 2010, 10:10pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/15 "2010-08-09T22:10:04Z")

</div>

Please allow me to take this question one step further. I have a need to log whenever someone uses the FTP APPEND command. My thought was that I do a TCP::collect in the client\_accepted, then a switch statement in the client\_data event. I cannot seem to get the order of the TCP::collect and TCP::release right as when I connect with the iRUle, I do not get anything past the connection. It is as if the iRule is waiting on a server response. I know that tcp::collect and release are slightly different than the http counterparts, but does anyone have a hint on how I can monitor basic data without affecting the flow of data between the client and server?

---

<div class="post-metadata">

**Author:** ![hoolio](https://avatars.discourse-cdn.com/v4/letter/h/f6c823/32.png) [@hoolio](https://community.f5.com/u/hoolio)\
**Post date:** [August 10, 2010, 12:14pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/16 "2010-08-10T12:14:56Z")

</div>

Hi Thomas,

I believe the problem is that the client waits for the server to send a message first. So there isn’t any client data to collect initially. Spark described an option to use the skip\_bytes flag on TCP::collect to handle this sort of scenario:

[http://devcentral.f5.com/Forums/tabid/1082223/asg/50/showtab/groupforums/aff/5/aft/24911/afv/topic/Default.aspx25028](http://devcentral.f5.com/Forums/tabid/1082223/asg/50/showtab/groupforums/aff/5/aft/24911/afv/topic/Default.aspx25028)

However, there might be a simpler option if all you want to do is look for APPEND in the request payloads. You might be better off using a blank stream profile and iRule which applies the stream filter only on requests and logs in the STREAM\_MATCHED event. You could try enabling the stream filter using STREAM::enable in CLIENT\_ACCEPTED and then disabling it in LB\_SELECTED or SERVER\_CONNECTED.

[http://devcentral.f5.com/wiki/default.aspx/iRules/stream](http://devcentral.f5.com/wiki/default.aspx/iRules/stream)

Aaron

---

<div class="post-metadata">

**Author:** ![Pav\_70755](https://avatars.discourse-cdn.com/v4/letter/p/54ee81/32.png) [@Pav\_70755](https://community.f5.com/u/Pav_70755)\
**Post date:** [October 18, 2011, 11:18am UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/17 "2011-10-18T11:18:36Z")

</div>

Shoudl something like this work combining the two rules?

i guess i would need one to log http traffic under http\_mx\_log and one for tcp under tcp\_mx\_log?

```tcl
when HTTP_REQUEST {
    set the URL here, log it on the response
   set url [HTTP::header Host][HTTP::uri]
   set vip [IP::local_addr]:[TCP::local_port]
}

when HTTP_RESPONSE {
   set client [IP::client_addr]:[TCP::client_port]
   set node [IP::server_addr]:[TCP::server_port]
   set nodeResp [HTTP::status]

     local0.* /var/log/ltm
filter f_local0 {
facility(local0) and level(info..emerg) and not match("http_mxa_log") and not match("http_mxb_log");
};

destination d_ltm {
file("/var/log/ltm" create_dirs(yes));
};

log {
source(local);
filter(f_local0);
destination(d_ltm);
}

```

---

<div class="post-metadata">

**Author:** ![nitass](https://d1p9zq3aats0t8.cloudfront.net/user_avatar/community.f5.com/nitass/32/2270_2.png) [@nitass](https://community.f5.com/u/nitass)\
**Post date:** [October 18, 2011, 2:21pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/18 "2011-10-18T14:21:39Z")

</div>

```tcl
[root@iris:Active] config b virtual bar list
virtual bar {
   snat automap
   pool foo
   destination 172.28.17.33:http
   ip protocol tcp
   rules myrule
}
[root@iris:Active] config b rule myrule list
rule myrule {
   when CLIENT_ACCEPTED {
        log local0. "[IP::client_addr]:[TCP::client_port]"
}
}

[root@iris:Active] config b syslog include
SYSLOG - Include Data:

filter f_local0 {
   facility(local0) and
   not match("myrule");
};
log {
   source(s_syslog_pipe);
   filter(f_local0);
   filter(f_no_audit);
   destination(d_ltm);
};

filter f_myrule {
   match("myrule");
};
destination d_myrule {
   file("/var/log/myrule" create_dirs(yes));
};
log {
   source(s_syslog_pipe);
   filter(f_myrule);
   destination(d_myrule);
};

[root@iris:Active] config cat /var/log/ltm

[root@iris:Active] config cat /var/log/myrule
Oct 18 22:19:40 local/tmm info tmm[4601]: Rule myrule : 192.168.206.102:53447
Oct 18 22:19:42 local/tmm info tmm[4601]: Rule myrule : 192.168.206.102:53449
Oct 18 22:19:45 local/tmm info tmm[4601]: Rule myrule : 192.168.206.102:53450
Oct 18 22:20:10 local/iris notice b[28110]: 012e0045:5: AUDIT - user root - rule myrule list

```

hope this helps.

---

<div class="post-metadata">

**Author:** ![Pav\_70755](https://avatars.discourse-cdn.com/v4/letter/p/54ee81/32.png) [@Pav\_70755](https://community.f5.com/u/Pav_70755)\
**Post date:** [October 18, 2011, 3:50pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/19 "2011-10-18T15:50:41Z")

</div>

I now have another question is it possible to log traffic based on the cookie value of the traffic going to a particular host?

e.g. we are using an external provider for a search which we want tomonitor response times too and the http requests to that external source use a particular cookie value if that could just be logged then that would give us the info we need?

---

<div class="post-metadata">

**Author:** ![nitass](https://d1p9zq3aats0t8.cloudfront.net/user_avatar/community.f5.com/nitass/32/2270_2.png) [@nitass](https://community.f5.com/u/nitass)\
**Post date:** [October 18, 2011, 4:03pm UTC](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311/20 "2011-10-18T16:03:42Z")

</div>

I now have another question is it possible to log traffic based on the cookie value of the traffic going to a particular host?yes, it’s possible. you may check if cookie exists and then log response time for the request/response.

HTTP::cookie

[http://devcentral.f5.com/wiki/iRules.HTTP\_\_cookie.ashx](http://devcentral.f5.com/wiki/iRules.HTTP%5C_%5C_cookie.ashx)

Log Tcp And Http Request Response Info by Aaron

[http://devcentral.f5.com/wiki/iRules.LogTcpAndHttpRequestResponseInfo.ashx](http://devcentral.f5.com/wiki/iRules.LogTcpAndHttpRequestResponseInfo.ashx)

hope this helps.

[Next page](https://community.f5.com/t/writing-an-irule-to-log-all-traffic/43311.md?page=2)