Writing to and rotating custom log files

Sometimes I need to log information from iRules to debug something. So I add a simple log statement, like this:

  when HTTP_REQUEST {
  if { [HTTP::uri] equals "/secure" } {
    log local0. "[IP::remote_addr] attempted to access /secure"
  }
}

This is fine, but it clutters up the /var/log/ltm log file. Ideally I want to log this information into a separate log file. To accomplish this, I first change the log statement to incorporate a custom string - I chose the string "##":
 
  when HTTP_REQUEST {
  if { [HTTP::uri] equals "/secure" } {
    log local0. "##[IP::remote_addr] attempted to access /secure"
  }
}

Now I have to customize syslog to catch this string, and send it somewhere other than /var/log/ltm. I do this by customizing syslog with an include statement:
 
  
tmsh modify sys syslog include '"
filter f_local0 {
    facility(local0) and not match(\": ##\");
};
 
filter f_local0_customlog {
    facility(local0) and match(\": ##\");
};
 
destination d_customlog {
    file(\"/var/log/customlog\" create_dirs(yes));
};
 
log {
    source(local);
    filter(f_local0_customlog);
    destination(d_customlog);
};
"'
save the configuration change:
 
  tmsh save / sys config
 
and restarting the syslog-ng service:
 
tmsh restart sys service syslog-ng
 
The included "f_local0" filter overrides the built-in "f_local0" syslog-ng filter, since the include statement will be the last one to load. The "not match" statement is regex which will prevent any statement containing a “##” string from being written to the /var/log/ltm log. The next filter, "f_local0_customlog", catches the "##" log statement and the remaining include statements handle the job of sending them to a new destination which is a file I chose to name "/var/log/customlog".

You may be asking yourself why I chose to match the string ": ##" instead of just "##". It turns out that specifying just "##" also catches AUDIT log entries which (in my configuration) are written every time an iRule with the string "##" is modified. But only the log statement from the actual iRule itself will contain the ": ##" string. This slight tweak keeps those two entries separated from each other.

So now I have a way to force my iRule logging statements to a custom log file. This is great, but how do I incorporate this custom log file into the log rotation scheme like most other log files? The answer is with a logrotate include statement:
  tmsh modify sys log-rotate syslog-include '"
/var/log/customlog {
    compress
    missingok
    notifempty
}"'
 
and save the configuration change:
 
  tmsh save / sys config
 
Logrotate is kicked off by cron, and the change should get picked up the next time it is scheduled to run.
 
And that's it. I now have a way to force iRule log statements to a custom log file which is rotated just like every other log file. It’s important to note that you must save the configuration with "tmsh save / sys config" whenever you execute an include statement. If you don't, your changes will be lost then next time your configuration is loaded. That's why I think this solution is so great - it's visible in the bigip_sys.conf file - not like customizing configuration files directly. And it's portable.
Published May 27, 2010
Version 1.0
  • cool idea, but i'm wondering why can't you just log to a different local facility and split that off with syslog-ng?
  • You certainly could - there are several different ways to accomplish what I have described. I chose this approach for two reasons. First, what's being logged in this article is based on iRules which are processed by the TMM process. TMM has been designated by F5 in syslog as facility local0, according to SOL5531. So using a different facility would contradict that designation. Secondly, our organization historically has tended to use the simple "log local0." statement in iRules. Customizing syslog and logrotate the way I have described allows us to continue using that standard.
  • svs's avatar
    svs
    Icon for Cirrostratus rankCirrostratus
    Great article. The configuration seems very dirty to me. In 11.6.0 I had issues with indentation. I got the message "Application error for confpp: syntax error in /etc/syslog-ng/syslog-ng.conf". So what worked for me was to go without indentation for the syslog-ng configuration and with indentation for the logrotate configuration: http://pastebin.com/ENq36JQG (Couldn't paste the config here directly. ASM of DevCentral throwed an exception... :-() I checked the configuration with 12.0 and 11.6.0. The only difference I was able to see was, that with 12.0 the logfile was created directly after restarting the syslog daemon, while in 11.6.0 there had to come logs from the iRule, before the logfile was created. Once again: Great article! Many thanks, saved my day. Greets, svs
  • SVS,

     

    I had to write a simple iRule and log Client IP address which is hitting one particular VIP(LDAP) and to a separate log file and I am running 11.6.0 version.

     

    Could you please tell me how my iRule should look like for the custom log file to work and how should that reflect in the syslog-ng file.

     

    when CLIENT_ACCEPTED { log local0. "Client IP address:{IP:client_addr]" } }

     

    Thanks Balaji

     

  • Hi! What happened with the syslog settings when you upgrade the F5?

  • I tried searching for more information in the TMSH Reference Guide 12.0 but it does not go into details about the coding for the "Include" option of syslog on F5.

     

    I would appreciate it if someone could point me to a resource where I can learn more on how to use this feature.

  • Marvin's avatar
    Marvin
    Icon for Cirrocumulus rankCirrocumulus

    I have an issue with the log rotation not working anymore but it seems the config is correct

     

    dehama01@(f5-lb-dmz-1)(cfg-sync Changes Pending)(Active)(/Common)(tmos)# list sys log-rotate syslog-include
    sys log-rotate {
        syslog-include "
    /var/log/IPI_L4 {
        compress
        missingok
        notifempty
    }
    var/log/GeoIP {
        compress
        missingok
        notifempty
    }
    "
    }

     

  • Marvin's avatar
    Marvin
    Icon for Cirrocumulus rankCirrocumulus

    I get this error with log rotation a verify config does not show any error

     

    [root@f5-lb-dmz-1:Active:In Sync] images # logrotate -f /etc/logrotate.conf
    error: could not change directory to '.'error: syslog-ng:70 unknown option 'var' -- ignoring line
    error: syslog-ng:74 unexpected }
    error: found error in file syslog-ng, skipping

     

    looking into the syslog-ng config file i see this

     

    70 # local0.none,local2.none;local4.none;local5.none;local6.none \
         71 #                                               /var/log/messages
         72 filter f_notLocal0 {
         73    not facility(local0);
         74 };

     

    I have a open case with F5 support for this log rotate issue