Forum Discussion

Simon_Wright_85's avatar
Simon_Wright_85
Icon for Nimbostratus rankNimbostratus
Apr 04, 2008

Reducing Web Log file size

Hi All

 

 

We are trying to reduce the size of our weblogs which run at many GB per day at the moment. Much of what is in them is simply noise such as image, css, js requests etc. I would like to find a way of just logging requests for specific page types such as .asp. This would reduce our log files by as much as 95% which would be a big help.

 

 

I had the idea of writing an irule that would send a request to a separate pool of 'logging' servers whenever a request was made for a page type we wanted to log. This would mean the logging server log files would only contain the specific information we want to log and so reduce considerably our log file size.

 

 

Before i dive off down the path of writing an irule for this i wondered if other people had tried to resolve this same problem and how they did it.

 

 

Thanks in advance

 

 

Simon
config
design
  • hoolio's avatar
    hoolio
    Icon for Cirrostratus rankCirrostratus
    Apr 04, 2008
    Hi,

    It would be pretty simple to define a list of object types you want to send requests for to a specific pool. But maybe it would be cleaner to try to change the web server configuration to log only the object types you care about?

    If you wanted to use an iRule, you could define a class of string objects like this:

    
class dynamic_object_types_class {
   ".asp"
   ".aspx"
}

    The rule could look like this:

    
when HTTP_REQUEST {
    Check if the path (URI minus the query string) ends with a dynamic object type
   if {[string tolower [HTTP::path]] ends_with $::dynamic_object_types_class}{
       Send request to the dynamic object type pool for logging
      pool dynamic_object_types_pool
   } else {
       Send request to the static object type pool to not log the request
      pool static_object_types_pool
   }
}

    Aaron