WA access logs

I am no stranger to your predicament. I have to manage 12gb of log files per day. What I do is at the end of the day I have the logs ftp'd to a log box with around 100gb of storage. A perl script processes the files and pulls out information that I need to have on a day to day basis, not simply to look at them be able to recall specific information I only need. The files are the zipped up and moved to another partision which they sit in near line storage for 1 month. There after it's backed up into tape.

 

 

I hope this helps

 

 

CB

 

 

 

 

 

 

CB,

 

I was thinking of a similar but opposite approach. We have some existing processes that will fetch the files, but I want to make sure I understand what I have to work with.

 

I'm curious: did you keep the default logrotate method on the WA?

 

If I understand it correctly, it only keeps the last 24hrs of data in 10MB chunks, correct?

 

I would like to have a little more room for error there and maybe keep a couple days worth in case the process hiccups. I'd also like a logical naming convention, but my initial response from F5 support is this is not easily changed.

Nope. I rotated based on the end of the day rather basing in file size. Even though I had local storage I didn't want to small files because it's too many files to search through. One giant file with perl script pretty much did the trick.

 

 

/CB

 

Out of curiosity are you referring to the HTTP Access logs or the system logs like the pvac.log?

 

 

The pvac.log and comm_srv.log is stored in /var/log/wa and is zipped on a daily basis and archived for 7 days on the system.

 

 

HTTP hit logs or access logs are stored in /var/wa/log/access and these are stored for 30 days on the system.

 

 

Looking at these directories you should see daily files as *.gz

 

 

It is definitely recommended that these files be transferred to external storage especially if they are going to be needed long term.

I am referring to the logs in: /var/log/wa/access/*.log

 

 

and referencing the default wa_logrotate script in cron.hourly that loads:

 

/var/run/config/logrotate.d/wa

 

 

 

 

THIS IS AN AUTO-GENERATED FILE -- DO NOT EDIT!!!

 

 

rotate 24

 

create

 

compress

 

notifempty

 

missingok

 

 

/var/log/wa/access*.log /var/log/wa/pvacLog_8081 {

 

size=10M

 

sharedscripts

 

postrotate

 

/bin/kill -HUP `/sbin/pidof -s pvac 2> /dev/null` 2> /dev/null || true

 

endscript

 

}

 

 

/var/log/wa/stats/*.log /var/log/wa/stats/system/*.log {

 

size=10M

 

}

 

 

/var/log/wa/stats/perfmonitor/*.log {

 

rotate 0

 

size=10M

 

}

 

wa

 

 

 

where do you get 30 days from this? You must be talking about an older version.

 

 

PS: I am working with the WA 9.4.3 Build 1.4 Final

 

 

actually , i just noticed a discrepancy:

 

the default logrotate script points to: /var/log/wa/access*.log

 

but the access logs are actually stored in: /var/log/wa/access/*.log

 

Surely, this seems like a 'bug'.

 

According to these defaults, my access logs would never get rotated.

 

 

I have a reply from F5 support on how to add the real directory to logrotate.

 

I will post it once I have tested it.

Turns out this is a known issue and fixed in hotfix 2:

 

https://support.f5.com/kb/en-us/solutions/public/8000/300/sol8320.html?sr=564072

This is what I ended up doing and is working as desired.

 

(Note: I opted for a daily rotation schedule. This unit is not in production yet. I may need to tune it down once it starts logging in a high traffic environment.)

 

 

1. Create a temp file /var/tmp/wa-logrotate.txt with following contents:

 

logrotate {

 

wa include "/var/log/wa/access/*.log {

 

compress

 

ifempty

 

rotate 4

 

daily

 

olddir /var/log/wa/access/archive

 

sharedscripts

 

postrotate

 

/bin/kill -HUP `/sbin/pidof -s pvac 2> /dev/null` 2> /dev/null || true

 

endscript

 

}

 

"

 

}

 

 

 

 

2. run: bpsh < /var/tmp/wa-logrotate.txt

 

 

3. run: bigpipe save all

 

 

4. test configuration by running: logrotate -d /etc/logrotate.d/wa

 

 

Note: I have not applied the above mentioned hot fixes yet, so i'm not sure how this will work once the box is updated (cross that bridge later). Until I do so, this is working fine.

well, as I suspected: installing the hotfix collides with the previously mentioned change. You end up with duplicate definitions and an error:

[root@webacc01:Active] tmp  logrotate -d /etc/logrotate.d/wa 
reading config file /etc/logrotate.d/wa
reading config info for /var/log/wa/access/*.log /var/log/wa/pvacLog_8081 
reading config info for /var/log/wa/stats/*.log /var/log/wa/stats/system/*.log 
reading config info for /var/log/wa/stats/perfmonitor/*.log 
error: /etc/logrotate.d/wa:26 duplicate log entry for /var/log/wa/access/access_log_web_si_v127.log

Waiting for guidance from support on how to back out the previous changes. Although at this point, I'd prefer to keep my custom access log rotation schedule and remove /var/log/wa/access/* from the default. I'll post solution here when I get it.

to back out the previous changes:

 

b logrotate wa include none

 

 

Great. Now i'm back to square one.