on 01-May-2012 11:34
Monitors are a part of just about any infrastructure that needs to be highly available. BIG-IP has a whole host of different monitoring options for keeping tabs on servers. It's important for your BIG-IP to understand the current state of the servers behind it, the services on those servers, what to do in the event of a failure, etc. The most common monitor types are simple ICMP, TCP or HTTP requests to the individual servers behind the BIG-IP device that will give some semblance of confidence that those services are still intact.
While you could simply rely on ICMP responses to show that a server is up, in the case of a web server it is usually far preferred to send an actual HTTP request to the application in question. This to avoid the quite possible scenario where the server itself is up and responding to pings, but the app tier is failing for some reason or another. These monitors are easy to create, and as a matter of fact I'll walk you through how to do just that in a moment, but the real point of this article is to address one of the few, minor concerns when creating multiple monitors such as this - log entries.
In high traffic deployments logs tend to become near sacred as they're amazingly useful in determining failure cases, current system health, traffic patterns and more. With many pieces of network infrastructure all sending out monitoring queries you can quickly muddy the waters or even, in some cases, fill up your access logs with HTTP monitors from your BIG-IP. One way to avoid that issue is to pass a custom HTTP User-Agent header along with your monitoring traffic and tell your webserver to ignore that particular User-Agent. To put it more simply:
Problem: HTTP access logs filling up with BIG-IP monitors
Solution: Pass custom HTTP User-Agent header with monitor requests and bypass logging based on User-Agent match
So let's take a look at how to achieve just that. We'll use Apache as an example, but the same concept holds true regardless of your back end webserver, so long as you can configure it to ignore log entries based on the User-Agent header.
First, we'll need to create a custom HTTP monitor on the BIG-IP. This can be done via the UI, command line, iControl or an iApp, but we'll go through the UI steps here:
Create a custom HTTP monitor
Now that you have a monitor created, next you'll need to assign that monitor to the pool in question:
Assign HTTP monitor to pool
All right, now the BIG-IP side of things is taken care of. Your BIG-IP will now send along a custom header with this particular monitor letting any webserver it hits know that it is an F5 BIG-IP Monitor. That's great and all, but without the other half of the equation, namely telling your web server to ignore those entries and not log them, it won't do you a lot of good. With that said, let's take a look at the necessary Apache config changes to make this happen.
Disable monitor logging for an Apache 2.x site
ErrorLog /var/log/apache2/error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warnSetEnvIf User-Agent "^F5\ BIG-IP\ Monitor$" monitor
CustomLog /var/log/apache2/access.log combined env=!monitor
There you have it! You now have a fully functioning monitor that will not appear in your Apache access log to clutter things up. This time saver will allow you grep free perusal of your logs, disk savings, and likely a much easier time sorting through things in whatever monitoring suite you're using. Thanks go to George Watkins for the idea on this one. Hopefully it helps you head down the path of monitors built to suit your needs. Keep in mind there is a lot more that you can do with monitors, this is just a basic example of making life a bit easier.
It would seem that this solution requires a bit more tweaking before implementing it in a Production environment...
monitoring string. If you are worried about this scenario, it is very easy
to block external requests with this User-Agent with an iRule:
when HTTP_REQUEST {
log local0. "[IP::client_addr]:[TCP::client_port]: User-Agent: [string
tolower [HTTP::header "User-Agent"]] requested [HTTP::host][HTTP::uri]"
if { [string tolower [HTTP::header "User-Agent"]] contains "F5 BIG-IP
Monitor"} {
log local0. "[IP::client_addr]:[TCP::client_port]: Rejected
request"
reject
}
}
You can also add this logic to the Apache server config. Since you've already set an environment variable for the monitoring User-Agent we can reuse that to enforce access. You'll want to locate the VirtualHost's Allow and Deny directives and change the order and parameters to block external monitoring requests:
Order Deny,Allow
Deny from env=monitor
set subnet or addresses of BIG-IP self-IPs
allow from 10.40.0.0/16
Great question. Hope that helps!
Colin