cancel
Showing results for 
Search instead for 
Did you mean: 
Colin_Walker_12
Historic F5 Account

Monitors are a part of just about any infrastructure that needs to be highly available. BIG-IP has a whole host of different monitoring options for keeping tabs on servers. It's important for your BIG-IP to understand the current state of the servers behind it, the services on those servers, what to do in the event of a failure, etc. The most common monitor types are simple ICMP, TCP or HTTP requests to the individual servers behind the BIG-IP device that will give some semblance of confidence that those services are still intact.

While you could simply rely on ICMP responses to show that a server is up, in the case of a web server it is usually far preferred to send an actual HTTP request to the application in question. This to avoid the quite possible scenario where the server itself is up and responding to pings, but the app tier is failing for some reason or another. These monitors are easy to create, and as a matter of fact I'll walk you through how to do just that in a moment, but the real point of this article is to address one of the few, minor concerns when creating multiple monitors such as this - log entries.

In high traffic deployments logs tend to become near sacred as they're amazingly useful in determining failure cases, current system health, traffic patterns and more. With many pieces of network infrastructure all sending out monitoring queries you can quickly muddy the waters or even, in some cases, fill up your access logs with HTTP monitors from your BIG-IP. One way to avoid that issue is to pass a custom HTTP User-Agent header along with your monitoring traffic and tell your webserver to ignore that particular User-Agent. To put it more simply:

Problem: HTTP access logs filling up with BIG-IP monitors
Solution: Pass custom HTTP User-Agent header with monitor requests and bypass logging based on User-Agent match

 

So let's take a look at how to achieve just that. We'll use Apache as an example, but the same concept holds true regardless of your back end webserver, so long as you can configure it to ignore log entries based on the User-Agent header.

 

First, we'll need to create a custom HTTP monitor on the BIG-IP. This can be done via the UI, command line, iControl or an iApp, but we'll go through the UI steps here:

 

Create a custom HTTP monitor

  • Create a new HTTP monitor (Local Traffic > Monitors > Create)
  • Fill in the following fields:
    • Name: (an intuitive name)
    • Description: (fill in with any details you might need for reference)
    • Type: HTTP
    • Send String - This will likely be a get to your desired host or IP, with the appropriate user agent tagged on, such as:
      • GET / HTTP/1.1\r\nHost: www.mytestsite.local\r\nUser-Agent: F5 BIG-IP Monitor\r\n\r\n
    • Receive String: (a string located in a page, which qualifies the pool member as healthy)
  • Click Finished

0151T000003d4C2QAI.png

 

 

Now that you have a monitor created, next you'll need to assign that monitor to the pool in question:


Assign HTTP monitor to pool

  • Navigate to the Local Traffic: Pool List (Local Traffic > Pools > Pool List)
  • Locate the pool you would like to monitor and click on its Name
  • Select the pool name from Available Health Monitors and use the << button to move it to the Active List
  • Click Update

All right, now the BIG-IP side of things is taken care of. Your BIG-IP will now send along a custom header with this particular monitor letting any webserver it hits know that it is an F5 BIG-IP Monitor. That's great and all, but without the other half of the equation, namely telling your web server to ignore those entries and not log them, it won't do you a lot of good. With that said, let's take a look at the necessary Apache config changes to make this happen.

 

0151T000003d4C3QAI.png
Disable monitor logging for an Apache 2.x site

  • Create a new site: vi /etc/apache/sites-available/default
  • Locate the CustomLog directive in the configuration and add the following:

ErrorLog /var/log/apache2/error.log

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

SetEnvIf User-Agent "^F5\ BIG-IP\ Monitor$" monitor

CustomLog /var/log/apache2/access.log combined env=!monitor

  • Save the file: :qx!
  • Reload Apache: /etc/init.d/apache2 restart
  • Tail logs and verify that monitors are no longer appearing: tail –f /var/log/apache/access.log

There you have it! You now have a fully functioning monitor that will not appear in your Apache access log to clutter things up. This time saver will allow you grep free perusal of your logs, disk savings, and likely a much easier time sorting through things in whatever monitoring suite you're using. Thanks go to George Watkins for the idea on this one. Hopefully it helps you head down the path of monitors built to suit your needs. Keep in mind there is a lot more that you can do with monitors, this is just a basic example of making life a bit easier.

 

Comments
Arie
Altostratus
Altostratus
Wouldn't this allow a hacker to run wild without a trace of his (or her?) evil deeds in the web logs?

 

 

It would seem that this solution requires a bit more tweaking before implementing it in a Production environment...
George_Watkins_
Historic F5 Account
In theory yes, but that would require someone to somehow be aware of this

 

monitoring string. If you are worried about this scenario, it is very easy

 

to block external requests with this User-Agent with an iRule:

 

 

when HTTP_REQUEST {

 

log local0. "[IP::client_addr]:[TCP::client_port]: User-Agent: [string

 

tolower [HTTP::header "User-Agent"]] requested [HTTP::host][HTTP::uri]"

 

 

if { [string tolower [HTTP::header "User-Agent"]] contains "F5 BIG-IP

 

Monitor"} {

 

log local0. "[IP::client_addr]:[TCP::client_port]: Rejected

 

request"

 

reject

 

}

 

}

 

 

You can also add this logic to the Apache server config. Since you've already set an environment variable for the monitoring User-Agent we can reuse that to enforce access. You'll want to locate the VirtualHost's Allow and Deny directives and change the order and parameters to block external monitoring requests:

 

 

Order Deny,Allow

 

Deny from env=monitor

 

set subnet or addresses of BIG-IP self-IPs

 

allow from 10.40.0.0/16

 

 

Great question. Hope that helps!
Nick_T_68319
Nimbostratus
Nimbostratus
Any idea how to do this with IIS?
Colin_Walker_12
Historic F5 Account
Actually Nick, I'm not sure. I've got way more experience with Apache than IIS, personally. I'll dig around a bit online and see what I can find though!

 

 

Colin
Kevin_Leicht_51
Nimbostratus
Nimbostratus
I know it's been a long time since this was posted, but has anyone found a way to do this with IIS? It would great not to be filling up our web servers with clutter.
JRahm
Community Manager
Community Manager
IIS is all or nothing by default, but you can modify or create your own log module. Details here:

 

 

http://www.iis.net/learn/extensions/advanced-logging-module/advanced-logging-for-iis-custom-logging
Kevin_Leicht_51
Nimbostratus
Nimbostratus
Many thanks!
dmgeurts_23541
Nimbostratus
Nimbostratus
How about using "DNT: 1" in the header? Regarding the use by hackers. One could always use an iRule to disallow connections from clients using the browser string suggested above. Same goes for other header options like DNT.
Version history
Last update:
‎01-May-2012 11:34
Updated by:
Contributors