IP to DNS reverse lookup + Geo location? Delimitted format?

Hi Matt,

 

 

I think it would be more efficient (and cheaper) to do this on an application server.

 

 

In v10.1, LTM adds an inbuilt Quova GeoIP database. See the whereis wiki page for details (http://devcentral.f5.com/wiki/iRules.whereis.ashx). But with the default license you get with LTM, you can only use the database to make load balancing decisions. You can't pass the GeoIP data onto any other non-LTM host. You could try getting a custom license for more than this from Quova, but I don't think it's cheap. Even if you could use the default Quova license for this, I don't think it makes sense to try to log every HTTP request or TCP connection via LTM and do IP lookups just for reporting. That would eat into the resources LTM should be using for app delivery.

 

 

On the other hand, you could pull your www server logs and perform GeoIP and reverse DNS lookups on them for reporting. MaxMind offers a free GeoIP database with a lot of options for querying.

 

 

Aaron

Actually, I need to correct myself. The whereis wiki page shows that this usage is allowed:

 

 

Logging data can contain location info and collected into a central logging solution for analysis of F5 logs.

 

 

So theoretically, you could upgrade to 10.1+ and then use High Speed Logging and send a message to a pool of syslog servers for each HTTP request with the GeoIP info. This should have relatively low overhead in terms of CPU/memory usage. I'd still avoid doing reverse DNS lookups on every client IP if you can do that easily off LTM.

 

 

http://devcentral.f5.com/wiki/iRules.whereis.ashx

 

http://devcentral.f5.com/wiki/iRules.hsl.ashx

 

 

Aaron

Arggg, 10.x. We've tried to find a memory module to upgrade our LTM 1500, as I really want to upgrade to 10.x, but simply cannot find one anywhere! Currently, we don't have the amount of recommended RAM to upgrade to 10.x, hence we are stuck at 9.4.8 :(

 

 

I guess I'm stuck with the MaxMind method and parsing through the logs? Thanks for the idea.

You won't be able to add RAM to a 1500 unless you buy it from F5 and F5 stopped stocking that a bit ago :(

 

 

Aaron

If i want to simply log traffic during it's initial connection, and not log the same IP over and over again while it's visting the site, what is the optimal way to capture that via an irule?

You'd need to track which IP's you've already logged. This could eat up considerable memory on LTM just for logging.

 

 

If you're on 9.4.8 you can use the remote logging option on the log command to log all connections to the VS. This should have a much lower impact than trying to write to LTM disk.

 

 

 

http://devcentral.f5.com/wiki/iRules.log.ashx

 

 

log [-noname] < remote_ip >[:< remote_port>] < facility>.[< level>] < message>

 

 

* (LTM only) Logs the specified message directly to the specified IP address (and optional alternate port when specified) via UDP. Facility and/or level are required. The iRule name prefixing the message text may optionally suppressed by including the -noname option. < remote_ip> must be a TMM-routed address. If you must route specific messages to a remote address via the management interface, you must log locally. syslog-ng is able to route messages via both TMM and management interfaces using the standard syntax. You can define an appropriate filter and remote log destination in LTM's syslog-ng service.

 

 

 

Aaron

Ok, thanks Aaron. I also just found a post you wrote back in 08, i might try that first as i'm not familiar with logging to a syslog server

 

 

to jondyke in nov of 08

 

 

Thanks for your help!

 

 

when CLIENT_ACCEPTED {

 

Add some logic for determining which clients to log for

 

if {[matchclass [IP::client_addr] equals $::filteredAddresses}{

 

Get time for start of TCP connection in milleseconds

 

set tcp_start_time [clock clicks -milliseconds]

 

Log the start of a new TCP connection

 

log "New TCP connection from [IP::client_addr]:[TCP::client_port] to [IP::local_addr]:[TCP::local_port]"

 

} else {

 

Disable all events for this rule and any other rule for this connection

 

event disable all

 

}

 

}

 

when HTTP_REQUEST {

 

Get time for start of HTTP request

 

set http_request_time [clock clicks -milliseconds]

 

Log the start of a new HTTP request

 

set LogString "Client [IP::client_addr]:[TCP::client_port] -> [HTTP::host][HTTP::uri]"

 

log local0. "$LogString (request)"

 

}

 

when HTTP_RESPONSE {

 

Received the response headers from the server. Log the pool name, IP and port, status and time delta

 

log local0. "$LogString (response) - pool info: [LB::server] - status: [HTTP::status] (request/response\

 

delta: [expr [clock clicks -milliseconds] - $http_request_time]ms)"

 

}

 

when CLIENT_CLOSED {

 

Log the end time of the TCP connection

 

log "Closed TCP connection from [IP::client_addr]:[TCP::client_port] to [IP::local_addr]:[TCP::local_port]\

 

(open for: [expr [clock clicks -milliseconds] - $tcp_start_time]ms)"

 

}

oh, and PS, that would be the only irule i have against the virtual server, per your comment about disabling events for any rule for that connection

 

that was a very creative idea by the way :)

Hi everyone, i've strayed away from this post for a while, but am trying to resurrect the above irule. However, i get a "no data received" error when i apply the irule to my existing site, for which i have a default pool selected. Any ideas on what i'd need to modify to make this work, or what i'm doing wrong? I just want to log one ip per user, per session, and then am planning on trying Hoolio's idea for Maxmind reverse dns and geo.

 

 

There's probably a simple answer here, i'm just not sure what that is....

Hi Matt,

 

 

Can you post the exact iRule you're testing in [ code ] [/ code ] blocks? Are you logging locally to /var/log/ltm? Or remotely to a syslog server? Where are you seeing the "no data received" error?

 

 

Aaron