Logging for Portal Access

Hi Nash,

The biggest hurdle that I see for what you are wanting to do is gathering and keeping that information for the period of time that you are wanting. Logs are rotating and normally only kept for around 7 days and then purged.

I'm not even sure if AVR keeps data that long (but it might).

I would suggest looking into an HSL Solution to get the logs off of the system and into an location where you can parse them to get what you are looking for. If you can do this then you can look into possibly creating an iRule to log the events that you are looking for.

I had to noodle on this one for a bit, but here's an idea:

When a portal access resource is requested, at least three things should be available

1. The HTTP URI - something that starts with "/f5-w-"
2. Hopefully the username that was used to authenticate the user
3. And the session ID

So at the VERY least, you could do something like this:

when HTTP_REQUEST {
    if { [HTTP::uri] starts_with "/f5-w-" } {
        catch {
            set resource [binary format H* [findstr [HTTP::uri] "/f5-w-" 6 "\$\$"]]
            log local0. "[ACCESS::session data get session.logon.last.username] has accessed $resource"
        }
    }
}

This will first decode the mangled URI and then send it and the username to syslog. Example:

chuck has accessed http://10.70.0.1

You'll need another mechanism to translate that resource back into the actual portal resource name. This will also of course log each request to that resource. Here's something a little fancier:

when HTTP_REQUEST {
    if { [HTTP::uri] starts_with "/f5-w-" } {
        catch {
            set resource [binary format H* [findstr [HTTP::uri] "/f5-w-" 6 "\$\$"]]
            if { [table lookup -subtable PORTALACCESS "[ACCESS::session sid]:[ACCESS::session data get session.logon.last.username]:$resource"] eq "" } {
                table set -subtable PORTALACCESS "[ACCESS::session sid]:[ACCESS::session data get session.logon.last.username]:$resource" [clock format [clock seconds] -format %Y%m%d-%H%M%S] 3600
                log local0. "[ACCESS::session data get session.logon.last.username] has accessed $resource"
            }           
        }
    }
}

This will create a session table entry based on SID:USER:RESOURCE and maintain that entry for at least an hour, so that no new logs are created for that user accessing that resource.

The next question then is what do you do with that information. I'm simply sending everything to syslog right now, but you could just as easily:

1. Send it to a remote syslog and/or High Speed Logger (HSL)
2. Send SNMP messages
3. Store it all in the session table and build a reports engine (iRule) based on that data.
4. Given that you may not care WHO is accessing the site, you could also potentially use a stat or iStat profile to capture unique visits over a period of time. The stat/iStat data would then be accessible from the shell, scriptable and exportable. And because you're accessing from the shell, you could also translate that resource URL back into the portal resource name.

That is totally awesome... Thanks very much Kevin, that iRule logs the perfect amount of data I need.

I think I need to do some performance benchmarking though. I "think" I noticed a slight performance hit when running the iRule. Having said that, it is running on my dev VE BigIP and not the 6900s that we have.

Our logs get shipped to a syslog now, so this will just be captured with that process and then the data is there for me to do what I need to.

For info -> I am using the "fancier" bit of code.

If you're seeing any kind of performance hit at all, it's likely because of the URI demangling. As an alternative you could just store the access username and "/f5-w-.../" URI value (literal) as the table key and only decode it once if its a new value.

Thanks for the suggestion for modifying the "URI demangling".. Will give that a go and see if it makes any difference.

Kevin,

Can you give more details on this rules-based 'Reporting Engine"? Is there an example on devcentral that I didn't find?

Thank you,

A "rules-based reporting engine" isn't really a thing, but rather an idea, and there are several references throughout DC of people using iRules to display reporting information. Example:

Heat Maps iRule by Colin

You have to:

1. Capture and store this information somewhere - depending on size, quantity, rate, and value, you could use tables, stats/iStats, remote services, etc.

    

2. Build an iRule that consumes that data, wherever you've put it, and display it however you need it displayed. In all practicality this isn't done much simply because to use an iRule implies that you're making the data accessible to the outside, but you could definitely create HTML content filled with reporting data, images, you name it. But again, it depends on complexity.

    

So if it's a lot of data, I'd highly recommend third-party products like Splunk (it's what they do for a living). If it's a little bit of data, stuff it into a table or stat/iStat and collect/display it in an iRule.

Or you could just decode the hex string...

I don't know how to do this programatically but this might be a good start: https://devcentral.f5.com/wiki/iRules.Log_binary_HTTP_payload_in_hex.ashx

From ( https://devcentral.f5.com/articles/apm-security-protecting-internal-resources-using-acls )

$$/

That long string of characters between f5-w- and $$ may look like random data. However it is a hex ASCII representation of a human readable string (68 = h, 74 = t, etc.).

As we humans don’t read hex as well as computers a converter quickly converts the hex string to human readable text (http://www.string-functions.com/hex-string.aspx). It turns out to be