Improving Log Analysis with Device ID Ratios inside Elasticsearch

Overview

Ratios Power Context - Log analysis is less about raw numbers than ratios. Ratios put numbers in context.

A security analyst follows multiple ratios and gains a sense of what is a healthy range for each ratio, and follows the trend of the ratio over time, knowing that certain changes indicate trouble. As an additional data field in a SIEM system, F5 Device ID+ adds tremendous security value not so much as a raw number, that is how many Device IDs appear in a log over a period, but rather because it factors into so many useful ratios, which we’ll cover in this article. Moreover, these ratios are actionable.

You can use F5 Device ID+ to identify the network requests that require further investigation.

Within the context of your organization and your apps, each of these ratios will have an organic range of values within which it fluctuates, likely a narrow range. Changes in these ratios, especially sudden changes over hours or days, should raise alerts and trigger further investigation.

 

Goal & Architecture

While I was already using HSL (High Speed Logging) to log BIG-IP LTM / AWF related information to ELK Stack and did correlation with help of different Dashboards, the goal was to enrich that log information with a unique Device Identifier to create simple ratios, that can provide strong intel on sudden fluctuations.

 

Simple ratios that can provide strong intel with sudden fluctuations

 

The overall Architecture - where Device ID+ has been onboarded to BIG-IP using the following documented process: Getting started with F5 Device ID+

 

 

Building Ratios

Example: Single Device accessing unauthorized accounts - Sudden fluctuations in Users per Device ID

In this example, the context to create is the number of times a single device accessing unauthorized accounts.

Basically, we measure the login success rate for a single device. The login success rate refers to the percentage of login attempts that succeed out of the total of all login attempts. Applications that offer a login can be instrumented to track this metric.

If this is not possible, you might be able to make the determination of success versus failure through the web log looking at the request path and the response status code and headers. The login success rate is quite significant. Every app will have an organic rate.

A decline in that rate indicates either an attack, either a credential stuffing or brute force attack, or increased login friction.

Analyzing the login success rate per Device ID is even more helpful. Investigating requests from Device IDs with particularly low login success rates is likely to uncover attacks. In other words, the success rate itself tells us whether there is an attack while Device ID helps us to locate the source of attack.

 

The visualization within ELK is showing a Unique Device Identifier as well as a Unique IP Address, however, this single device tried to access multiple accounts on an application.

 

 

Example: Deliberate use of proxy networks - Sudden fluctuations in IPs per Device ID

We expect the ratio of IP address per Device ID to be low for desktop computers, higher for laptops, and highest for mobile devices.

Overall, the ratio across all user devices will not vary significantly over time, especially not suddenly.

A sudden change over a short duration, such an hour or a day, may indicate that some users are deliberately changing their IP addresses through the use of proxy networks, a likely signal of an attack. If discovered, investigate the network requests from Device IDs with high numbers of IP addresses.

 

 

The visualization within ELK is showing a Unique Device Identifier associated with different IP Addresses, however, this single device tried to access multiple accounts on an application.

Example: Unusual Devices accessing user accounts - Sudden fluctuations in Device IDs per User

We would expect that many users access our site from more than one device. Like the ratio of IPs to Device ID, we would expect the ratio of Device IDs per user to remain fairly stable. Likewise, a sudden change is a red flag. If suddenly more devices than expected are accessing a user account, it likely indicates an attack. Most likely, not all of those devices belong to real users. 

 

 

 

The visualization within ELK is showing multiple Device Identifier behind a single IP Address trying to access an application unauthorized by using a user account called "OttoGood".

 

 

Conclusion

By collection data into an ELK Stack and by using F5 Device ID+ to enhance your data, you can build ratios and correlations, which are helping to increase your security posture.

 

If you want to onboard F5 Device ID+ and build your own ratios, please visit Getting started with F5 Device ID+