Enriching AFM with public domain Threat Intelligence

Introduction

F5 BIG-IP Advanced Firewall Manager (AFM) is F5's network firewall product - it does all of the things you might expect of a network firewall, albeit with a focus on application visibility that only F5 can provide. One truism of application security in recent years (decades?) is while the effacacy of a traditional network firewall may have diminished (e.g. layer 4 constructs cannot address application security), a firewall is still a valuable tool for any application owner. Even considering the perimeter-less nature of modern organisations, a firewall provides the capability to discard malicious traffic that should never be allowed anywhere near your applications. Additionally, BIG-IP AFM can go far beyond static layer 4 policies.

There are quite a few great BIG-IP AFM introductory documents for those that aren't familiar:

IP Intelligence

One of my favourite BIG-IP AFM features is IP intelligence. IP Intelligence is a categorised dynamic list of suspicious IP addresses. By identifying IP addresses and security categories associated with malicious activity, the IP Intelligence service can incorporate dynamic lists of malicious IP addresses into firewall policy, adding context to policy decisions. IP Intelligence service reduces risk and increases data center efficiency by eliminating the effort to process bad traffic.

IP Intellgence provides the following default categories:

Additional
Appiq_badactors
Application_denial_of_service
Attacked_ips
Botnets
Denial_of_service
Infected_sources
Mobile_threats
Phishing
Scanners
Spam_sources
Tor_proxy
Web_attacks
Windows_exploits

The IP Intelligence data is a dynamic list provided by a 3rd party (Webroot), and is commonly licensed in many network firewall (& NGFW) products under the guise of Threat Intelligence. It is a paid subscription service available with BIG-IP AFM (and/or AWAF).

Public Domain Threat Intelligence sources

There also exists many publically available Threat Intelligence resources, these are lists compiled by security engineers at the coalface of many security projects and organisations. Some notable sources include projects such as Spamhaus, Dshield & Abuse.ch.

There are many potential sources of Threat Intelligence in the public domain.

Below are some that I am using in my environment:

To use such lists in BIG-IP AFM, we just need a URL we can query that returns a plain-text list of IPs (or subnets).

Defining your own IP Intelligence feeds

IP Intellgence allows security engineers to create and import their own feed of bad actors. Multiple feeds can be created, and custom categories can be assigned also.

The following configuration is what I am using:

security ip-intelligence feed-list custom_blocklist {
    feeds {
        binarydefense {
            default-blacklist-category binarydefense
            poll {
                url http://192.168.0.1/feed/binarydefense.txt
            }
        }
        emergingthreats {
            default-blacklist-category emergingthreats 
            poll {
                url http://192.168.0.1/feed/emergingthreats.txt
            }
        }
        pfsense {
            default-blacklist-category pfsense
            poll {
                url http://192.168.0.1/feed/pfsense.txt
            }
        }
    }
}

Note that I am downloading my external feeds onto an management host (192.168.0.1), this is so that I can do some formatting changes to the data. The IP Intelligence feed service expects to receive a list of IPs (and/or subnets), in a CSV format. Often publically available blocklists have comments and other junk in them that confuse the poller. I have automated this into a number of cron jobs:

In my configuration, I am using a separate category per blocklist. It is also possible to set a category for each IP in the feed, simply by populating a category as the second field in each line. E.g. '1.2.3.4,category2'.

Example cron job on my management host:

# Emergingthreats blocklist
#
* 5 * * * curl -s https://gist.githubusercontent.com/BBcan177/bf29d47ea04391cb3eb0/raw/ |egrep -v '^#' |awk '{print $1}' > /var/www/feed/pfsense.txt

The grep & awk commands are to remove lines that start with a comment (#) and any additional fields after the IP.

Firewall configuration

There are several ways to configure IP Intelligence; you can configure it Globally, at a Route Domain level, on a Virtual Server and with BIG-IP AFM provisioned also in a firewall policy. The most common way to configure IP Intelligence is at a Global level, there is a nice guide on how to do that here:

https://techdocs.f5.com/en-us/bigip-15-0-0/big-ip-afm-getting-started/afm-ip-intelligence-and-reputation.html

I prefer to configure IP Intelligence at a firewall policy level, this allows more granularity on how to treat the individual IP Intelligence categories, and I can also combine the IP Intelligence information with other constructs such as particular destinations, ports or Geolocations. Below is an example firewall rule list:

security firewall rule-list block_threat_intel {
    rules {
        block_threat_src {
            action drop
            ip-protocol any
            log yes
            source {
                ipi-category {
                    additional
                    appiq_badactors
                    application_denial_of_service
                    attacked_ips
                    binarydefense
                    botnets
                    denial_of_service
                    emergingthreats
                    infected_sources
                    mobile_threats
                    pfsense
                    phishing
                    scanners
                    spam_sources
                    tor_proxy
                    web_attacks
                    windows_exploits
                }
            }
        }
    }
}

Is it working?

As with any firewall policy change, the best way to see if it is working is to review the firewall logs:

You can also use tmsh to view the category associated with any given IP address:

root@(bigip1)(cfg-sync Standalone)(ModuleNotLicensed::Active)(/Common)(tmos)# show security ip-intelligence info address 188.93.229.73
Security::IP Intelligence Address  :  188.93.229.73
  Global context
  IP Intelligence Sources          :  User-defined
  Whitelisted (Source)             :  no
  Whitelisted (Destination)        :  no
  Policy Action (Source)           :  allow
  Policy Action (Destination)      :  allow
  Match Type                       :  Source and Destination
  Categories (Source) (1)          :    pfsense
  Categories (Destination) (1)     :    pfsense
root@(bigip1)(cfg-sync Standalone)(ModuleNotLicensed::Active)(/Common)(tmos)#

Conclusion

What I find most interesting about all of the various available Threat Intelligence sources, is just how much of it there is, and also how not much of it completely overlaps. As you will see for yourself, there isn't 100% coverage in any single source. Each Threat Intelligence effort has its own strengths and unique view of the threat landscape. Sometimes this is due to obvious bias in the way the data is collected (e.g. Email providers are excellent at detecting spammers, perhaps not so good at detecting other attacks), alternatively, it may just be the size of the associated community or a regional bias.

There are many factors involved, but as with most things, having more data is better. In the case of an edge firewall, more threat intelligence data equals better protection.

Stay tuned for some upcoming articles on combining Threat Intelligence with WAF and nginx.

Updated Apr 06, 2023

Version 2.0