Traffic Intelligence in AFM

Introduction

Traffic Intelligence is a feature created for PEM module years ago, and therefore used typically for integrations into Service Providers. But we can also integrate it with AFM in order to provide more granularity to our firewall policies by including URL classification into our firewall rules. In this way we can, as a typical example, allow Internet traffic from internal company clients while, at the same time, we can control this traffic by filtering it using different parameters like the type of application this traffic belongs to, or the category associated to this traffic.

How to

In order to filtering traffic attending to the category it belongs to, we have two options, using existing categories or creating our own categories. This is a very important difference since in fact for the former option we do need access to Webroot because we will need to download the classification DB (also called im package), whilst the latter option does not require access to this DB.

Regardless above described options we choose, in both cases the flow is the same, let's see it with a simple example:

We work for a company that wants to avoid their employees access to gaming webs, so we create a classification policy to reject this type of traffic.
At some point an employee tries to access to a gaming web during work time.
The request reaches AFM and pass through different internal modules.
At some point packet reaches Global FW rules and packet is matched against its configured rules. If there is not any match then packet moves forward to other modules.
Let's imagine that we have configured a virtual server that allows http/s traffic to Internet from internal clients, and this virtual server has a firewall rule which allows this specific traffic. But also this firewall rule it has been configured for logging and rejecting packets requesting access to gaming websites.
Once packet reaches the firewall policy configured in this virtual server it matches the rule that allows http/s traffic.
Traffic is allowed by commented firewall rule.
Now AFM checks classification policy applied to this rule and it confirms that client request match the configured category (Games).
AFM applies the configured action for this classification policy, it sends specific log to /var/log/ltm and it rejects the packet.

Using webroot categories

For this option to work we need to download the im package:

# tmsh run ltm classification updates

⚠Warning: If our device has not at least 16GB RAM im package will not be downloaded and we will see the following log in /var/log/wr_urldbd.out once we restart wr_urldbd daemon:

nInterprocInit: Zeroed out common structs.
BcSdkInit() setting F5 callback func to 0x4160e0
process_license: webroot_licensed=1; m_wr_licensed=0
process_license: webroot license added/renewed
Module afm is provisioned.
process_provisioning: afm_provisioned=1
Host memory less than 16GB, database will not be downloaded
Marking webroot related process for termination. sig 0

Also we will see Webroot license Absent/Expired when we try to run an URL lookup:

# tmsh show ltm urlcat-query www.facebook.com

---------------------------------------------------------------------
URL : www.facebook.com
---------------------------------------------------------------------
URL-CATEGORIES(Combined) :                             Uncategorized
URL-CATEGORIES(Webroot) :  Skipped - Webroot License Absent/Expired 
URL-CATEGORIES(Custom)  :
URL-CATEGORIES(Cloud)   :                             Uncategorized

Once we have im package downloaded we can start to configure. In our example we will instruct AFM to allow http/s traffic from internal clients to Internet but we will reject traffic to any gaming web. Also we will log in /var/log/ltm when a client tries to access to a gaming web.

In order to do this we need to follow three simple steps:

Create a classification profile. This profile will be the base profile for any classification configuration:

ltm profile classification classifficationProfile {
   app-detection on
   avr-publisher varLogLtm
   avr-stat-collect on
   defaults-from classification
   irule-event on
   log-publisher varLogLtm
   log-unclassified-domain on
   preset ce
   urlcat on
}

*Note that varLogLtm is a Publisher that includes local-syslog as destination for logs since I want to confirm functioning by checking /var/log/ltm logs.

Create a classification policy defining desired actions when traffic includes a request to Game category:

ltm policy ClassificationPolicy {
   controls { classification }
   requires { classification }
   rules {
       R1 {
           actions {
               0 {
                   log
                   classification-detected
                   write
                   facility local0
                   message "An employee tried to access to a gaming website!"
                   priority info
               }
               1 {
                   classification
                   classification-detected
                   reject
               }
           }
           conditions {
               0 {
                   classification
                   classification-detected
                   url-category
                   contains
                   values { Games }
               }
           }
       }
   }
   status published
   strategy best-match
}

Finally we just need to add this classification policy into a firewall rule, for example:

security firewall policy InternetAccess {
   rules {
       R0 {
           action accept
           classification-policy /Common/ClassificationPolicy
           ip-protocol tcp
           log yes
           rule-number 1
       }
[....]

At this point all needed configuration has been done. Now we can try to access from a client to a gaming web and see the result. If we want we can run a URL category lookup first in order to confirm that website belongs to Games category:

# tmsh show ltm urlcat-query gamespot.com

------------------------------------------
URL : gamespot.com
------------------------------------------
URL-CATEGORIES(Combined):          Games
URL-CATEGORIES(Webroot) :          Games
URL-CATEGORIES(Custom)  :
URL-CATEGORIES(Cloud)   :  Uncategorized


# curl gamespot.com
curl: (56) Recv failure: Connection reset by peer

AFM will show that firewall policy is Accepting the traffic, since it is http, but also the subsequent expected log and reject defined in the classification policy:

May 28 06:35:55 bigip1 info tmm[18809]: 23003137 "10.201.198.77","bigip1","Virtual Server","/Common/VS_FWD","No-lookup","10.10.10.10","No-lookup","10.10.20.209","32996","80","/Common/External","TCP","0","","","","","","","","Enforced","/Common/InternetAccess","R0","","Accept","","","","0000000000000000","unknown","unknown","unknown","unknown","","","","","No-lookup","No-lookup"
May 28 06:35:55 bigip1 info tmm1[18809]: [/Common/ClassificationPolicy/R1]: An employee tried to access to a gaming website!                    
May 28 06:35:55 bigip1 info tmm[18809]: 27656193 "27656193","bigip1","10.10.10.10","32996","10.10.20.209","80","/Common/External","","","","","","Unknown","Unknown","TCP","00015657058d4ea0","/Common/VS_FWD","/Common/ClassificationPolicy","/Common/InternetAccess","R0","May 28 2020 06:35:55","reject","","tcp","http","","","","","","","1","Network_Management_and_Services","Network_Management_and_Services","","","","","","","Games","","",""
May 28 06:35:55 bigip1 info tmm[18809]: 27131905 "27131905","bigip1","10.10.10.10","32996","10.10.20.209","80","TCP","/Common/VS_FWD","/Common/classifficationProfile","","","allow","tcp","","","","","","","","1","Network_Management_and_Services","","","","","","","","","","","","331","112","4","","","","","",""

Using own categories

In this case we are going to create our own category and we are going to feed this category with our own URLs. When clients try to access to one these URLs then traffic will be rejected and logged.

In order to do this we need to follow below steps:

Create a new category. Note we have to define an ID between 28672 and 32768:

ltm classification category velasco {
  category-id 28673
}

Create a Category Feed list and associate the above category to it

ltm classification url-category velasco {
  urlcat-id 28673
}

Create a URL DB feed list in a TXT/CSV file with the URLs I am going to test with my client. The format is: URL, category-id:

cat /var/local/pem/velascoFeed.txt
velasco.com, 28673
test.velasco.com, 28673

In above Category Feed list I assign this TXT file. This step is shown in config as below:

ltm classification urldb-feed-list velascoFeed {
  default-url-category velasco
  enabled
  url file:///var/local/pem/velascoFeed.txt           
}

Create a policy: "IF url-category CONTAINS 'velasco' THEN reject and log":

ltm policy ClassificationPolicy2 {
    controls { classification }
    requires { classification }
    rules {
       R1 {
           actions {
               0 {
                   log
                   classification-detected
                   write
                   facility local0
                   message "I am watching you!!!"
                   priority info
               }
               1 {
                   classification
                   classification-detected
                   reject
               }
           }
           conditions {
               0 {
                   classification
                   classification-detected
                   url-category
                   contains
                   values { velasco }
               }
           }
       }
    }
    status published
    strategy best-match
}

No we just need to apply classification policy to firewall rule and try to access from clients as we did before for Webroot categories. We will see in logs how we categorize traffic from clients and apply classification policy action:

tmctl gpa_urlcat_stats
name         count irule customdb wrdb cloud srdb bytes_in bytes_out pkts_in

------------- ----- ----- -------- ---- ----- ---- -------- --------- -------
Uncategorized   56    0       0  52    0   4   47679   589104    731
Unknown          0    0       0   0    0   0       0        0      0
velasco         12    0      12   0    0   0   10834   138432    168

Basic usage

Besides AFM configurations described above we can just use classification profile as described in PEM manuals. Since this is not the intention of this article I will just comment briefly the most simple usage, for more information check related manuals.

Only requirements to control classified traffic is creating a classification profile as described in previous steps and apply it to a virtual server directly, instead to an AFM firewall rule:

ltm virtual VS_FWD {
   destination 0.0.0.0:any
   fw-enforced-policy InternetAccess
   ip-protocol tcp
   mask any
   profiles {
       classifficationProfile {
           context clientside
       }
       tcp { }
   }
   security-log-profiles {
       varLogLtm
   }
   security-nat-policy {
       policy NAT
   }
   serverssl-use-sni disabled
   source 10.10.10.0/24
   translate-address enabled
   translate-port disabled
}

Just with this basic configuration we can start to check classified traffic passing through this virtual server:

# tmctl -w 200 gpa_classification_stats
result            count cec flbl srdb custom bytes_in bytes_out pkts_in pkts_out
------------------ ----- --- ---- ---- ------ -------- --------- ------- --------
tcp                  32 32   0   0     0   10620     3584     96      64
tcp.http             23 23   0   0     0   20746   265328    322     253
tcp.http.actvision    2  2   0   0     0    1810    23072     28      22
tcp.http.adcolony     6  6   0   0     0    5366    69372     83      69
tcp.http.gamespot     2  2   0   0     0    1806    23072     28      22
udp                   0  0   0   0     0       0        0      0       0


# tmctl -w 200 gpa_urlcat_stats
name                count irule customdb wrdb cloud srdb bytes_in bytes_out pkts_in pkts_out
-------------------- ----- ----- -------- ---- ----- ---- -------- --------- ------- --------
Advertisements          6    0       0   6    0   0    5366    69372     83      69
Business_and_Economy   22    0       0   0   22   0   19844   253792    308     242
Games                   2    0       0   2    0   0    1806    23072     28      22
Uncategorized           3    0       0   3    0   0    2712    34608     42      33
Unknown                 0    0       0   0    0   0       0        0      0       0