Declarative Advanced WAF policy lifecycle in a CI/CD pipeline
The purpose of this article is to show the configuration used to deploy a declarative Advanced WAF policy to a BIG-IP and automatically configure it to protect an API workload by consuming an OpenAPI file describing the application.
For this experiment, a Gitlab CI/CD pipeline was used to deploy an API workload to Kubernetes, configure a declarative Adv. WAF policy to a BIG-IP device and tuning it by incorporating learning suggestions exported from the BIG-IP. Lastly, the F5 WAF tester tool was used to determine and improve the defensive posture of the Adv. WAF policy.
Deploying the declarative Advanced WAF policy through a CI/CD pipeline
To deploy the Adv. WAF policy, the Gitlab CI/CD pipeline is calling an Ansible playbook that will in turn deploy an AS3 application referencing the Adv.WAF policy from a separate JSON file. This allows the application definition and WAF policy to be managed by 2 different groups, for example NetOps and SecOps, supporting separation of duties.
The following Ansible playbook was used;
---
- hosts: bigip
connection: local
gather_facts: false
vars:
my_admin: "xxxx"
my_password: "xxxx"
bigip: "xxxx"
tasks:
- name: Deploy AS3 API AWAF policy
uri:
url: "https://{{ bigip }}/mgmt/shared/appsvcs/declare"
method: POST
headers:
"Content-Type": "application/json"
"Authorization": "Basic xxxxxxxxxx
body: "{{ lookup('file','as3_waf_openapi.json') }}"
body_format: json
validate_certs: no
status_code: 200
The Advanced WAF policy 'as3_waf_openapi.json' was specified as follows:
{
"class": "AS3",
"action": "deploy",
"persist": true,
"declaration": {
"class": "ADC",
"schemaVersion": "3.2.0",
"id": "Prod_API_AS3",
"API-Prod": {
"class": "Tenant",
"defaultRouteDomain": 0,
"arcadia": {
"class": "Application",
"template": "generic",
"VS_API": {
"class": "Service_HTTPS",
"remark": "Accepts HTTPS/TLS connections on port 443",
"virtualAddresses": ["xxxxx"],
"redirect80": false,
"pool": "pool_NGINX_API",
"policyWAF": {
"use": "Arcadia_WAF_API_policy"
},
"securityLogProfiles": [{
"bigip": "/Common/Log all requests"
}],
"profileTCP": {
"egress": "wan",
"ingress": { "use": "TCP_Profile" } },
"profileHTTP": { "use": "custom_http_profile" },
"serverTLS": { "bigip": "/Common/arcadia_client_ssl" }
},
"Arcadia_WAF_API_policy": {
"class": "WAF_Policy",
"url": "http://xxxx/root/awaf_openapi/-/raw/master/WAF/ansible/bigip/policy-api.json",
"ignoreChanges": true
},
"pool_NGINX_API": {
"class": "Pool",
"monitors": ["http"],
"members": [{
"servicePort": 8080,
"serverAddresses": ["xxxx"]
}]
},
"custom_http_profile": {
"class": "HTTP_Profile",
"xForwardedFor": true
},
"TCP_Profile": {
"class": "TCP_Profile",
"idleTimeout": 60 }
}
}
}
}
The AS3 declaration will provision a separate Administrative Partition ('API-Prod') containing a Virtual Server ('VS_API'), an Adv. WAF policy ('Arcadia_WAF_API_policy') and a pool ('pool_NGINX_API'). The Adv.WAF policy being referenced ('policy-api.json') is stored in the same Gitlab repository but can be downloaded from a separate location.
{
"policy": {
"name": "policy-api-arcadia",
"description": "Arcadia API",
"template": {
"name": "POLICY_TEMPLATE_API_SECURITY"
},
"enforcementMode": "transparent",
"server-technologies": [
{
"serverTechnologyName": "MySQL"
},
{
"serverTechnologyName": "Unix/Linux"
},
{
"serverTechnologyName": "MongoDB"
}
],
"signature-settings": {
"signatureStaging": false
},
"policy-builder": {
"learnOnlyFromNonBotTraffic": false
},
"open-api-files": [
{
"link": "http://xxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml"
}
]
},
"modifications": [
]
}
The declarative Adv.WAF policy is referencing in turn the OpenAPI file ('openapi3-arcadia.yaml') that describes the application being protected.
Executing the Ansible playbook results in the AS3 application being deployed, along with the Adv.WAF policy that is automatically configured according to the OpenAPI file.
Handling learning suggestions in a CI/CD pipeline
The next step in the CI/CD pipeline used for this experiment was to send legitimate traffic using the API and collect the learning suggestions generated by the Adv.WAF policy, which will allow a simple way to customize the WAF policy further for the specific application being protected.
The following Ansible playbook was used to retrieve the learning suggestions:
---
- hosts: bigip
connection: local
gather_facts: true
vars:
my_admin: "xxxx"
my_password: "xxxx"
bigip: "xxxxx"
tasks:
- name: Get all Policy_key/IDs for WAF policies
uri:
url: 'https://{{ bigip }}/mgmt/tm/asm/policies?$select=name,id'
method: GET
headers:
"Authorization": "Basic xxxxxxxxxxx"
validate_certs: no
status_code: 200
return_content: yes
register: waf_policies
- name: Extract Policy_key/ID of Arcadia_WAF_API_policy
set_fact: Arcadia_WAF_API_policy_ID="{{ item.id }}"
loop: "{{ (waf_policies.content|from_json)['items'] }}"
when: item.name == "Arcadia_WAF_API_policy"
- name: Export learning suggestions
uri:
url: "https://{{ bigip }}/mgmt/tm/asm/tasks/export-suggestions"
method: POST
headers:
"Content-Type": "application/json"
"Authorization": "Basic xxxxxxxxxxx"
body: "{ \"inline\": \"true\", \"policyReference\": { \"link\": \"https://{{ bigip }}/mgmt/tm/asm/policies/{{ Arcadia_WAF_API_policy_ID }}/\" } }"
body_format: json
validate_certs: no
status_code:
- 200
- 201
- 202
- name: Get learning suggestions
uri:
url: "https://{{ bigip }}/mgmt/tm/asm/tasks/export-suggestions"
method: GET
headers:
"Authorization": "Basic xxxxxxxxx"
validate_certs: no
status_code: 200
register: result
- name: Print learning suggestions
debug: var=result
A sample learning suggestions output is shown below:
"json": {
"items": [
{
"endTime": "xxxxxxxxxxxxx",
"id": "ZQDaRVecGeqHwAW1LDzZTQ",
"inline": true,
"kind": "tm:asm:tasks:export-suggestions:export-suggestions-taskstate",
"lastUpdateMicros": 1599953296000000.0,
"result": {
"suggestions": [
{
"action": "add-or-update",
"description": "Enable Evasion Technique",
"entity": {
"description": "Directory traversals"
},
"entityChanges": {
"enabled": true
},
"entityType": "evasion"
},
{
"action": "add-or-update",
"description": "Enable HTTP Check",
"entity": {
"description": "Check maximum number of parameters"
},
"entityChanges": {
"enabled": true
},
"entityType": "http-protocol"
},
{
"action": "add-or-update",
"description": "Enable HTTP Check",
"entity": {
"description": "No Host header in HTTP/1.1 request"
},
"entityChanges": {
"enabled": true
},
"entityType": "http-protocol"
},
{
"action": "add-or-update",
"description": "Enable enforcement of policy violation",
"entity": {
"name": "VIOL_REQUEST_MAX_LENGTH"
},
"entityChanges": {
"alarm": true,
"block": true
},
"entityType": "violation"
}
Incorporating the learning suggestions in the Adv.WAF policy can be done by simple copy&pasting the self-contained learning suggestions blocks into the "modifications" list of the Adv.WAF policy:
{
"policy": {
"name": "policy-api-arcadia",
"description": "Arcadia API",
"template": {
"name": "POLICY_TEMPLATE_API_SECURITY"
},
"enforcementMode": "transparent",
"server-technologies": [
{
"serverTechnologyName": "MySQL"
},
{
"serverTechnologyName": "Unix/Linux"
},
{
"serverTechnologyName": "MongoDB"
}
],
"signature-settings": {
"signatureStaging": false
},
"policy-builder": {
"learnOnlyFromNonBotTraffic": false
},
"open-api-files": [
{
"link": "http://xxxxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml"
}
]
},
"modifications": [
{
"action": "add-or-update",
"description": "Enable Evasion Technique",
"entity": {
"description": "Directory traversals"
},
"entityChanges": {
"enabled": true
},
"entityType": "evasion"
}
]
}
Enhancing Advanced WAF policy posture by using the F5 WAF tester
The F5 WAF tester is a tool that generates known attacks and checks the response of the WAF policy. For example, running the F5 WAF tester against a policy that has a "transparent" enforcement mode will cause the tests to fail as the attacks will not be blocked. The F5 WAF tester can suggest possible enhancement of the policy, in this case the change of the enforcement mode. An abbreviated sample output of the F5 WAF Tester:
................................................................
"100000023": {
"CVE": "",
"attack_type": "Server Side Request Forgery",
"name": "SSRF attempt (AWS Metadata Server)",
"results": {
"parameter": {
"expected_result": {
"type": "signature",
"value": "200018040"
},
"pass": false,
"reason": "ASM Policy is not in blocking mode",
"support_id": ""
}
},
"system": "All systems"
},
"100000024": {
"CVE": "",
"attack_type": "Server Side Request Forgery",
"name": "SSRF attempt - Local network IP range 10.x.x.x",
"results": {
"request": {
"expected_result": {
"type": "signature",
"value": "200020201"
},
"pass": false,
"reason": "ASM Policy is not in blocking mode",
"support_id": ""
}
},
"system": "All systems"
}
},
"summary": {
"fail": 48,
"pass": 0
}
Changing the enforcement mode from "transparent" to "blocking" can easily be done by editing the same Adv. WAF policy file:
{
"policy": {
"name": "policy-api-arcadia",
"description": "Arcadia API",
"template": {
"name": "POLICY_TEMPLATE_API_SECURITY"
},
"enforcementMode": "blocking",
"server-technologies": [
{
"serverTechnologyName": "MySQL"
},
{
"serverTechnologyName": "Unix/Linux"
},
{
"serverTechnologyName": "MongoDB"
}
],
"signature-settings": {
"signatureStaging": false
},
"policy-builder": {
"learnOnlyFromNonBotTraffic": false
},
"open-api-files": [
{
"link": "http://xxxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml"
}
]
},
"modifications": [
{
"action": "add-or-update",
"description": "Enable Evasion Technique",
"entity": {
"description": "Directory traversals"
},
"entityChanges": {
"enabled": true
},
"entityType": "evasion"
}
]
}
A successful run will will be achieved when all the attacks will be blocked.
.........................................
"100000023": {
"CVE": "",
"attack_type": "Server Side Request Forgery",
"name": "SSRF attempt (AWS Metadata Server)",
"results": {
"parameter": {
"expected_result": {
"type": "signature",
"value": "200018040"
},
"pass": true,
"reason": "",
"support_id": "17540898289451273964"
}
},
"system": "All systems"
},
"100000024": {
"CVE": "",
"attack_type": "Server Side Request Forgery",
"name": "SSRF attempt - Local network IP range 10.x.x.x",
"results": {
"request": {
"expected_result": {
"type": "signature",
"value": "200020201"
},
"pass": true,
"reason": "",
"support_id": "17540898289451274344"
}
},
"system": "All systems"
}
},
"summary": {
"fail": 0,
"pass": 48
}
Conclusion
By adding the Advanced WAF policy into a CI/CD pipeline, the WAF policy can be integrated in the lifecycle of the application it is protecting, allowing for continuous testing and improvement of the security posture before it is deployed to production. The flexible model of AS3 and declarative Advanced WAF allows the separation of roles and responsibilities between NetOps and SecOps, while providing an easy way for tuning the policy to the specifics of the application being protected.
Links
UDF lab environment link.
Short instructional video link.
Published Sep 22, 2020
Version 1.0
Valentin_Tobi
Employee
EmployeeValentin_TobiEmployee
Employee- Nikoolayy1
MVP
Great article! I have made an article https://community.f5.com/t5/codeshare/comparison-between-deploying-as3-or-fast-iapp-declarations-with/ta-p/309613 , where for AS3 deployments I use the new F5 ansible module "bigip_as3_deploy" that does some better checks than the Ansbile build in URI module.
For not referencing a policy from Github but including it in the as3 file directly or as a a variable, using Ansible facts I just need to replace "url": "http://xxxx/root/awaf_openapi/-/raw/master/WAF/ansible/bigip/policy-api.json" and directly place the file content seems not possible as of now so probably the only other way is to push a file with the policy as mentioned in https://community.f5.com/t5/technical-articles/advanced-waf-v16-0-declarative-api/tac-p/309802#M13890 , make a policy from it and then reference the existing policy.
B_EarpAltocumulus
Using the uri option in the declaration it is unclear how the policy arrives at the BIG-IP.
"My_ASM_Policy": { "class": "WAF_Policy", "url": "https://example.com/asm-policy.xml", "ignoreChanges": trueHow does the data from that url get uploaded/downloaded to the big-ip.
1. PUSH: Does the AS3 software include the json data from that url as part of it's declaration?
2. PULL: Does the BIG-IP download the json from that url?
