Advanced WAF v16.0 - Declarative API
Since v15.1 (in draft), F5® BIG-IP® Advanced WAF™ canimport Declarative WAF policy in JSON format. The F5® BIG-IP® Advanced Web Application Firewall (Advanced WAF) security policies can be deployed using the declarative JSON format, facilitating easy integration into a CI/CD pipeline. The declarative policies are extracted from a source control system, for example Git, and imported into the BIG-IP. Using the provided declarative policy templates, you can modify the necessary parameters, save the JSON file, and import the updated security policy into your BIG-IP devices. The declarative policy copies the content of the template and adds the adjustments and modifications on to it. The templates therefore allow you to concentrate only on the specific settings that need to be adapted for the specific application that the policy protects. ThisDeclarative WAF JSON policyis similar toNGINX App Protect policy. You can find more information on theDeclarative Policyhere : NAP :https://docs.nginx.com/nginx-app-protect/policy/ Adv. WAF :https://techdocs.f5.com/en-us/bigip-15-1-0/big-ip-declarative-security-policy.html Audience This guide is written for IT professionals who need to automate their WAF policy and are familiar with Advanced WAF configuration. These IT professionals can fill a variety of roles: SecOps deploying and maintaining WAF policy in Advanced WAF DevOps deploying applications in modern environment and willing to integrate Advanced WAF in their CI/CD pipeline F5 partners who sell technology or create implementation documentation This article covershow to PUSH/PULL a declarative WAF policy in Advanced WAF: With Postman With AS3 Table of contents Upload Policy in BIG-IP Check the import Apply the policy OpenAPI Spec File import AS3 declaration CI/CD integration Find the Policy-ID Update an existing policy Video demonstration First of all, you need aJSON WAF policy, as below : { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false } } } 1. Upload Policy in BIG-IP There are 2 options to upload a JSON file into the BIG-IP: 1.1 Either youPUSHthe file into the BIG-IP and you IMPORT IT OR 1.2 the BIG-IPPULLthe file froma repository (and the IMPORT is included)<- BEST option 1.1PUSH JSON file into the BIG-IP The call is below. As you can notice, it requires a 'Content-Range' header. And the value is 0-(filesize-1)/filesize. In the example below, the file size is 662 bytes. This is not easy to integrate in a CICD pipeline, so we created the PULL method instead of the PUSH (in v16.0) curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/file-transfer/uploads/policy-api.json' \ --header 'Content-Range: 0-661/662' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --header 'Content-Type: application/json' \ --data-binary '@/C:/Users/user/Desktop/policy-api.json' At this stage,the policy is still a filein the BIG-IP file system. We need toimportit into Adv. WAF. To do so, the next call is required. This call import the file "policy-api.json" uploaded previously. AnCREATEthe policy /Common/policy-api-arcadia curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/javascript' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "filename":"policy-api.json", "policy": { "fullPath":"/Common/policy-api-arcadia" } }' 1.2PULL JSON file from a repository Here, theJSON file is hosted somewhere(in Gitlab or Github ...). And theBIG-IP will pull it. The call is below. As you can notice, the call refers to the remote repo and the body is a JSON payload. Just change the link value with your JSON policy URL. With one call, the policy isPULLEDandIMPORTED. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" } }' Asecond versionof this call exists, and refer to the fullPath of the policy.This will allow you to update the policy, from a second version of the JSON file, easily.One call for the creation and the update. As you can notice below, we add the"policy":"fullPath" directive. The value of the "fullPath" is thepartitionand thename of the policyset in the JSON policy file. This method is VERY USEFUL for CI/CD integrations. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" }, "policy": { "fullPath":"/Common/policy-api-arcadia" } }' 2. Check the IMPORT Check if the IMPORT worked. To do so, run the next call. curl --location --request GET 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ You should see a 200 OK, with the content below (truncated in this example). Please notice the"status":"COMPLETED". { "kind": "tm:asm:tasks:import-policy:import-policy-taskcollectionstate", "selfLink": "https://localhost/mgmt/tm/asm/tasks/import-policy?ver=16.0.0", "totalItems": 11, "items": [ { "isBase64": false, "executionStartTime": "2020-07-21T15:50:22Z", "status": "COMPLETED", "lastUpdateMicros": 1.595346627e+15, "getPolicyAttributesOnly": false, ... From now, your policy is imported and created in the BIG-IP. You can assign it to a VS as usual (Imperative Call or AS3 Call).But in the next session, I will show you how to create a Service with AS3 including the WAF policy. 3. APPLY the policy As you may know, a WAF policy needs to be applied after each change. This is the call. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/apply-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{"policy":{"fullPath":"/Common/policy-api-arcadia"}}' 4. OpenAPI spec file IMPORT As you know,Adv. WAF supports OpenAPI spec (2.0 and 3.0). Now, with the declarative WAF, we can import the OAS file as well. The BEST solution, is toPULL the OAS filefrom a repo. And in most of the customer' projects, it will be the case. In the example below, the OAS file is hosted in SwaggerHub(Github for Swagger files). But the file could reside in a private Gitlab repo for instance. The URL of the projectis :https://app.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3 The URL of the OAS file is :https://api.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3 This swagger file (OpenAPI 3.0 Spec file) includes all the application URL and parameters. What's more, it includes the documentation (for NGINX APIm Dev Portal). Now, it ispretty easy to create a WAF JSON Policy with API Security template, referring to the OAS file. Below, you can notice thenew section "open-api-files"with the link reference to SwaggerHub. And thenew templatePOLICY_TEMPLATE_API_SECURITY. Now, when I upload / import and apply the policy, Adv. WAF will download the OAS file from SwaggerHub and create the policy based on API_Security template. { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "https://api.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3" } ] } } 5. AS3 declaration Now, it is time to learn how we cando all of these steps in one call with AS3(3.18 minimum). The documentation is here :https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/declarations/application-security.html?highlight=waf_policy#virtual-service-referencing-an-external-security-policy With thisAS3 declaration, we: Import the WAF policy from a external repo Import the Swagger file (if the WAF policy refers to an OAS file) from an external repo Create the service { "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.2.0", "id": "Prod_API_AS3", "API-Prod": { "class": "Tenant", "defaultRouteDomain": 0, "API": { "class": "Application", "template": "generic", "VS_API": { "class": "Service_HTTPS", "remark": "Accepts HTTPS/TLS connections on port 443", "virtualAddresses": ["10.1.10.27"], "redirect80": false, "pool": "pool_NGINX_API_AS3", "policyWAF": { "use": "Arcadia_WAF_API_policy" }, "securityLogProfiles": [{ "bigip": "/Common/Log all requests" }], "profileTCP": { "egress": "wan", "ingress": { "use": "TCP_Profile" } }, "profileHTTP": { "use": "custom_http_profile" }, "serverTLS": { "bigip": "/Common/arcadia_client_ssl" } }, "Arcadia_WAF_API_policy": { "class": "WAF_Policy", "url": "http://10.1.20.4/root/as3-waf-api/-/raw/master/policy-api.json", "ignoreChanges": true }, "pool_NGINX_API_AS3": { "class": "Pool", "monitors": ["http"], "members": [{ "servicePort": 8080, "serverAddresses": ["10.1.20.9"] }] }, "custom_http_profile": { "class": "HTTP_Profile", "xForwardedFor": true }, "TCP_Profile": { "class": "TCP_Profile", "idleTimeout": 60 } } } } } 6. CI/CID integration As you can notice, it is very easy to create a service with a WAF policy pulled from an external repo. So, it is easy to integrate these calls (or the AS3 call) into a CI/CD pipeline. Below, an Ansible playbook example. This playbook run the AS3 call above. That's it :) --- - hosts: bigip connection: local gather_facts: false vars: my_admin: "admin" my_password: "admin" bigip: "10.1.1.12" tasks: - name: Deploy AS3 WebApp uri: url: "https://{{ bigip }}/mgmt/shared/appsvcs/declare" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic YWRtaW46YWRtaW4=" body: "{{ lookup('file','as3.json') }}" body_format: json validate_certs: no status_code: 200 7. FIND the Policy-ID When the policy is created, a Policy-ID is assigned. By default, this ID doesn't appearanywhere. Neither in the GUI, nor in the response after the creation. You have to calculate it or ask for it. This ID is required for several actions in a CI/CD pipeline. 7.1 Calculate the Policy-ID Wecreated this python script to calculate the Policy-ID. It is an hash from the Policy name (including the partition). For the previous created policy named"/Common/policy-api-arcadia",the policy ID is"Ar5wrwmFRroUYsMA6DuxlQ" Paste this python codein a newwaf-policy-id.pyfile, and run the commandpython waf-policy-id.py "/Common/policy-api-arcadia" Outcome will beThe Policy-ID for /Common/policy-api-arcadia is: Ar5wrwmFRroUYsMA6DuxlQ #!/usr/bin/python from hashlib import md5 import base64 import sys pname = sys.argv[1] print 'The Policy-ID for', sys.argv[1], 'is:', base64.b64encode(md5(pname.encode()).digest()).replace("=", "") 7.2 Retrieve the Policy-ID and fullPath with a REST API call Make this call below, and you will see in the response, all the policy creations. Find yours and collect thePolicyReference directive.The Policy-ID is in the link value "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0" You can see as well, at the end of the definition, the "fileReference"referring to the JSON file pulled by the BIG-IP. And please notice the"fullPath", required if you want to update your policy curl --location --request GET 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Range: 0-601/601' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ { "isBase64": false, "executionStartTime": "2020-07-22T11:23:42Z", "status": "COMPLETED", "lastUpdateMicros": 1.595417027e+15, "getPolicyAttributesOnly": false, "kind": "tm:asm:tasks:import-policy:import-policy-taskstate", "selfLink": "https://localhost/mgmt/tm/asm/tasks/import-policy/B45J0ySjSJ9y9fsPZ2JNvA?ver=16.0.0", "filename": "", "policyReference": { "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0", "fullPath": "/Common/policy-api-arcadia" }, "endTime": "2020-07-22T11:23:47Z", "startTime": "2020-07-22T11:23:42Z", "id": "B45J0ySjSJ9y9fsPZ2JNvA", "retainInheritanceSettings": false, "result": { "policyReference": { "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0", "fullPath": "/Common/policy-api-arcadia" }, "message": "The operation was completed successfully. The security policy name is '/Common/policy-api-arcadia'. " }, "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" } }, 8 UPDATE an existing policy It is pretty easy to update the WAF policy from a new JSON file version. To do so, collect from the previous call7.2 Retrieve the Policy-ID and fullPath with a REST API callthe"Policy" and"fullPath"directive. This is the path of the Policy in the BIG-IP. Then run the call below, same as1.2 PULL JSON file from a repository,but add thePolicy and fullPath directives Don't forget to APPLY this new version of the policy3. APPLY the policy curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" }, "policy": { "fullPath":"/Common/policy-api-arcadia" } }' TIP : this call, above, can be used in place of the FIRST call when we created the policy "1.2PULL JSON file from a repository". But be careful, the fullPath is the name set in the JSON policy file. The 2 values need to match: "name": "policy-api-arcadia" in the JSON Policy file pulled by the BIG-IP "policy":"fullPath" in the POST call 9 Video demonstration In order to help you to understand how it looks with the BIG-IP, I created this video covering 4 topics explained in this article : The JSON WAF policy Pull the policy from a remote repository Update the WAF policy with a new version of the declarative JSON file Deploy a full service with AS3 and Declarative WAF policy At the end of this video, you will be able to adapt the REST Declarative API calls to your infrastructure, in order to deploy protected services with your CI/CD pipelines. Direct link to the video on DevCentral YouTube channel : https://youtu.be/EDvVwlwEFRwDeclarative Advanced WAF policy lifecycle in a CI/CD pipeline
The purpose of this article is to show the configuration used to deploy a declarative Advanced WAF policy to a BIG-IP and automatically configure it to protect an API workload by consuming an OpenAPI file describing the application. For this experiment, a Gitlab CI/CD pipeline was used to deploy an API workload to Kubernetes, configure a declarative Adv. WAF policy to a BIG-IP device and tuning it by incorporating learning suggestions exported from the BIG-IP. Lastly, the F5 WAF tester tool was used to determine and improve the defensive posture of the Adv. WAF policy. Deploying the declarative Advanced WAF policy through a CI/CD pipeline To deploy the Adv. WAF policy, the Gitlab CI/CD pipeline is calling an Ansible playbook that will in turn deploy an AS3 application referencing the Adv.WAF policy from a separate JSON file. This allows the application definition and WAF policy to be managed by 2 different groups, for example NetOps and SecOps, supporting separation of duties. The following Ansible playbook was used; --- - hosts: bigip connection: local gather_facts: false vars: my_admin: "xxxx" my_password: "xxxx" bigip: "xxxx" tasks: - name: Deploy AS3 API AWAF policy uri: url: "https://{{ bigip }}/mgmt/shared/appsvcs/declare" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic xxxxxxxxxx body: "{{ lookup('file','as3_waf_openapi.json') }}" body_format: json validate_certs: no status_code: 200 The Advanced WAF policy 'as3_waf_openapi.json' was specified as follows: { "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.2.0", "id": "Prod_API_AS3", "API-Prod": { "class": "Tenant", "defaultRouteDomain": 0, "arcadia": { "class": "Application", "template": "generic", "VS_API": { "class": "Service_HTTPS", "remark": "Accepts HTTPS/TLS connections on port 443", "virtualAddresses": ["xxxxx"], "redirect80": false, "pool": "pool_NGINX_API", "policyWAF": { "use": "Arcadia_WAF_API_policy" }, "securityLogProfiles": [{ "bigip": "/Common/Log all requests" }], "profileTCP": { "egress": "wan", "ingress": { "use": "TCP_Profile" } }, "profileHTTP": { "use": "custom_http_profile" }, "serverTLS": { "bigip": "/Common/arcadia_client_ssl" } }, "Arcadia_WAF_API_policy": { "class": "WAF_Policy", "url": "http://xxxx/root/awaf_openapi/-/raw/master/WAF/ansible/bigip/policy-api.json", "ignoreChanges": true }, "pool_NGINX_API": { "class": "Pool", "monitors": ["http"], "members": [{ "servicePort": 8080, "serverAddresses": ["xxxx"] }] }, "custom_http_profile": { "class": "HTTP_Profile", "xForwardedFor": true }, "TCP_Profile": { "class": "TCP_Profile", "idleTimeout": 60 } } } } } The AS3 declaration will provision a separate Administrative Partition ('API-Prod') containing a Virtual Server ('VS_API'), an Adv. WAF policy ('Arcadia_WAF_API_policy') and a pool ('pool_NGINX_API'). The Adv.WAF policy being referenced ('policy-api.json') is stored in the same Gitlab repository but can be downloaded from a separate location. { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "transparent", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "http://xxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml" } ] }, "modifications": [ ] } The declarative Adv.WAF policy is referencing in turn the OpenAPI file ('openapi3-arcadia.yaml') that describes the application being protected. Executing the Ansible playbook results in the AS3 application being deployed, along with the Adv.WAF policy that is automatically configured according to the OpenAPI file. Handling learning suggestions in a CI/CD pipeline The next step in the CI/CD pipeline used for this experiment was to send legitimate traffic using the API and collect the learning suggestions generated by the Adv.WAF policy, which will allow a simple way to customize the WAF policy further for the specific application being protected. The following Ansible playbook was used to retrieve the learning suggestions: --- - hosts: bigip connection: local gather_facts: true vars: my_admin: "xxxx" my_password: "xxxx" bigip: "xxxxx" tasks: - name: Get all Policy_key/IDs for WAF policies uri: url: 'https://{{ bigip }}/mgmt/tm/asm/policies?$select=name,id' method: GET headers: "Authorization": "Basic xxxxxxxxxxx" validate_certs: no status_code: 200 return_content: yes register: waf_policies - name: Extract Policy_key/ID of Arcadia_WAF_API_policy set_fact: Arcadia_WAF_API_policy_ID="{{ item.id }}" loop: "{{ (waf_policies.content|from_json)['items'] }}" when: item.name == "Arcadia_WAF_API_policy" - name: Export learning suggestions uri: url: "https://{{ bigip }}/mgmt/tm/asm/tasks/export-suggestions" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic xxxxxxxxxxx" body: "{ \"inline\": \"true\", \"policyReference\": { \"link\": \"https://{{ bigip }}/mgmt/tm/asm/policies/{{ Arcadia_WAF_API_policy_ID }}/\" } }" body_format: json validate_certs: no status_code: - 200 - 201 - 202 - name: Get learning suggestions uri: url: "https://{{ bigip }}/mgmt/tm/asm/tasks/export-suggestions" method: GET headers: "Authorization": "Basic xxxxxxxxx" validate_certs: no status_code: 200 register: result - name: Print learning suggestions debug: var=result A sample learning suggestions output is shown below: "json": { "items": [ { "endTime": "xxxxxxxxxxxxx", "id": "ZQDaRVecGeqHwAW1LDzZTQ", "inline": true, "kind": "tm:asm:tasks:export-suggestions:export-suggestions-taskstate", "lastUpdateMicros": 1599953296000000.0, "result": { "suggestions": [ { "action": "add-or-update", "description": "Enable Evasion Technique", "entity": { "description": "Directory traversals" }, "entityChanges": { "enabled": true }, "entityType": "evasion" }, { "action": "add-or-update", "description": "Enable HTTP Check", "entity": { "description": "Check maximum number of parameters" }, "entityChanges": { "enabled": true }, "entityType": "http-protocol" }, { "action": "add-or-update", "description": "Enable HTTP Check", "entity": { "description": "No Host header in HTTP/1.1 request" }, "entityChanges": { "enabled": true }, "entityType": "http-protocol" }, { "action": "add-or-update", "description": "Enable enforcement of policy violation", "entity": { "name": "VIOL_REQUEST_MAX_LENGTH" }, "entityChanges": { "alarm": true, "block": true }, "entityType": "violation" } Incorporating the learning suggestions in the Adv.WAF policy can be done by simple copy&pasting the self-contained learning suggestions blocks into the "modifications" list of the Adv.WAF policy: { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "transparent", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "http://xxxxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml" } ] }, "modifications": [ { "action": "add-or-update", "description": "Enable Evasion Technique", "entity": { "description": "Directory traversals" }, "entityChanges": { "enabled": true }, "entityType": "evasion" } ] } Enhancing Advanced WAF policy posture by using the F5 WAF tester The F5 WAF tester is a tool that generates known attacks and checks the response of the WAF policy. For example, running the F5 WAF tester against a policy that has a "transparent" enforcement mode will cause the tests to fail as the attacks will not be blocked. The F5 WAF tester can suggest possible enhancement of the policy, in this case the change of the enforcement mode. An abbreviated sample output of the F5 WAF Tester: ................................................................ "100000023": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt (AWS Metadata Server)", "results": { "parameter": { "expected_result": { "type": "signature", "value": "200018040" }, "pass": false, "reason": "ASM Policy is not in blocking mode", "support_id": "" } }, "system": "All systems" }, "100000024": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt - Local network IP range 10.x.x.x", "results": { "request": { "expected_result": { "type": "signature", "value": "200020201" }, "pass": false, "reason": "ASM Policy is not in blocking mode", "support_id": "" } }, "system": "All systems" } }, "summary": { "fail": 48, "pass": 0 } Changing the enforcement mode from "transparent" to "blocking" can easily be done by editing the same Adv. WAF policy file: { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "http://xxxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml" } ] }, "modifications": [ { "action": "add-or-update", "description": "Enable Evasion Technique", "entity": { "description": "Directory traversals" }, "entityChanges": { "enabled": true }, "entityType": "evasion" } ] } A successful run will will be achieved when all the attacks will be blocked. ......................................... "100000023": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt (AWS Metadata Server)", "results": { "parameter": { "expected_result": { "type": "signature", "value": "200018040" }, "pass": true, "reason": "", "support_id": "17540898289451273964" } }, "system": "All systems" }, "100000024": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt - Local network IP range 10.x.x.x", "results": { "request": { "expected_result": { "type": "signature", "value": "200020201" }, "pass": true, "reason": "", "support_id": "17540898289451274344" } }, "system": "All systems" } }, "summary": { "fail": 0, "pass": 48 } Conclusion By adding the Advanced WAF policy into a CI/CD pipeline, the WAF policy can be integrated in the lifecycle of the application it is protecting, allowing for continuous testing and improvement of the security posture before it is deployed to production. The flexible model of AS3 and declarative Advanced WAF allows the separation of roles and responsibilities between NetOps and SecOps, while providing an easy way for tuning the policy to the specifics of the application being protected. Links UDF lab environment link. Short instructional video link.
F5 Predicts: Education gets personal
The topic of education is taking centre stage today like never before. I think we can all agree that education has come a long way from the days where students and teachers were confined to a classroom with a chalkboard. Technology now underpins virtually every sector and education is no exception. The Internet is now the principal enabling mechanism by which students assemble, spread ideas and sow economic opportunities. Education data has become a hot topic in a quest to transform the manner in which students learn. According to Steven Ross, a professor at the Centre for Research and Reform in Education at Johns Hopkins University, the use of data to customise education for students will be the key driver for learning in the future[1].This technological revolution has resulted in a surge of online learning courses accessible to anyone with a smart device. A two-year assessment of the massive open online courses (MOOCs) created by HarvardX and MITxrevealed that there were 1.7 million course entries in the 68 MOOC [2].This translates to about 1 million unique participants, who on average engage with 1.7 courses each. This equity of education is undoubtedly providing vast opportunities for students around the globe and improving their access to education. With more than half a million apps to choose from on different platforms such as the iOS and Android, both teachers and students can obtain digital resources on any subject. As education progresses in the digital era, here are some considerations for educational institutions to consider: Scale and security The emergence of a smogasborad of MOOC providers, such as Coursera and edX, have challenged the traditional, geographical and technological boundaries of education today. Digital learning will continue to grow driving the demand for seamless and user friendly learning environments. In addition, technological advancements in education offers new opportunities for government and enterprises. It will be most effective if provided these organisations have the ability to rapidly scale and adapt to an all new digital world – having information services easily available, accessible and secured. Many educational institutions have just as many users as those in large multinational corporations and are faced with the issue of scale when delivering applications. The aim now is no longer about how to get fast connection for students, but how quickly content can be provisioned and served and how seamless the user experience can be. No longer can traditional methods provide our customers with the horizontal scaling needed. They require an intelligent and flexible framework to deploy and manage applications and resources. Hence, having an application-centric infrastructure in place to accelerate the roll-out of curriculum to its user base, is critical in addition to securing user access and traffic in the overall environment. Ensuring connectivity We live in a Gen-Y world that demands a high level of convenience and speed from practically everyone and anything. This demand for convenience has brought about reform and revolutionised the way education is delivered to students. Furthermore, the Internet of things (IoT), has introduced a whole new raft of ways in which teachers can educate their students. Whether teaching and learning is via connected devices such as a Smart Board or iPad, seamless access to data and content have never been more pertinent than now. With the increasing reliance on Internet bandwidth, textbooks are no longer the primary means of educating, given that students are becoming more web oriented. The shift helps educational institutes to better personalise the curriculum based on data garnered from students and their work. Duty of care As the cloud continues to test and transform the realms of education around the world, educational institutions are opting for a centralised services model, where they can easily select the services they want delivered to students to enhance their learning experience. Hence, educational institutions have a duty of care around the type of content accessed and how it is obtained by students. They can enforce acceptable use policies by only delivering content that is useful to the curriculum, with strong user identification and access policies in place. By securing the app, malware and viruses can be mitigated from the institute’s environment. From an outbound perspective, educators can be assured that students are only getting the content they are meant to get access to. F5 has the answer BIG-IP LTM acts as the bedrock for educational organisations to provision, optimise and deliver its services. It provides the ability to publish applications out to the Internet in a quickly and timely manner within a controlled and secured environment. F5 crucially provides both the performance and the horizontal scaling required to meet the highest levels of throughput. At the same time, BIG-IP APM provides schools with the ability to leverage virtual desktop infrastructure (VDI) applications downstream, scale up and down and not have to install costly VDI gateways on site, whilst centralising the security decisions that come with it. As part of this, custom iApps can be developed to rapidly and consistently deliver, as well as reconfigure the applications that are published out to the Internet in a secure, seamless and manageable way. BIG-IP Application Security Manager (ASM) provides an application layer security to protect vital educational assets, as well as the applications and content being continuously published. ASM allows educational institutes to tailor security profiles that fit like a glove to wrap seamlessly around every application. It also gives a level of assurance that all applications are delivered in a secure manner. Education tomorrow It is hard not to feel the profound impact that technology has on education. Technology in the digital era has created a new level of personalised learning. The time is ripe for the digitisation of education, but the integrity of the process demands the presence of technology being at the forefront, so as to ensure the security, scalability and delivery of content and data. The equity of education that technology offers, helps with addressing factors such as access to education, language, affordability, distance, and equality. Furthermore, it eliminates geographical boundaries by enabling the mass delivery of quality education with the right policies in place. [1] http://www.wsj.com/articles/SB10001424052702304756104579451241225610478 [2] http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2586847Nessus 6 XSLT Conversion for ASM Generic Scanner Import
It is important to understand while reading this, I am not an ASM SME... The goal was to create a simple conversion of the Nessus Vulnerability Scan reports to import into ASM. The first step was figuring out what the scan results needed to look like. So I exported the generic schema from ASM (13.0), which translates to: <?xml version="1.0" ?> <scanner_vulnerabilities> <vulnerability> <attack_type></attack_type> <name></name> <url></url> <parameter></parameter> <cookie></cookie> <threat></threat> <score></score> <severity></severity> <status></status> <opened></opened> </vulnerability> </scanner_vulnerabilities> That seems pretty simple, but thats a lot of attack types to map to some logic, so for now I will leave it generic. The next step is to get a vulnerability scan of a vulnerable web application. I wont go into how to use Nessus here, but one of the export options is a ".nessus" which is just an XML file. There is actually too much data in this file, but you can leave it as is. If you want to read it you can remove the <Policy> sections because all we want are the Reports. For this test, I ran a scan againstgoogle-gruyere.appspot.com, which is an unsecured app available to the internet. Dont do this from AWS or someone will come looking for you, ask me how I know... Example of results: <?xml version="1.0" ?> <NessusClientData_v2> <Report name="ASMv2" xmlns:cm="http://www.nessus.org/cm"> <ReportHost name="google-gruyere.appspot.com"> <HostProperties> <tag name="HOST_END">Tue Aug 29 14:08:04 2017</tag> <tag name="LastUnauthenticatedResults">1504015684</tag> <tag name="Credentialed_Scan">false</tag> <tag name="policy-used">Advanced Scan</tag> <tag name="patch-summary-total-cves">16</tag> <tag name="cpe">cpe:/o:linux:linux_kernel</tag> <tag name="os">linux</tag> <tag name="cpe-2">cpe:/o:linux:linux_kernel:2.6</tag> <tag name="cpe-1">cpe:/o:linux:linux_kernel:2.4</tag> <tag name="cpe-0">cpe:/o:linux:linux_kernel:2.2</tag> <tag name="system-type">general-purpose</tag> <tag name="operating-system">Linux Kernel 2.2 Linux Kernel 2.4 Linux Kernel 2.6</tag> <tag name="traceroute-hop-0">?</tag> <tag name="host-ip">172.217.3.212</tag> <tag name="host-fqdn">google-gruyere.appspot.com</tag> <tag name="HOST_START">Tue Aug 29 13:21:20 2017</tag> </HostProperties> <ReportItem pluginFamily="CGI abuses" pluginID="39470" pluginName="CGI Generic Tests Timeout" port="443" protocol="tcp" severity="0" svc_name="www"> <description>Some generic CGI tests ran out of time during the scan. The results may be incomplete.</description> <fname>torture_cgi_timeout.nasl</fname> <plugin_modification_date>2016/09/21</plugin_modification_date> <plugin_name>CGI Generic Tests Timeout</plugin_name> <plugin_publication_date>2009/06/19</plugin_publication_date> <plugin_type>summary</plugin_type> <risk_factor>None</risk_factor> <script_version>$Revision: 1.13 $</script_version> <solution>Consider increasing the 'maximum run time (minutes)' preference for the 'Web Applications Settings' in order to prevent the CGI scanning from timing out. Less ambitious options could also be used, such as : - Test more that one parameter at a time per form : 'Test all combinations of parameters' is much slower than 'Test random pairs of parameters' or 'Test all pairs of parameters (slow)'. - 'Stop after one flaw is found per web server (fastest)' under 'Do not stop after the first flaw is found per web page' is quicker than 'Look for all flaws (slowest)'. - In the Settings/Advanced menu, try reducing the value for 'Max number of concurrent TCP sessions per host' or 'Max simultaneous checks per host'.</solution> <synopsis>Some generic CGI attacks ran out of time.</synopsis> <plugin_output>The following tests timed out without finding any flaw : - XSS (on HTTP headers) - blind SQL injection - local file inclusion - blind SQL injection (time based) - unseen parameters - directory traversal (extended test) - directory traversal - arbitrary command execution - SQL injection (on HTTP headers) - SQL injection </plugin_output> </ReportItem> So far so good... Some of this will be a mess, but we can take a stab at it... So lets mash it all together! So how to tie it all together? There are some tools to help, online, I used to have on for Windows but I dont remember what its called, but I have a MAC now, so... xsltproc ASM_Nessus.xsl Nessus_Scan.xml > ASM_Import.xml FYI, for Windows users... https://www.microsoft.com/en-us/download/details.aspx?id=21714 Which gives me a pretty file to import to ASM. Its too big to post as text, but it looks like this: Alright, so the final test, lets import to ASM... Nice! It could use some work around Attack Type mapping and Parameter mapping, but it looks like it works. Well, thats as far as I got, I hope it helps someone. Now take it and run! XSLT can be found on github: https://github.com/Mikej81/NessusGenericASMSchemaDevCentral Top 5: Sep 8, 2014
But soft! What light through yonder window breaks? It is the east, and this week's edition of the DevCentral Top 5 is the sun. Yep, you guessed it. The top 5 is back...but unlike Shakespeare's Romeo and Juliet, this is no tragedy. Rather, it's a celebration of the most awesome articles you'll read anywhere on the Internet. Our DevCentral authors have been writing with freakish speed and determination, and they have turned out quality articles that are simply second to none. Choosing only five articles was a tough task given all the great content out there, but here's my take on the top articles since our last posting. F5 SOC Malware Summary Report: Neverquest I literally could have chosen five Lori MacVittie articles for this "top 5" but I resisted the urge and only chose one. In this article, Lori explains the details of a Trojan known as "Neverquest" that has been active since July 2013. Most of us get that warm, fuzzy, secure feeling when using 2-factor authentication because, you know, it's got 2 factors! Maybe automated malware has a shot at cracking one factor, but two? No way. Well, apparently Neverquest has found a way to automate the demise of our beloved 2FA. Lori does a magnificent job of explaining how Neverquest works, and then she discusses the amazing work that was completed by our F5 Security Operations Center in their analysis of this malware (in case you didn't know, F5 has a Security Operations Center that analyzes malware like this and provides amazing reports that are free for anyone to read). Lori provides links to the downloads of the executive summary as well as the full technical analysis of Neverquest. This one is not optional...if you care about anything at all, you gotta read this one. Leveraging BIG-IP APM for seamless client NTLM Authentication Michael Koyfman reminds us why we love the BIG-IP APM...transparent seamless authentication for users. In this article, Michael specifically discusses how to configure the APM to perform client NTLM authentication and use it in the context of sending a SAML assertion to the Office 365 service. This is a step-by-step masterpiece that shows you exactly what to do at every turn. In the end, you point your browser to the FQDN of the APM virtual server and you will be silently authenticated (let's be honest...silent authentication is a bucket-list item for each and every one of us). Michael also reminds us of the SSO options at the end of his article. Webshells Nir Zigler introduces us to Webshells (web scripts that act as a control panel for the server running them), and talks about some of the common uses for these scripts. But you know the story...scripts that were created for good can also be used for evil. After Nir explains all the valid uses for legitimate webshells, he takes us to a place where mere mortals dare not tread...through a webshell attack. He gives us an overview of how a webshell attack works, and then he explains some of the specific tools that are used for these nefarious actions. After walking through the power and functionality of an open source webshell called b374k, Nir shows how this tool can be used to attack an unsuspecting user. But have no fear! Nir finishes up the article by discussing the power of the BIG-IP ASM and how it will detect and prevent webshell attacks. Continuing the DDoS Arms Race How long have DDoS attacks been around, and why are they still news today? Because they are consistently one of the top attack vectors that companies face today. Shauntine'z discusses the DDoS arms race and provides some poignant statistics that remind us of the very real and credible DDoS threat. But the article doesn't stop there...it goes on to provide some excellent tips on what to do to strengthen your DDoS defense posture (it even has a well-placed picture of Professor John Frink...you gotta check this one out). Last, Shauntine'z reveals new features that are loaded in the latest release of the BIG-IP...version 11.6. The AFM and ASM have some new and exciting capabilities that are "must haves" for any company that is serious about securing their applications and critical business functions. (Editors note: the LineRate product has been discontinued for several years. 09/2023) Why ECC and PFS Matter: SSL offloading with LineRate We all know that sensitive data traverses our networks every day. We also know it's critically important to secure this information. We also know that SSL/TLS is the primary method used to secure said information. Andrew Ragone discusses SSL offloading and tells us why Elliptical Curve Cryptography (ECC) and Perfect Forward Secrecy (PFS) are great candidates for securing your information. He highlights the advantages of the software based LineRate solution, and gives great examples of why LineRate is the clear-cut winner over any existing software-based or hardware-based SSL/TLS offload solutions. Andrew also published another series of articles related to this very topic, and in these articles he walks you through the exact steps needed to configure SSL certificates and offload SSL on LineRate. On that subject...if you haven't had a chance to check out LineRate and learn all about the awesomeness that it is, do yourself a favor and visitHow I did it - "Visualizing Data with F5 TS and Splunk"
The new Splunk Add-on for F5 BIG-IPincludes several objects, (modular inputs, CIM-knowledge, etc.) that work to “normalize” incoming BIG-IP data for use with other Splunk apps, such as Splunk Enterprise Security and the Splunk App for PCI Compliance. The add-on includes a mechanism for pulling network traffic data, system logs, system settings, performance metrics, and traffic statistics from the F5 BIG-IP platform using F5’s iControl API, (see below). But what I'm really excited about is that the add-on now integrates with F5 Telemetry Streaming, (TS).With TS I am easily able to declaratively aggregate, normalize, and push BIG-IP statistics and events, (JSON-formatted) to a variety of third-party analytics vendors. For the remainder of this article, we’ll take a look at how I integrate F5 TS with Splunk Enterprise.I’ll be working with an existing BIG-IP deployment as well as a newly deployed Splunk Enterprise instance.As an added bonus, (and since it’s part of the article’s title) I’ll import a couple custom dashboards, (see below) to visualize our newly ingested telemetry data. Oh! As an "Extra" added bonus, here is a link to a video walk through of this solution. Installing the Splunk Add-on for F5 BIG-IP and Splunk CIM Installing the Splunk F5 add-on is very simple.Additionally, to make use of the add-on I’ll need to install Splunk’s Common Information Model, (CIM). 1.From the top Splunk the search page, I select ‘Apps’ → ‘Find More Apps’. 2.I browse for “CIM” and select the Splunk Common Information Model add-on. 3.I accept the license agreement, provide my Splunk account login credentials and select ‘Login and Install’. 4.I’ll repeat steps 2-3 to install the Splunk Add-on for F5 BIG-IP. Setup Splunk HTTP Event Collector To receive incoming telemetry data into my Splunk Enterprise environment over HTTP/HTTPs I will need to create an HTTP Event Collector. 1.From the UI I select ‘Settings’ → ‘Data Inputs’.I select ‘HTTP Event Collector’ from the input list. 2.Prior to creating a new event collector token, I must first enable token access for my Splunk environment. On the ‘HTTP Event Collector’ page, I select ‘Global Settings’.I set‘All Tokens’ to enabled, default index, incoming port and ensure SSL is enabled.I click ‘Save’ to exit. 3.I select ‘New Token’ and provide a name for the new collector and select ‘Next’. 4.On the ‘Input Settings’ tab I’ll select my allowed index(es) and select ‘Review’ then ‘Submit’. 5.Once the token is created, I will need to copy the token for use with my F5 TS configuration. Configure Telemetry Streaming With my Splunk environment ready to receive telemetry data, I now turn my attention to configuring the BIG-IP for telemetry streaming.Fortunately, F5’s Automation Toolchain configuring the BIG-IP is quite simple. 1.I’ll use Postman to POST an AS3 declaration to configure telemetry resources, (telemetry listener, log publisher, logging profiles, etc.). The above AS3 declaration, (available here) deploys the required BIG-IP objects for pushing event data to a third-party vendor. Notably, it creates four (4) logging profiles I’ll attach to my application’s virtual server. 2.Still using Postman, I POST my TS declaration, (sample).I will need to provide my Splunk HTTP Collector endpoint address/port as well as the token generated previously. Associate Logging Profiles to Virtual Server The final step to configuring the BIG-IP for telemetry streaming is associating the logging profiles I just created with my existing virtual server. In addition to system telemetry, these logging profiles, when assigned to a virtual,will send LTM, AVR, and ASM telemetry. 1.From the BIG-IP management UI, I select ‘Local Traffic’ → ‘Virtual Servers’ → <virtual>. 2.Under ‘Configuration’ I select ‘Advanced’, scroll down and select the HTTP, TCP, and request logging profiles previously created.I select ‘Update’ at the bottom of the page to save 3.From the top of the virtual server page, I select ‘Security’ → ‘Policies’.From the policy settings page, I can see that there is an existing WAF policy associated with my application.To enable ASM logging, I select the previously created ASM logging profile from the available logging profiles and select ‘Update’ to save my changes. With the configuration process complete, I should now start seeing event data in my Splunk Environment. Import Dashboards “Ok, so I have event data streaming into my Splunk environment; now what?” Since I have installed the Splunk F5 add-on, I can integrate my “normalized” data with other data sources to populate various Splunk applications like Splunk Enterprise Security and Splunk App for PCI Compliance.Likewise, I can use dashboards to visualizemy telemetry data as well as monitor BIG-IP resources/processes.To finish up, I’ll use the following steps to create custom dashboards visualizing BIG-IP metrics and Advanced WAF, (formerly ASM) attack information. 1.From the Splunk Search page, I navigate to the Dashboards page by selecting ‘Dashboards’. 2.Select ‘Create New Dashboard’ from the Dashboards page. 3.Provide a name for the new dashboard and select ‘Create Dashboard’.The dashboard name, (ID will remain unchanged) will be updated in the next step where I replace the newly created dashboard’s XML source with one of the community-supported dashboard XML files here. 4.On the ‘Edit Dashboard' screen I select ‘Source’ to edit the dashboard XML.I replace the existing XML data with the contents of the ‘advWafInsights.xml’ file.Select ‘Save’ to install the new dashboard. 5.I’ll repeat steps 1-4 using ‘bigipSystemMetrics.xml’ to install the BIG-IP metrics dashboard, Additional Links ·F5 Telemetry Streaming ·Splunk Add-on for F5 BIG-IP ·Splunk Common Information Model ·F5 Automation ToolchainConfiguring Unified Bot Defense with BIG-IQ Centralized Management
While estimates vary, it is believed that more than half of the Internet traffic is being generated by bots, out of which unwanted or malicious ones (like spam or malware bots) account for more than half of the traffic, the remaining traffic being generated by “good” bots (like crawlers or feed fetcher bots). It is therefore important to differentiate between different classes of bots and treat them according to site-specific security policies. The Unified Bot Defense profiles, first released in TMOS version 14.1, package bot protection features like Bot Signatures and Proactive Bot Defense previously found in L7 DoS profiles and Web Scraping protection found in ASM policies. Configuring Unified Bot Defense profiles through BIG-IQ ensures configuration consistency over the centralized managed BIG-IP estate and enhanced reporting capabilities. This article will guide you through the configuration of Unified Bot Defense profiles using BIG-IQ CM User Interface. It is assumed that the BIG-IP device where the Bot Defense profile will be deployed is currently managed by the BIG-IQ cluster, at least one BIG-IQ Logging Node / Data Collection Device is available and the Virtual Server to be protected is already configured (in the example below, VS_12BOX) - the configuration of these elements will not be part of this article. This article covers: configuring the Shared Security / Application Security Event Logging Profile configuring the Bot Defense profile monitoring the Bot Defense profiles Configuration of the Security Log Profile 1. Go to Configuration->LOCAL TRAFFIC->Pools, click Create and fill in the settings: -Name: Pool_DCD -Device: select the BIG-IP device -Health monitors: gateway_icmp -New member: - Select "New Node" - Address: Type the Log Node / DCD IP address - Port: 8514 (this is the port that Web Application Security Service is listening on the Logging Node / DCD) Note: Ensure that the Logging Node / Data Collection Device has the Web Application Security Service activated and the managed BIG-IP has LTM, SSM and ASM services Discovered/Imported. 2. Go to Configuration->LOCAL TRAFFIC->Logs ->Log Destination, click Create and fill in the settings: - Name: Log_dst_HSL_DCD - Type: Remote High-Speed - Device: select the BIG-IP device - Pool: select /Common/Pool_DCD 3. Go to Configuration->LOCAL TRAFFIC->Logs ->Log Destination, click Create and fill in the settings: - Name: Log_dst_Splunk_DCD - Type: SPLUNK - Forward to: select /Common/Log_dst_HSL_DCD 4. Go to Configuration->LOCAL TRAFFIC->Logs ->Log Publishers, click Create and fill in the settings: - Name: Log_pub_DCD - Log destinations: select /Common/Log_dst_Splunk_DCD 5. Go to Configuration->LOCAL TRAFFIC->Pinning Policies and select the BIG-IP device - Filter the available Local Traffic Manager (LTM) objects by selecting Log Publishers from the dropdown menu - Check Log_pub_DCD and click Add Selected button 6. Go to Configuration->SECURITY->Shared Security ->Logging Profiles, click Create and fill in the settings: -Name: Log_bot_protect_demo -Bot Defense: -Status: Enabled -Local Publisher: Enabled -Remote Publisher: /Common/Log_pub_DCD Attach the Log_bot_protect_demo log profile to the protected Virtual Server (in this example, VS_12BOX VS) 1. Go to Configuration->SECURITY->Shared Security->Virtual Servers and click on VS_12BOX VS 2. Select the Log_bot_protect_demo log profile for Logging profiles Deploy the configuration to the BIG-IP 1.Go to Deployment->EVALUATE & DEPLOY-> Local Traffic & Network, create a new Deployment. Once the evaluation has finished, click on Deploy. 2. Go to Deployment->EVALUATE & DEPLOY-> Shared Security, create a new Deployment. Once the evaluation has finished, click on Deploy. Configuration of the Bot Defense Profile Go to Configuration->SECURITY->Shared Security ->Bot Defense-> Bot Profiles, click Create and fill in the settings: -Name: bot_defense_demo -Enforcement Mode: Blocking -Profile Template: Strict -Browser Verification: -Browser Access: Allowed -Browser Verification: Verify After Access (Blocking) Note: As per K42323285: Overview of the unified Bot Defense profile the available options for the configuration elements used in this examples are: Enforcement Mode: Select one of the following modes, depending on the readiness of your application environment and requirements: Transparent—The system logs traffic mitigation and verification actions, according to your logging profile settings, but does not provide the following: JavaScript-based verification. Device ID collection. CAPTCHA challenge. Blocking—The system performs traffic mitigation and verification, and logsthem according to yourlogging profile settings. Profile Template: The template you select determines the default values for mitigation and verification settings. However, you can customize these settings to meet your application security requirements. After the system saves the profile, you can't change this setting. The following list contains descriptions of the available templates: Relaxed—Performs basic verification of browsers and blocks malicious bots based on bot signatures. Balanced—This is the default selection. Performs advanced verification of browsers,including: CAPTCHA challenges for suspicious browsers. Anomaly detection algorithms and bot signatures todetectand blockmalicious bots. Limitingthe total request rate for unknown bots. Strict—This is the strictest policy; it has settings that: Only allowbrowsers access if they pass proactive verification. Blockall bots except trusted ones. Browser Verification: Specifies what and when the system sends challenges. None—The system does not perform JavaScript and header-based verification. However, some anomaly detection (such as Session Opening) still occurs. Challenge-Free Verification—The default value when Profile Template is set to Relaxed. The system performs header-based verification but does not perform JavaScript verification. Verify Before Access—The default value when Profile Template is set to Strict. The system sends a white page with JavaScript to challenge the client. If the client fails the challenge, the system performs the configured mitigation action and reports the anomaly. If the client passes the challenge, the system forwards the request to the server. Verify After Access (Blocking)—The default value when Profile Template is set to Balanced. The system injects a JavaScript challenge in the server response prior to sending the response to the client. If the client fails the challenge, the systemperforms the configured mitigation action and reports the anomaly. If the client passes the challenge, the system forwards the request to the server. Verify After Access (Detection Only)—The system injects JavaScript challenge in the server response prior to sending the response to the client. If the client fails the challenge, the system only reports the anomaly but does not perform any mitigation action. If the client passes the challenge, the system forwards the request to the server. Device ID Mode: A unique identifier that BIG-IP ASM creates by sending JavaScript to get information about the client device. The default value for this setting is determined by your selection inProfile Template (under General Settings). F5 recommends you use the default values set by the Profile Template you selected unless you have specific application requirements. None—The default value when Profile Template is set to Relaxed. The system does not send JavaScript to collect the device ID. Generate After Access—The default value when Profile Template is set to Balanced. The system injects the JavaScript in the server response before forwarding to the client. This is less intrusive and has less of a latency impact. Generate Before Access—The default value when Profile Template is set to Strict. The system sends the JavaScript challenge to the client before forwarding the client request to the server. This guarantees that every request that reaches the server has a device ID. This has more of a latency impact compared to the previous option. The system blocks bots that attempt to present themselves asbrowsers but are unable to execute the JavaScript challenge. Attach the bot_defense_demo bot protect profile to the VS_12BOX VS 1. Go to Configuration->SECURITY->Shared Security->Virtual Servers and click on VS_12BOX VS 2. Select the bot_defense_demo profile for Bot Defense profile Deploy the Bot Defense profile to the BIG-IP Go to Deployment->EVALUATE & DEPLOY-> Shared Security, create a new Deployment. Once the evaluation has finished, click on Deploy Monitoring Bot Defense Profiles To monitor Bot Protection operation, check the Monitoring->DASHBOARDS->Bot Traffic Dashboard and Monitoring->EVENTS->Bot->Bot requests logsOWASP Mitigation Strategies Part 1: Injection Attacks
The OWASP organization lists “injection” attacks as the number one security flaw on the Internet today. In fact, injection attacks have made the OWASP top ten list for the past 13years and have been listed as the number one attack for the past 9years. Needless to say, these attacks are serious business and should be guarded against very carefully. Every web application environment allows the execution of external commands such as system calls, shell commands, and SQL requests. What’s more, almost all external calls can be attacked if the web application is not properly coded. In the general sense of the word, an injection attack is one whereby malicious code is “injected” into one of these external calls. These attacks are typically used against SQL, LDAP, Xpath, or NoSQL queries; OS commands; XML parsers, SMTP Headers, program arguments, etc. When a web application passes information from an HTTP request to an external system, it must be carefully scrubbed. Otherwise, the attacker can inject malicious commands into the request and the web application will blindly pass these on to the external system for execution. One of the most common types of injection attacks in a SQL injection. Many web applications utilize a backend database to store critical information like usernames, passwords, credit card numbers, personal information (address/phone number), etc. When you login to a website, you present your authentication credentials (i.e. username/password), and something on that backend has to validate that you are the correct user. There’s a database back there somewhere holding all that information. In effect, what has happened is that the website you are logging into sent an external command to a backend database and used the credentials you provided in the HTTP request to validate you against the stored database information. Along the way somewhere, some malevolent peeps had some crazy thoughts that went something like this: “hey, if my HTTP request is going to ultimately populate a SQL database command, maybe I could supply some crazy info in that HTTP request so that the SQL command ends up doing some really wild things to that backend database!!” That’s the idea behind the SQL injection attack. SQL commands are used to manipulate database information. You can add, delete, edit, display, etc any information in the database. Like any other system commands, these can be very helpful or very dangerous…it just depends on how they are used. Of course, these commands need to be locked down so that only trusted administrators can use them when running queries on the database. But imagine a web application that wasn’t built correctly and it allows SQL commands to be passed from the HTTP request to the backend database with no error checking at all. That would be a ripe web application for an injection attack! SQL Injection Attack The screenshot below shows a web application for an online auction site called “Hack-it-yourself auction”. This is a fictitious site that was purposely built with bad coding techniques and tons of vulnerabilities. It provides a great testing venue for things like injection attacks. It also showcases the power of a Web Application Firewall (WAF) and the protection it can provide even when your web application is totally vulnerable like this one. Notice in the screenshot that a user can login to the auction site via the “username” and “password” fields on the right side of the page. When a user submits login information on the website, the HTTP request takes the values from the “username” and “password” fields and uses them to formulate a SQL command to validate the user against the backend database that stores all the user information (name, address, credit card number, etc). The SQL command is designed to look something like this: SELECT id FROM users WHERE username = “username”and password = “password” This SQL command grabs the values for “username” and “password” that the user typed into each of those fields. If the username and password are correct, then the user is validated and the web application will allow the user’s information to be displayed. A SQL injection takes advantage of the fact that the HTTP request values are used to build the SQL command. An attacker could replace the “username” info with an interesting string of characters that don’t look very normal to the casual observer but are still understood by the backend database. Check out the screenshot below to see the unique SQL injection username. In this example, the username is ‘ or 1=1# and that will result in the SQL statement looking something like this: SELECT id FROM users WHERE username = ‘’ or 1=1#and password = '".md5($MD5_PREFIX.$password)."' Essentially, this statement is asking the database to select all records in the users table. It takes advantage of the fact that the web application is not validating user inputs and it uses the "or" expression on the username and then states a fact (in this case, 1=1) so that the database will accept it as a true statement. It also uses some creative MD5 hash techniques to access the password. Notice the username in the screenshot below: Apparently this SQL injection worked because now I am logged in as user ‘ or 1=1#. Not your typical, everyday username! So, the authentication was allowed to happen and the database is now ready to serve up any and all records that are located in the users table. I wonder what kind of goodness we can find there? Notice the “Your control panel” link on the page…that link takes you to the information stored for each user in the database. The database is ready to serve up ALL the records in the users table, so let’s click on that “Control Panel” link and see what kind of goodies we can find. Check out the screenshot below and notice in the address bar a familiar-looking username at the end of the URL: Clearly we have some serious issues here. At this point, the attacker has all the names, credit card numbers, emails, phone numbers, and addresses of every single user of this auction site. The attacker could simply copy all this information and use it later or he could manipulate it using other SQL commands or he could delete it all…the possibilities are seemingly endless. Think about some of the recent data breaches you’ve read about (or experienced) that have disclosed sensitive information…some of them happened because of a situation very similar to what I’ve just shown above. The correct answer for this problem is to fix the code that was used to develop this web application. Obviously, this specific code doesn’t check for valid usernames…it will pass along anything you put in that field. OWASP has a list of proactive controls that are great techniques to help prevent exploitation of one or more of the OWASP top ten vulnerabilities. For injection attacks specifically, code developers should do things like parameterize queries, encode data, and validate inputs. Ideally, code developers will write phenomenal code and you won’t ever have to worry about these vulnerabilities. And, in recent years, developers have done a great job of using secure coding practices to do just that. However, not all developers are perfect, and some of the code we use today is very old and hard to change…it’s also full of security vulnerabilities that allow for things like SQL injections. Web Application Firewall Mitigation The BIG-IP Application Security Manager (ASM) is a Web Application Firewall (WAF) that protects your web applications from attacks like the ones listed in the OWASP top ten. While it’s true that code should always be developed in a secure manner, those of us who live in the real world understand that we can’t rely on the hope of secure coding practices to protect our critical information all the time. That’s why you need a WAF. In the case of the very vulnerable “Hack-it-yourself auction” site, a WAF will protect the web application when it cannot protect itself. In the case of a SQL injection, a typical network firewall would have never blocked that attack because it was, in fact, a valid HTTP request on an open port (80, in this case). However, a WAF will inspect the HTTP request, notice that something isn’t quite right, and block it before it ever has a chance to do any damage to your backend database. I created a security policy using the BIG-IP ASM and then turned it on to protect the auction site. The ASM now sits between the user and the web application, so every HTTP request will now flow through the ASM before it heads over to the website. The ASM will inspect each request and either let it through or block it based on the configuration of the security policy. I won’t go into the details of how to build a policy in this article, but you can read more about how to build a policy here. After the security policy was created and activated, I tried the same SQL injection again. This time, instead of getting access to all those credit card numbers, I got a screen that looks like this (you can manipulate the screen to say whatever you want, by the way): Clearly the ASM is doing its job. Another cool thing about the ASM is that you can review the HTTP request logs to see exactly what an attacker is attempting to do when attacking your web application. The screenshot below shows some of the details of the SQL injection attempt. Notice the “Detected Keyword” portion of the screen…see anything familiar? Stay tuned for more exciting details on how the ASM can protect your web applications against other OWASP top ten (and more) vulnerabilities!
ImageTragick - ImageMagick Remote Code Execution Vulnerability
Abstract Recently, a number of vulnerabilities have been found in a very popular library, which is used to process image files. The vulnerabilities allow the attacker to execute code, move, read or delete remote files and issue outgoing requests from a web server. In certain scenarios, the vulnerability even can be exploited without authentication, making this a very powerful vulnerability and dangerous to unpatched web servers. This article will explain how those vulnerabilities can be mitigated using F5s Big-IP with ASM provisioned. ImageMagic ImageMagick is very popular piece of software, many programming languages have interface for ImageMagick allowing the programmatic access to image processing and editing. Itruns on Linux, Windows, Mac, iOS and many more: Quoting the ImageMagick.org The functionality of ImageMagick® is typically utilized from thecommand-lineor you can use the features from programs written in your favorite language. Choose from these interfaces:G2F(Ada),MagickCore(C),MagickWand(C),ChMagick(Ch),ImageMagickObject(COM+),Magick++(C++),JMagick(Java),L-Magick(Lisp),Lua(LuaJIT),NMagick(Neko/haXe),Magick.NET(.NET),PascalMagick(Pascal),PerlMagick(Perl),MagickWand for PHP(PHP),IMagick(PHP),PythonMagick(Python),RMagick(Ruby), orTclMagick(Tcl/TK). With a language interface, use ImageMagick to modify or create images dynamically and automagically . MVG format MVG stands forMagickVectorGraphics, and it is a modularized language for describing two-dimensional vector graphics using the ImageMagick engine. It can look like this: push graphic-context viewbox 0 0 400 400 image over 0,0 0,0 'label:@/home/files/nice_text' pop graphic-context In this example, the ImageMagick engine reads the text from the 'nice_text' file and draws a label with its contents over the picture. Vulnerability - ImageTragick The group of vulnerabilities was named ImageTragick because they exploit the ImageMagick package. The package contains several “coders” supporting instructions and commands for image manipulation. For instance, using the “label” instruction one can add a free text from a file to an image. Several instructions were found to be vulnerable to different types of attack. More detailed information on each vulnerability can be found in the researchers’ website: https://imagetragick.com/ We could distinguish those findings as three types of vulnerabilities: Shell Command Execution Application Abuse of Functionality Server Side Request Forgery Shell Command Execution CVE-2016-3714 By abusing this type of vulnerability, the attacker can cause the ImageMagick library to pass commands to the operation system shell potentially ending up in complete system compromise. Considering ImageMagick’s availability for all the popular operating systems and programming languages makes this vulnerability a very dangerous one. Attacker can create a file in ImageMagick MVG and SVG format and upload it to a webserver, which will process this file. Successfully performing this operation will cause ImageMagick to execute the command from the crafted file. Example: fill 'url(https://example.com/image.jpg";|ls -la")' The vulnerable code takes the URL and without proper validation concatenates it to the “wget” system command to fetch the image. Attacker is able to provide a URL ending with a double quote character and concatenate additional arbitrary system commands, such as a “ping” command (“ping www.f5.com”) in the above example. If the server is vulnerable, it will contact www.f5.com. Attackers can replace the “ping” command by something less naive, such as, changing the password of the 'root' account or installing a malware or a backdoor to gain control over the server. ImageTragick Application Abuse It was found that some coders include functionality that could be misused in order to perform the following operations on the server: CVE-2016-3715 – Delete file CVE-2016-3716 – File moving CVE-2016-3717 – Local File Read The “EPHEMERAL” coder could be abused to delete files from the server just by providing a file name, as it deletes the file after reading it. Example : image over 0,0 0,0 'ephemeral://critical.file' The “label” instruction of the MVG coder could be abused to read the content of an arbitrary file on the server. Example : image over 0,0 0,0 'label:@/etc/passwd' However, CVE-2016-3716 is even more interesting application abuse use case as, just by using the “read” and “write” directives of the MSL coder an attacker could potentially compromise a remote server. In the full attack scenario, attacker will upload an “image” file containing PHP code with a valid image extension, say “image.gif”. <?php phpinfo(); //Show all information, defaults to INFO_ALL ?> <!-- image.gif that will be renamed to backdoor.php --> The next step will be uploading an MSL file with the “read” directive which will read the content of the previously uploaded “image” file and also with the “write” directive which will write this content to a web accessible directive while renaming the file to have a PHP extension to make it executable. Example: <?xml version="1.0" encoding="UTF-8"?> <image> <read filename="/tmp/image.gif" /> <write filename="/var/www/shell.php" /> </image> <!-- script.msl --!> Finally, the attacker will upload another MVG file that will execute previously uploaded MSL file using the “msl” pseudo protocol. Example: image over 0,0 0,0 'msl:/files/script.msl' This example uses the 'phpinfo()' function in the PHP backdoor file, as a simple indication that the exploit worked, however attacker can replace this command with a webshell such as described in our previous posts. Server Side Request Forgery Same “url” directive used in the MVG/SVG format could be also abused to cause the server send HTTP or FTP requests for example to another server inside the server’s internal network that could not be accessible from the outside world, by providing a URL with an IP address of the internal network. This vulnerability is tracked as CVE-2016-3718. Mitigation It is possible to protect against the ImageTragick vulnerabilities using Big-IP ASM. Using WAF for “virtually patching” the application could be even critical as a protection tool especially before the patch for the backend code is available or has been deployed. Although the “Shell Command Execution” vulnerabilities are 0-day vulnerabilities, meaning previously unknown attack vector, the post-exploitation has the same pattern, attempting to smuggle and execute operation system commands on the attacked server and will be mitigated by the existing “Command Execution” attack signatures. For instance, signature ID 200003041 (“ls” execution attempt) will spot an attacker who is trying to smuggle the “ls –la” system command while exploiting CVE-2016-3714. We have also released Attack Signature Update to detect and mitigate the specific ImageMagick application abuse and server side request forgery vulnerabilities. It is recommended to patch the vulnerabilities in the code and follow some basic remediation steps: Consider disabling the unsafe coders Consult the ImageMagick recommendation: http://www.imagemagick.org/discourse-server/viewtopic.php?t=29588 Webserver should run with the lowest permissions Always patch the systemsImageTragick - The Tragick continues
Abstract We keep talking about the fact that ASM is an effective tool for protecting the 0-day attacks. Two years ago we were able to detect the shellshock exploitations attempts by detecting the carried commands that the shellshock was used to execute. More details are here: Mitigating the Unknown. ImageMagick ImageMagick, a very popular image editing library, has a new vulnerability which allows a code execution abusing an incorrectly parsed file format. We already wrote about ImageMagick in June because there were multiple CVEs about mishandling the MVG and SVG file formats, allowing abuse of ImageMagick based software to create and delete files, move them, and run commands, only by abusing incomplete input validation in ImageMagick’s file format. Sometime after the original CVEs another CVE showed up, allowing another attacker to execute an arbitrary shell command using the pipe character (“|”) at the start of the processed filename. CVE-2016-5118 CVE ID for the new vulnerability is CVE-2016-5118andjust like the earlier CVE-2016-3714, it may allow the attacker to remotely smuggle and run shell commands on the server. Luckily for us, ASM is great with mitigating the 0-day shell and code execution vectors and, therefore, nothing had to be reconfigured to keep our server safe from this attack. In this particular example, ASM has detected using 3 ASM signatures: echo sig_id 200003045, cat sig_id 200003065 /usr sig_id 200003060 Conclusion I believe that we have not seen the last CVE for the ImageMagick based software but ASM should be able to detect and block the exploitation attempts for the vulnerabilities that were already discovered and those which are yet to come. We will continue to follow the issue and update the signatures when needed.
