From ASM to Advanced WAF: Advancing your Application Security
TL;DR: As of April 01, 2021, F5 has officially placed Application Security Manager (ASM) into End of Sale (EoS) status, signifying the eventual retirement of the product. (F5 Support Announcement - K72212499 ) Existing ASM,or BEST bundle customers, under a valid support contract running BIG-IP version 14.1 or greater can simply reactivate their licenses to instantly upgrade to Advanced WAF (AdvWAF) completely free of charge. Introduction Protecting your applications is becoming more challenging every day; applications are getting more complex, and attackers are getting more advanced. Over the years we have heard your feedback that managing a Web Application Firewall (WAF) can be cumbersome and you needed new solutions to protect against the latest generation of attacks. Advanced Web Application Firewall, or AdvWAF, is an enhanced version of the Application Security Manager (ASM) product that introduces new attack mitigation techniques and many quality-of-life features designed to reduce operational overhead. On April 01, 2021 – F5 started providing free upgrades for existing Application Security Manager customers to the Advanced WAF license. Keep on reading for: A brief history of ASM and AdvWAF How the AdvWAF license differs from ASM (ASM vs AdvWAF How to determine if your BIG-IPs are eligible for this free upgrade Performing the license upgrade How did we get here? For many years, ASM has been the gold standard Web Application Firewall (WAF) used by thousands of organizations to help secure their most mission-critical web applications from would-be attackers. F5 acquired the technology behind ASM in 2004 and subsequently ‘baked’ it into the BIG-IP product, immediately becoming the leading WAF product on the market. In 2018, after nearly 14 years of ASM development, F5 released the new, Advanced WAF license to address the latest threats. Since that release, both ASM and AdvWAF have coexisted, granting customers the flexibility to choose between the traditional or enhanced versions of the BIG-IP WAF product.As new features were released, they were almost always unique to AdvWAF, creating further divergence as time went on, and often sparking a few common questions (all of which we will inevitably answer in this very article) such as: Is ASM going away? What is the difference between ASM and AdvWAF? Will feature X come to ASM too? I need it! How do I upgrade from ASM to AdvWAF? Is the BEST bundle no longer really the BEST? To simplify things for our customers (and us too!), we decided to announce ASM as End of Sale (EoS), starting on April 01, 2021. This milestone, for those unfamiliar, means that the ASM product can no longer be purchased after April 01 of this year – it is in the first of 4 stages of product retirement. An important note is that no new features will be added to ASM going forward. So, what’s the difference? A common question we get often is “How do I migrate my policy from ASM to AdvWAF?” The good news is that the policies are functionally identical, running on BIG-IP, with the same web interface, and have the same learning engine and underlying behavior. In fact, our base policies can be shared across ASM, AdvWAF, and NGINX App Protect (NAP). The AdvWAF license simply unlocks additional features beyond what ASM has, that is it – all the core behaviors of the two products are identical otherwise. So, if an engineer is certified in ASM and has managed ASM security policies previously, they will be delighted to find that nothing has changed except for the addition of new features. This article does not aim to provide an exhaustive list of every feature difference between ASM and AdvWAF. Instead, below is a list of the most popular features introduced in the AdvWAF license that we hope you can take advantage of. At the end of the article, we provide more details on some of these features: Secure Guided Configurations Unlimited L7 Behavioral DoS DataSafe (Client-side encryption) OWASP Compliance Dashboard Threat Campaigns (includes Bot Signature updates) Additional ADC Functionality Micro-services protection Declarative WAF Automation I’m interested, what’s the catch? There is none! F5 is a security company first and foremost, with a mission to provide the technology necessary to secure our digital world. By providing important useability enhancements like Secure Guided Config and OWASP Compliance Dashboard for free to existing ASM customers, we aim to reduce the operational overhead associated with managing a WAF and help make applications safer than they were yesterday - it’s a win-win. If you currently own a STANDALONE, ADD-ON or BEST Bundle ASM product running version 14.1 or later with an active support contract, you are eligible to take advantage of this free upgrade. This upgrade does not apply to customers running ELA licensing or standalone ASM subscription licenses at this time. If you are running a BIG-IP Virtual Edition you must be running at least a V13 license. To perform the upgrade, all you need to do is simply REACTIVATE your license, THAT IS IT! There is no time limit to perform the license reactivation and this free upgrade offer does not expire. *Please keep in mind that re-activating your license does trigger a configuration load event which will cause a brief interruption in traffic processing; thus, it is always recommended to perform this in a maintenance window. Step 1: Step 2: Choose “Automatic” if your BIG-IP can communicate outbound to the Internet and talk to the F5 Licensing Server. Choose Manual if your BIG-IP cannot reach the F5 Licensing Server directly through the Internet. Click Next and the system will re-activate your license. After you’ve completed the license reactivation, the quickest way to know if you now have AdvWAF is by looking under the Security menu. If you see "Guided Configuration”, the license upgrade was completed successfully. You can also login to the console and look for the following feature flags in the /config/bigip.license file to confirm it was completed successfully by running: grep -e waf_gc -e mod_waf -e mod_datasafe bigip.license You should see the following flags set to enabled: Waf_gc: enabled Mod_waf: enabled Mod_datasafe: enabled *Please note that the GUI will still reference ASM in certain locations such as on the resource provisioning page; this is not an indication of any failure to upgrade to the AdvWAF license. *Under Resource Provisioning you should now see that FPS is licensed. This will need to be provisioned if you plan on utilizing the new AdvWAF DataSafe feature explained in more detail in the Appendix below. For customers with a large install base, you can perform license reactivation through the CLI. Please refer to the following article for instructions: https://support.f5.com/csp/article/K2595 Conclusion F5 Advanced WAF is an enhanced WAF license now available for free to all existing ASM customers running BIG-IP version 14.1 or greater, only requiring a simple license reactivation. The AdvWAF license will provide immediate value to your organization by delivering visibility into the OWASP Top 10 compliance of your applications, configuration wizards designed to build robust security policies quickly, enhanced automation capabilities, and more. If you are running ASM with BIG-IP version 14.1 or greater, what are you waiting for? (Please DO wait for your change window though 😊) Acknowledgments Thanks to Brad Scherer , John Marecki , Michael Everett , and Peter Scheffler for contributing to this article! Appendix: More details on select AdvWAF features Guided Configurations One of the most common requests we hear is, “can you make WAF easier?” If there was such a thing as an easy button for WAF configurations, Guided Configs are that button. Guided Configurations easily take you through complex configurations for various use-cases such as Web Apps, OWASP top 10, API Protection, DoS, and Bot Protection. L7DoS – Behavioral DoS Unlimited Behavioral DoS - (BaDoS) provides automatic protection against DoS attacks by analyzing traffic behavior using machine learning and data analysis. With ASM you were limited to applying this type of DoS profile to a maximum of 2 Virtual Servers. The AdvWAF license completely unlocks this capability, removing the 2 virtual server limitation from ASM. Working together with other BIG-IP DoS protections, Behavioral DoS examines traffic flowing between clients and application servers in data centers, and automatically establishes the baseline traffic/flow profiles for Layer 7 (HTTP) and Layers 3 and 4. DataSafe *FPS must be provisioned DataSafe is best explained as real-time L7 Data Encryption. Designed to protect websites from Trojan attacks by encrypting data at the application layer on the client side. Encryption is performed on the client-side using a public key generated by the BIG-IP system and provided uniquely per session. When the encrypted information is received by the BIG-IP system, it is decrypted using a private key that is kept on the server-side. Intended to protect, passwords, pins, PII, and PHI so that if any information is compromised via MITB or MITM it is useless to the attacker. DataSafe is included with the AdvWAF license, but the Fraud Protection Service (FPS) must be provisioned by going to System > Resource Provisioning: OWASP Compliance Dashboard Think your policy is air-tight? The OWASP Compliance Dashboard details the coverage of each security policy for the top 10 most critical web application security risks as well as the changes needed to meet OWASP compliance. Using the dashboard, you can quickly improve security risk coverage and perform security policy configuration changes. Threat Campaigns (includes Bot Signature updates) Threat campaigns allow you to do more with fewer resources. This feature is unlocked with the AdvWAF license, it, however, does require an additional paid subscription above and beyond that. This paid subscription does NOT come with the free AdvWAF license upgrade. F5’s Security Research Team (SRT) discovers attacks with honeypots – performs analysis and creates attack signatures you can use with your security policies. These signatures come with an extremely low false-positive rate, as they are strictly based on REAL attacks observed in the wild. The Threat Campaign subscription also adds bot signature updates as part of the solution. Additional ADC Functionality The AdvWAF license comes with all of the Application Delivery Controller (ADC) functionality required to both deliver and protect a web application. An ASM standalone license came with only a very limited subset of ADC functionality – a limit to the number of pool members, zero persistence profiles, and very few load balancing methods, just to name a few. This meant that you almost certainly required a Local Traffic Manager (LTM) license in addition to ASM, to successfully deliver an application. The AdvWAF license removes many of those limitations; Unlimited pool members, all HTTP/web pertinent persistence profiles, and most load balancing methods, for example.Making WAF Simple: Introducing the OWASP Compliance Dashboard
Whether you are a beginner or an expert, there is a truth that I want to let you in on; building and maintaining Web Application Firewall (WAF) security policies can be challenging. How much security do you really need? Is your configuration too much or too little? Have you created an operational nightmare? Many well-intentioned administrators will initially enable every available feature, thinking that it is providing additional security to the application, when in truth, it is hindering it. How, you may ask? False positives and noise. The more noise and false positives, the harder it becomes to find the real attacks and the increased likelihood that you begin disabling features that ARE providing essential security for your applications. So… less is better then? That isn't the answer either, what good are our security solutions if they aren't protecting against anything? The key to success and what we will look at further in this article, is implementing best practice controls that are both measurable and manageable for your organization. The OWASP Application Security Top 10 is a well-respected list of the ten most prevalent and dangerous application layer attacks that you almost certainly should protect your applications from. By first focusing your security controls on the items in the OWASP Top 10, you are improving the manageability of your security solution and getting the most "bang for your buck". Now, the challenge is, how do you take such a list and build real security protections for your applications? Introducing the OWASP Compliance Dashboard Protecting your applications against the OWASP Top 10 is not a new thing, in fact, many organizations have been taking this approach for quite some time. The challenge is that most implementations that claim to "protect" against the OWASP Top 10 rely solely on signature-based protections for only a small subset of the list and provide zero insight into your compliance status. The OWASP Compliance Dashboard introduced in version 15.0 on BIG-IP Advanced WAF reinvents this idea by providing a holistic and interactive dashboard that clearly measures your compliancy against the OWASP Application Security Top 10. The Top 10 is then broken down into specific security protections including both positive and negative security controls that can be enabled, disabled, or ignored directly on the dashboard. We realize that a WAF policy alone may not provide complete protection across the OWASP Top 10, this is why the dashboard also includes the ability to review and track the compliancy of best practices outside the scope of a WAF alone, such as whether the application is subject to routine patching or vulnerability scanning. To illustrate this, let’s assume I have created a brand new WAF policy using the Rapid Deployment policy template and accepted all default settings, what compliance score do you think this policy might have? Let's take a look. Interesting. The policy is 0/10 compliant and only A2 Broken Authentication and A3 Sensitive Data Exposure have partial compliance. Why is that? The Rapid Deployment template should include some protections by default, shouldn't it? Expanding A1 Injection, we see a list of protections required in order to be marked as compliant. Hovering over the list of attack signatures, we see that each category of signature is in 'Staging' mode, aha! Signatures in staging mode are not enforced and therefore cannot block traffic. Until the signature set is in enforced, we do not mark that protection as compliant. For those of you who have mistakenly left entities such as Signatures in staging for longer than desired, this is also a GREAT way to quickly find them. I also told you we could also interact with the dashboard to influence the compliancy score, lets demonstrate that. Each item can be enforced DIRECTLY on the dashboard by selecting the "Enforce" checkmark on the right. No need to go into multiple menus, you can enforce all these security settings on a single page and preview the compliance status immediately. If you are happy with your selection, click on "Review & Update" to perform a final review of what the dashboard will be configuring on your behalf before you can click on "Save & Apply Policy". Note: Enforcing signatures before a period of staging may not be a good idea depending on your environment. Staging provides a period to assess signature matches in order to eliminate false positives. Enforcing these signatures too quickly could result in the denying of legitimate traffic. Let's review the compliancy of our policy now with these changes applied. As you can see, A1 Injection is now 100% compliant and other categories have also had their score updated as a result of enforcing these signatures. The reason for this is because there is overlap in the security controls applied acrossthese other categories. Not all security controls can be fully implemented directly via the dashboard, and as mentioned previously, not all security controls are signature-based. A6 Cross-Site Scripting was recalculated as 50% complaint with the signatures we enforced previously so let's take a look at what else it required for full compliancy. The options available to us are to IGNORE the requirement, meaning we will be granted full compliancy for that item without implementing any protection, or we can manually configure the protection referenced. We may want to ignore a protection if it is not applicable to the application or if it is not in scope for your deployment. Be mindful that ignoring an item means you are potentially misrepresenting the score of your policy, be very certain that the protection you are ignoring is in fact not applicable before doing so. I've selected to ignore the requirement for "Disallowed Meta Characters in Parameters" and my policy is now 100% compliance for A7 Cross-Site Scripting (XSS). Lastly, we will look at items within the dashboard that fall outside the scope of WAF protections. Under A9 Using Components with Known Vulnerabilities, we are presented with a series of best practices such as “Application and system hardening”, “Application and system patching” and “Vulnerability scanner integration”. Using the dashboard, you can click on the checkmark to the right for "Requirement fulfilled" to indicate that your organization implements these best practices. By doing so, the OWASP Compliance score updates, providing you with real-time visibility into the compliancy for your application. Conclusion The OWASP Compliance Dashboard on BIG-IP Advanced WAF is a perfect fit for the security administrator looking to fine-tune and measure either existing or new WAF policies against the OWASP App Security Top 10. The OWASP Compliance Dashboard not only tracks WAF-specific security protections but also includes general best practices, allowing you to use the dashboard as your one-stop-shop to measure the compliancy for ALL your applications. For many applications, protection against the OWASP Top 10 may be enough, as it provides you with best practices to follow without having to worry about which features to implement and where. Note: Keep in mind that some applications may require additional controls beyond the protections included in the OWASP Top 10 list. For teams heavily embracing automation and CI/CD pipelines, logging into a GUI to perform changes likely does not sound appealing. In that case, I suggest reading more about our Declarative Advanced WAF policy framework which can be used to represent the WAF policies in any CI/CD pipeline. Combine this with the OWASP Compliance Dashboard for an at-a-glance assessment of your policy and you have the best of both worlds. If you're not already using the OWASP Compliance Dashboard, what are you waiting for? Look out for Bill Brazill, Victor Granic and myself (Kyle McKay) on June 10th at F5 Agility 2020 where we will be presenting and facilitating a class called "Protecting against the OWASP Top 10". In this class, we will be showcasing the OWASP Compliance Dashboard on BIG-IP Advanced WAF further and providing ample hands-on time fine-tuning and measuring WAF policies for OWASP Compliance. Hope to see you there! To learn more, visit the links below. Links OWASP Compliance Dashboard: https://support.f5.com/csp/article/K52596282 OWASP Application Security Top 10: https://owasp.org/www-project-top-ten/ Agility 2020: https://www.f5.com/agility/attend
Separating False Positives from Legitimate Violations
Imagine you make yourself a cup of tea and now want to extract the tannins and caffeine from the homogeneous mixture. How do you do it? Similarly, when building and protecting applications, you brew a blend of legitimate violations and false positives (distinct substances with similar properties) that match the rules you have defined in your web application firewall (WAF). How do you separate the false positives from legitimate violations? Before you decaffeinate your tea, let’s review some context around false positives and the challenges they pose in application security. False positives occur when legitimate requests are identified as attacks or violations. Due to the complex nature of web applications, false positives are a normal aspect of application security. It is preferable for a WAF to trigger false positives than to allow false negatives (attacks perceived as legitimate traffic). However, reducing the rate of false positives without compromising application security remains a significant challenge for security professionals. A high false positive rate has the following disadvantages: · Obstructs legitimate traffic · Increases maintenance · Burdens computer resources One way to extract the caffeine from your tea, chemically, is to introduce a solvent to neutralize the caffeine. Similarly, separating false positives from legitimate violations require the introduction of a solvent, of sorts, to neutralize the false positives among all the legitimate violations found in our WAF log. The challenge is designing the so-called solvent. Applications are unique and require a deep understanding of architecture, business flow and also your configured WAF rules in order to define the necessary properties for each so-called solvent. For example, using ‘square brackets in parameter name’ as a so-called solvent, is a common practice in the Drupal application development, yet many default WAF rules will flag it as a violation and trigger the false positive.[1]Christian Folini, https://www.christian-folini.ch You should also consider other volumetric and statistical factors such as violation type, volume, and density when you design your so-called solvent. Generally 3 components are taken in consideration : Traffic: What are the characteristics (origin, destination, frequency, payload ...) of the traffic (HTTP Request/Response) that triggered the violation ?. Violation: What are the characteristics (Type of attack, Rule triggered , ) of the violation triggered ?. Application: What are the characteristics of the application that is processing the request ?. Once you have subtracted false positives, you can correct your WAF configurations and monitor for new violations, then repeat the process. The workflow looks like the following: 1. Design a solvent. 2. Sample web traffic and log all violations. 3. Apply all violations to the solvent. 4. Use analytical extraction or manual extraction to subtract false positives. 5. Correct your WAF with Rules Exclusion for the identified false positives. 6. Monitor both web traffic and application changes. 7. Repeat steps 1 through 6. As solvent properties are specific to applications, in the next article, we'll provide additional example of solvent properties for the most popular applications.HTTP Brute Force Mitigation Playbook: Overview - Chapter 1
Overview When we talk about Brute Force attacks, we usually tend to think about a malicious actor using a script or botnet to inject credentials into a login form in order to try to brute force their way past an authentication mechanism, but that is far from the only kind of brute force attack we see in the wild today, with attacks against API endpoints becoming increasingly common as traditional web development gives way to an API-centric, cloud-driven microservices model alongside moves to federated authentication for services like Office 365. While many of these moves are great for scalability and accessibility, they also open up an increasingly large attack surface that malicious actors are beginning to take advantage of. In this document, we aim to show you some of the BIG-IP tools and techniques available to mitigate brute force attacks against your organisation, as well as sample configurations you can use as a basis for part of your security configuration. Introduction In this series of articles we will show you the BIG-IP tools and techniques you can leverage to understand, classify and mitigate brute force attacks using: BIG-IP AVR Analytics BIG-IP LTM, iRules and Local Traffic Policies BIG-IP ASM with ASM Brute Force protections Bot Defence Fingerprinting (TLS Fingerprinting & HTTP Fingerprinting) L7DoS protections We will coverthe following kinds of Brute Force attack: Attacks against traditional HTML form-based authentication pages "Low and slow" attacks against form-based authentication or other form-based submissions API attacks against authenticated and non-authenticated API endpoints Outlook Web Access/Outlook 365 authentication brute force attacks All configuration examples and suggested mitigation methods will be based on features available in BIG-IP 14.1 and later, and at the end of this document you will find an Appendix with example configurations summarised and presented for easy deployment. Chapters Bad Actor Behaviours and Gathering Statistics using BIG-IP LTM Policies, iRules and BIG-IP AVR | Chapter 2 BIG-IP LTM Mitigation Options for HTTP Brute Force Attacks | Chapter 3 Protecting HTML Form Based Authorization using ASM | Chapter 4 Using the Bot Profile for Brute Force Mitigation | Chapter 5 Slow Brute Force Protection Using Behavioural DOS | Chapter 6 AppendixAdvanced WAF v16.0 - Declarative API
Since v15.1 (in draft), F5® BIG-IP® Advanced WAF™ canimport Declarative WAF policy in JSON format. The F5® BIG-IP® Advanced Web Application Firewall (Advanced WAF) security policies can be deployed using the declarative JSON format, facilitating easy integration into a CI/CD pipeline. The declarative policies are extracted from a source control system, for example Git, and imported into the BIG-IP. Using the provided declarative policy templates, you can modify the necessary parameters, save the JSON file, and import the updated security policy into your BIG-IP devices. The declarative policy copies the content of the template and adds the adjustments and modifications on to it. The templates therefore allow you to concentrate only on the specific settings that need to be adapted for the specific application that the policy protects. ThisDeclarative WAF JSON policyis similar toNGINX App Protect policy. You can find more information on theDeclarative Policyhere : NAP :https://docs.nginx.com/nginx-app-protect/policy/ Adv. WAF :https://techdocs.f5.com/en-us/bigip-15-1-0/big-ip-declarative-security-policy.html Audience This guide is written for IT professionals who need to automate their WAF policy and are familiar with Advanced WAF configuration. These IT professionals can fill a variety of roles: SecOps deploying and maintaining WAF policy in Advanced WAF DevOps deploying applications in modern environment and willing to integrate Advanced WAF in their CI/CD pipeline F5 partners who sell technology or create implementation documentation This article covershow to PUSH/PULL a declarative WAF policy in Advanced WAF: With Postman With AS3 Table of contents Upload Policy in BIG-IP Check the import Apply the policy OpenAPI Spec File import AS3 declaration CI/CD integration Find the Policy-ID Update an existing policy Video demonstration First of all, you need aJSON WAF policy, as below : { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false } } } 1. Upload Policy in BIG-IP There are 2 options to upload a JSON file into the BIG-IP: 1.1 Either youPUSHthe file into the BIG-IP and you IMPORT IT OR 1.2 the BIG-IPPULLthe file froma repository (and the IMPORT is included)<- BEST option 1.1PUSH JSON file into the BIG-IP The call is below. As you can notice, it requires a 'Content-Range' header. And the value is 0-(filesize-1)/filesize. In the example below, the file size is 662 bytes. This is not easy to integrate in a CICD pipeline, so we created the PULL method instead of the PUSH (in v16.0) curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/file-transfer/uploads/policy-api.json' \ --header 'Content-Range: 0-661/662' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --header 'Content-Type: application/json' \ --data-binary '@/C:/Users/user/Desktop/policy-api.json' At this stage,the policy is still a filein the BIG-IP file system. We need toimportit into Adv. WAF. To do so, the next call is required. This call import the file "policy-api.json" uploaded previously. AnCREATEthe policy /Common/policy-api-arcadia curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/javascript' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "filename":"policy-api.json", "policy": { "fullPath":"/Common/policy-api-arcadia" } }' 1.2PULL JSON file from a repository Here, theJSON file is hosted somewhere(in Gitlab or Github ...). And theBIG-IP will pull it. The call is below. As you can notice, the call refers to the remote repo and the body is a JSON payload. Just change the link value with your JSON policy URL. With one call, the policy isPULLEDandIMPORTED. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" } }' Asecond versionof this call exists, and refer to the fullPath of the policy.This will allow you to update the policy, from a second version of the JSON file, easily.One call for the creation and the update. As you can notice below, we add the"policy":"fullPath" directive. The value of the "fullPath" is thepartitionand thename of the policyset in the JSON policy file. This method is VERY USEFUL for CI/CD integrations. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" }, "policy": { "fullPath":"/Common/policy-api-arcadia" } }' 2. Check the IMPORT Check if the IMPORT worked. To do so, run the next call. curl --location --request GET 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ You should see a 200 OK, with the content below (truncated in this example). Please notice the"status":"COMPLETED". { "kind": "tm:asm:tasks:import-policy:import-policy-taskcollectionstate", "selfLink": "https://localhost/mgmt/tm/asm/tasks/import-policy?ver=16.0.0", "totalItems": 11, "items": [ { "isBase64": false, "executionStartTime": "2020-07-21T15:50:22Z", "status": "COMPLETED", "lastUpdateMicros": 1.595346627e+15, "getPolicyAttributesOnly": false, ... From now, your policy is imported and created in the BIG-IP. You can assign it to a VS as usual (Imperative Call or AS3 Call).But in the next session, I will show you how to create a Service with AS3 including the WAF policy. 3. APPLY the policy As you may know, a WAF policy needs to be applied after each change. This is the call. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/apply-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{"policy":{"fullPath":"/Common/policy-api-arcadia"}}' 4. OpenAPI spec file IMPORT As you know,Adv. WAF supports OpenAPI spec (2.0 and 3.0). Now, with the declarative WAF, we can import the OAS file as well. The BEST solution, is toPULL the OAS filefrom a repo. And in most of the customer' projects, it will be the case. In the example below, the OAS file is hosted in SwaggerHub(Github for Swagger files). But the file could reside in a private Gitlab repo for instance. The URL of the projectis :https://app.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3 The URL of the OAS file is :https://api.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3 This swagger file (OpenAPI 3.0 Spec file) includes all the application URL and parameters. What's more, it includes the documentation (for NGINX APIm Dev Portal). Now, it ispretty easy to create a WAF JSON Policy with API Security template, referring to the OAS file. Below, you can notice thenew section "open-api-files"with the link reference to SwaggerHub. And thenew templatePOLICY_TEMPLATE_API_SECURITY. Now, when I upload / import and apply the policy, Adv. WAF will download the OAS file from SwaggerHub and create the policy based on API_Security template. { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "https://api.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3" } ] } } 5. AS3 declaration Now, it is time to learn how we cando all of these steps in one call with AS3(3.18 minimum). The documentation is here :https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/declarations/application-security.html?highlight=waf_policy#virtual-service-referencing-an-external-security-policy With thisAS3 declaration, we: Import the WAF policy from a external repo Import the Swagger file (if the WAF policy refers to an OAS file) from an external repo Create the service { "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.2.0", "id": "Prod_API_AS3", "API-Prod": { "class": "Tenant", "defaultRouteDomain": 0, "API": { "class": "Application", "template": "generic", "VS_API": { "class": "Service_HTTPS", "remark": "Accepts HTTPS/TLS connections on port 443", "virtualAddresses": ["10.1.10.27"], "redirect80": false, "pool": "pool_NGINX_API_AS3", "policyWAF": { "use": "Arcadia_WAF_API_policy" }, "securityLogProfiles": [{ "bigip": "/Common/Log all requests" }], "profileTCP": { "egress": "wan", "ingress": { "use": "TCP_Profile" } }, "profileHTTP": { "use": "custom_http_profile" }, "serverTLS": { "bigip": "/Common/arcadia_client_ssl" } }, "Arcadia_WAF_API_policy": { "class": "WAF_Policy", "url": "http://10.1.20.4/root/as3-waf-api/-/raw/master/policy-api.json", "ignoreChanges": true }, "pool_NGINX_API_AS3": { "class": "Pool", "monitors": ["http"], "members": [{ "servicePort": 8080, "serverAddresses": ["10.1.20.9"] }] }, "custom_http_profile": { "class": "HTTP_Profile", "xForwardedFor": true }, "TCP_Profile": { "class": "TCP_Profile", "idleTimeout": 60 } } } } } 6. CI/CID integration As you can notice, it is very easy to create a service with a WAF policy pulled from an external repo. So, it is easy to integrate these calls (or the AS3 call) into a CI/CD pipeline. Below, an Ansible playbook example. This playbook run the AS3 call above. That's it :) --- - hosts: bigip connection: local gather_facts: false vars: my_admin: "admin" my_password: "admin" bigip: "10.1.1.12" tasks: - name: Deploy AS3 WebApp uri: url: "https://{{ bigip }}/mgmt/shared/appsvcs/declare" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic YWRtaW46YWRtaW4=" body: "{{ lookup('file','as3.json') }}" body_format: json validate_certs: no status_code: 200 7. FIND the Policy-ID When the policy is created, a Policy-ID is assigned. By default, this ID doesn't appearanywhere. Neither in the GUI, nor in the response after the creation. You have to calculate it or ask for it. This ID is required for several actions in a CI/CD pipeline. 7.1 Calculate the Policy-ID Wecreated this python script to calculate the Policy-ID. It is an hash from the Policy name (including the partition). For the previous created policy named"/Common/policy-api-arcadia",the policy ID is"Ar5wrwmFRroUYsMA6DuxlQ" Paste this python codein a newwaf-policy-id.pyfile, and run the commandpython waf-policy-id.py "/Common/policy-api-arcadia" Outcome will beThe Policy-ID for /Common/policy-api-arcadia is: Ar5wrwmFRroUYsMA6DuxlQ #!/usr/bin/python from hashlib import md5 import base64 import sys pname = sys.argv[1] print 'The Policy-ID for', sys.argv[1], 'is:', base64.b64encode(md5(pname.encode()).digest()).replace("=", "") 7.2 Retrieve the Policy-ID and fullPath with a REST API call Make this call below, and you will see in the response, all the policy creations. Find yours and collect thePolicyReference directive.The Policy-ID is in the link value "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0" You can see as well, at the end of the definition, the "fileReference"referring to the JSON file pulled by the BIG-IP. And please notice the"fullPath", required if you want to update your policy curl --location --request GET 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Range: 0-601/601' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ { "isBase64": false, "executionStartTime": "2020-07-22T11:23:42Z", "status": "COMPLETED", "lastUpdateMicros": 1.595417027e+15, "getPolicyAttributesOnly": false, "kind": "tm:asm:tasks:import-policy:import-policy-taskstate", "selfLink": "https://localhost/mgmt/tm/asm/tasks/import-policy/B45J0ySjSJ9y9fsPZ2JNvA?ver=16.0.0", "filename": "", "policyReference": { "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0", "fullPath": "/Common/policy-api-arcadia" }, "endTime": "2020-07-22T11:23:47Z", "startTime": "2020-07-22T11:23:42Z", "id": "B45J0ySjSJ9y9fsPZ2JNvA", "retainInheritanceSettings": false, "result": { "policyReference": { "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0", "fullPath": "/Common/policy-api-arcadia" }, "message": "The operation was completed successfully. The security policy name is '/Common/policy-api-arcadia'. " }, "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" } }, 8 UPDATE an existing policy It is pretty easy to update the WAF policy from a new JSON file version. To do so, collect from the previous call7.2 Retrieve the Policy-ID and fullPath with a REST API callthe"Policy" and"fullPath"directive. This is the path of the Policy in the BIG-IP. Then run the call below, same as1.2 PULL JSON file from a repository,but add thePolicy and fullPath directives Don't forget to APPLY this new version of the policy3. APPLY the policy curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" }, "policy": { "fullPath":"/Common/policy-api-arcadia" } }' TIP : this call, above, can be used in place of the FIRST call when we created the policy "1.2PULL JSON file from a repository". But be careful, the fullPath is the name set in the JSON policy file. The 2 values need to match: "name": "policy-api-arcadia" in the JSON Policy file pulled by the BIG-IP "policy":"fullPath" in the POST call 9 Video demonstration In order to help you to understand how it looks with the BIG-IP, I created this video covering 4 topics explained in this article : The JSON WAF policy Pull the policy from a remote repository Update the WAF policy with a new version of the declarative JSON file Deploy a full service with AS3 and Declarative WAF policy At the end of this video, you will be able to adapt the REST Declarative API calls to your infrastructure, in order to deploy protected services with your CI/CD pipelines. Direct link to the video on DevCentral YouTube channel : https://youtu.be/EDvVwlwEFRw
Implementing BIG-IP WAF logging and visibility with ELK
Scope This technical article is useful for BIG-IP users familiar with web application security and the implementation and use of the Elastic Stack.This includes, application security professionals, infrastructure management operators and SecDevOps/DevSecOps practitioners. The focus is for WAF logs exclusively.Firewall, Bot, or DoS mitigation logging into the Elastic Stack is the subject of a future article. Introduction This article focusses on the required configuration for sending Web Application Firewall (WAF) logs from the BIG-IP Advanced WAF (or BIG-IP ASM) module to an Elastic Stack (a.k.a. Elasticsearch-Logstash-Kibana or ELK). First, this article goes over the configuration of BIG-IP.It is configured with a security policy and a logging profile attached to the virtual server that is being protected. This can be configured via the BIG-IP user interface (TMUI) or through the BIG-IP declarative interface (AS3). The configuration of the Elastic Strack is discussed next.The configuration of filters adapted to processing BIP-IP WAF logs. Finally, the article provides some initial guidance to the metrics that can be taken into consideration for visibility.It discusses the use of dashboards and provides some recommendations with regards to the potentially useful visualizations. Pre-requisites and Initial Premise For the purposes of this article and to follow the steps outlined below, the user will need to have at least one BIG-IP Adv. WAF running TMOS version 15.1 or above (note that this may work with previous version but has not been tested).The target BIG-IP is already configured with: A virtual Server A WAF policy An operational Elastic Stack is also required. The administrator will need to have configuration and administrative privileges on both the BIG-IP and Elastic Stack infrastructure.They will also need to be familiar with the network topology linking the BIG-IP with the Elastic Search cluster/infrastructure. It is assumed that you want to use your Elastic Search (ELK) logging infrastructure to gain visibility into BIG-IP WAF events. Logging Profile Configuration An essential part of getting WAF logs to the proper destination(s) is the Logging Profile.The following will go over the configuration of the Logging Profile that sends data to the Elastic Stack. Overview of the steps: Create Logging Profile Associate Logging Profile with the Virtual Server After following the procedure below On the wire, logs lines sent from the BIG-IP are comma separated value pairs that look something like the sample below: Aug 25 03:07:19 localhost.localdomainASM:unit_hostname="bigip1",management_ip_address="192.168.41.200",management_ip_address_2="N/A",http_class_name="/Common/log_to_elk_policy",web_application_name="/Common/log_to_elk_policy",policy_name="/Common/log_to_elk_policy",policy_apply_date="2020-08-10 06:50:39",violations="HTTP protocol compliance failed",support_id="5666478231990524056",request_status="blocked",response_code="0",ip_client="10.43.0.86",route_domain="0",method="GET",protocol="HTTP",query_string="name='",x_forwarded_for_header_value="N/A",sig_ids="N/A",sig_names="N/A",date_time="2020-08-25 03:07:19",severity="Error",attack_type="Non-browser Client,HTTP Parser Attack",geo_location="N/A",ip_address_intelligence="N/A",username="N/A",session_id="0",src_port="39348",dest_port="80",dest_ip="10.43.0.201",sub_violations="HTTP protocol compliance failed:Bad HTTP version",virus_name="N/A",violation_rating="5",websocket_direction="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="N/A",staged_threat_campaign_names="N/A",blocking_exception_reason="N/A",captcha_result="not_received",microservice="N/A",tap_event_id="N/A",tap_vid="N/A",vs_name="/Common/adv_waf_vs",sig_cves="N/A",staged_sig_cves="N/A",uri="/random",fragment="",request="GET /random?name=' or 1 = 1' HTTP/1.1\r\n",response="Response logging disabled" Please choose one of the methods below.The configuration can be done through the web-based user interface (TMUI), the command line interface (TMSH), directly with a declarative AS3 REST API call, or with the BIG-IP native REST API.This last option is not discussed herein. TMUI Steps: Create Profile Connect to the BIG-IP web UI and login with administrative rights Navigate to Security >> Event Logs >> Logging Profiles Select “Create” Fill out the configuration fields as follows: Profile Name (mandatory) Enable Application Security Set Storage Destination to Remote Storage Set Logging Format to Key-Value Pairs (Splunk) In the Server Addresses field, enter an IP Address and Port then click on Add as shown below: Click on Create Add Logging Profile to virtual server with the policy Select target virtual server and click on the Security tab (Local Traffic >> Virtual Servers : Virtual Server List >> [target virtualserver] ) Highlight the Log Profile from the Available column and put it in the Selected column as shown in the example below (log profile is “log_all_to_elk”): Click on Update At this time the BIG-IP will forward logs Elastic Stack. TMSH Steps: Create profile ssh into the BIG-IP command line interface (CLI) from the tmsh prompt enter the following: create security log profile [name_of_profile] application add { [name_of_profile] { logger-type remote remote-storage splunk servers add { [IP_address_for_ELK]:[TCP_Port_for_ELK] { } } } } For example: create security log profile dc_show_creation_elk application add { dc_show_creation_elk { logger-type remote remote-storage splunk servers add { 10.45.0.79:5244 { } } } } 3. ensure that the changes are saved: save sys config partitions all Add Logging Profile to virtual server with the policy 1.From the tmsh prompt (assuming you are still logged in) enter the following: modify ltm virtual [VS_name] security-log-profiles add { [name_of_profile] } For example: modify ltm virtual adv_waf_vs security-log-profiles add { dc_show_creation_elk } 2.ensure that the changes are saved: save sys config partitions all At this time the BIG-IP sends logs to the Elastic Stack. AS3 Application Services 3 (AS3) is a BIG-IP configuration API endpoint that allows the user to create an application from the ground up.For more information on F5’s AS3, refer to link. In order to attach a security policy to a virtual server, the AS3 declaration can either refer to a policy present on the BIG-IP or refer to a policy stored in XML format and available via HTTP to the BIG-IP (ref. link). The logging profile can be created and associated to the virtual server directly as part of the AS3 declaration. For more information on the creation of a WAF logging profile, refer to the documentation found here. The following is an example of a pa rt of an AS3 declaration that will create security log profile that can be used to log to Elastic Stack: "secLogRemote": { "class": "Security_Log_Profile", "application": { "localStorage": false, "maxEntryLength": "10k", "protocol": "tcp", "remoteStorage": "splunk", "reportAnomaliesEnabled": true, "servers": [ { "address": "10.45.0.79", "port": "5244" } ] } In the sample above, the ELK stack IP address is 10.45.0.79 and listens on port 5244 for BIG-IP WAF logs.Note that the log format used in this instance is “Splunk”.There are no declared filters and thus, only the illegal requests will get logged to the Elastic Stack.A sample AS3 declaration can be found here. ELK Configuration The Elastic Stack configuration consists of creating a new input on Logstash.This is achieved by adding an input/filter/ output configuration to the Logstash configuration file.Optionally, the Logstash administrator might want to create a separate pipeline – for more information, refer to this link. The following is a Logstash configuration known to work with WAF logs coming from BIG-IP: input { syslog { port => 5244 } } filter { grok { match => { "message" => [ "attack_type=\"%{DATA:attack_type}\"", ",blocking_exception_reason=\"%{DATA:blocking_exception_reason}\"", ",date_time=\"%{DATA:date_time}\"", ",dest_port=\"%{DATA:dest_port}\"", ",ip_client=\"%{DATA:ip_client}\"", ",is_truncated=\"%{DATA:is_truncated}\"", ",method=\"%{DATA:method}\"", ",policy_name=\"%{DATA:policy_name}\"", ",protocol=\"%{DATA:protocol}\"", ",request_status=\"%{DATA:request_status}\"", ",response_code=\"%{DATA:response_code}\"", ",severity=\"%{DATA:severity}\"", ",sig_cves=\"%{DATA:sig_cves}\"", ",sig_ids=\"%{DATA:sig_ids}\"", ",sig_names=\"%{DATA:sig_names}\"", ",sig_set_names=\"%{DATA:sig_set_names}\"", ",src_port=\"%{DATA:src_port}\"", ",sub_violations=\"%{DATA:sub_violations}\"", ",support_id=\"%{DATA:support_id}\"", "unit_hostname=\"%{DATA:unit_hostname}\"", ",uri=\"%{DATA:uri}\"", ",violation_rating=\"%{DATA:violation_rating}\"", ",vs_name=\"%{DATA:vs_name}\"", ",x_forwarded_for_header_value=\"%{DATA:x_forwarded_for_header_value}\"", ",outcome=\"%{DATA:outcome}\"", ",outcome_reason=\"%{DATA:outcome_reason}\"", ",violations=\"%{DATA:violations}\"", ",violation_details=\"%{DATA:violation_details}\"", ",request=\"%{DATA:request}\"" ] } break_on_match => false } mutate { split => { "attack_type" => "," } split => { "sig_ids" => "," } split => { "sig_names" => "," } split => { "sig_cves" => "," } split => { "staged_sig_ids" => "," } split => { "staged_sig_names" => "," } split => { "staged_sig_cves" => "," } split => { "sig_set_names" => "," } split => { "threat_campaign_names" => "," } split => { "staged_threat_campaign_names" => "," } split => { "violations" => "," } split => { "sub_violations" => "," } } if [x_forwarded_for_header_value] != "N/A" { mutate { add_field => { "source_host" => "%{x_forwarded_for_header_value}"}} } else { mutate { add_field => { "source_host" => "%{ip_client}"}} } geoip { source => "source_host" } } output { elasticsearch { hosts => ['localhost:9200'] index => "big_ip-waf-logs-%{+YYY.MM.dd}" } } After adding the configuration above to the Logstash parameters, you will need to restart the Logstash instance to take the new logs into configuration.The sample above is also available here. The Elastic Stack is now ready to process the incoming logs.You can start sending traffic to your policy and start seeing logs populating the Elastic Stack. If you are looking for a test tool to generate traffic to your Virtual Server, F5 provides a simpleWAF tester tool that can be found here. At this point, you can start creating dashboards on the Elastic Stack that will satisfy your operational needs with the following overall steps: ·Ensure that the log index is being created (Stack Management >> Index Management) ·Create a Kibana Index Pattern (Stack Management>>Index patterns) ·You can now peruse the logs from the Kibana discover menu (Discover) ·And start creating visualizations that will be included in your Dashboards (Dashboards >> Editing Simple WAF Dashboard) A complete Elastic Stack configuration can be found here – note that this can be used with both BIG-IP WAF and NGINX App Protect. Conclusion You can now leverage the widely available Elastic Stack to log and visualize BIG-IP WAF logs.From dashboard perspective it may be useful to track the following metrics: -Request Rate -Response codes -The distribution of requests in term of clean, blocked or alerted status -Identify the top talkers making requests -Track the top URL’s being accessed -Top violator source IP An example or the dashboard might look like the following:How I did it - "Visualizing Data with F5 TS and Splunk"
The new Splunk Add-on for F5 BIG-IPincludes several objects, (modular inputs, CIM-knowledge, etc.) that work to “normalize” incoming BIG-IP data for use with other Splunk apps, such as Splunk Enterprise Security and the Splunk App for PCI Compliance. The add-on includes a mechanism for pulling network traffic data, system logs, system settings, performance metrics, and traffic statistics from the F5 BIG-IP platform using F5’s iControl API, (see below). But what I'm really excited about is that the add-on now integrates with F5 Telemetry Streaming, (TS).With TS I am easily able to declaratively aggregate, normalize, and push BIG-IP statistics and events, (JSON-formatted) to a variety of third-party analytics vendors. For the remainder of this article, we’ll take a look at how I integrate F5 TS with Splunk Enterprise.I’ll be working with an existing BIG-IP deployment as well as a newly deployed Splunk Enterprise instance.As an added bonus, (and since it’s part of the article’s title) I’ll import a couple custom dashboards, (see below) to visualize our newly ingested telemetry data. Oh! As an "Extra" added bonus, here is a link to a video walk through of this solution. Installing the Splunk Add-on for F5 BIG-IP and Splunk CIM Installing the Splunk F5 add-on is very simple.Additionally, to make use of the add-on I’ll need to install Splunk’s Common Information Model, (CIM). 1.From the top Splunk the search page, I select ‘Apps’ → ‘Find More Apps’. 2.I browse for “CIM” and select the Splunk Common Information Model add-on. 3.I accept the license agreement, provide my Splunk account login credentials and select ‘Login and Install’. 4.I’ll repeat steps 2-3 to install the Splunk Add-on for F5 BIG-IP. Setup Splunk HTTP Event Collector To receive incoming telemetry data into my Splunk Enterprise environment over HTTP/HTTPs I will need to create an HTTP Event Collector. 1.From the UI I select ‘Settings’ → ‘Data Inputs’.I select ‘HTTP Event Collector’ from the input list. 2.Prior to creating a new event collector token, I must first enable token access for my Splunk environment. On the ‘HTTP Event Collector’ page, I select ‘Global Settings’.I set‘All Tokens’ to enabled, default index, incoming port and ensure SSL is enabled.I click ‘Save’ to exit. 3.I select ‘New Token’ and provide a name for the new collector and select ‘Next’. 4.On the ‘Input Settings’ tab I’ll select my allowed index(es) and select ‘Review’ then ‘Submit’. 5.Once the token is created, I will need to copy the token for use with my F5 TS configuration. Configure Telemetry Streaming With my Splunk environment ready to receive telemetry data, I now turn my attention to configuring the BIG-IP for telemetry streaming.Fortunately, F5’s Automation Toolchain configuring the BIG-IP is quite simple. 1.I’ll use Postman to POST an AS3 declaration to configure telemetry resources, (telemetry listener, log publisher, logging profiles, etc.). The above AS3 declaration, (available here) deploys the required BIG-IP objects for pushing event data to a third-party vendor. Notably, it creates four (4) logging profiles I’ll attach to my application’s virtual server. 2.Still using Postman, I POST my TS declaration, (sample).I will need to provide my Splunk HTTP Collector endpoint address/port as well as the token generated previously. Associate Logging Profiles to Virtual Server The final step to configuring the BIG-IP for telemetry streaming is associating the logging profiles I just created with my existing virtual server. In addition to system telemetry, these logging profiles, when assigned to a virtual,will send LTM, AVR, and ASM telemetry. 1.From the BIG-IP management UI, I select ‘Local Traffic’ → ‘Virtual Servers’ → <virtual>. 2.Under ‘Configuration’ I select ‘Advanced’, scroll down and select the HTTP, TCP, and request logging profiles previously created.I select ‘Update’ at the bottom of the page to save 3.From the top of the virtual server page, I select ‘Security’ → ‘Policies’.From the policy settings page, I can see that there is an existing WAF policy associated with my application.To enable ASM logging, I select the previously created ASM logging profile from the available logging profiles and select ‘Update’ to save my changes. With the configuration process complete, I should now start seeing event data in my Splunk Environment. Import Dashboards “Ok, so I have event data streaming into my Splunk environment; now what?” Since I have installed the Splunk F5 add-on, I can integrate my “normalized” data with other data sources to populate various Splunk applications like Splunk Enterprise Security and Splunk App for PCI Compliance.Likewise, I can use dashboards to visualizemy telemetry data as well as monitor BIG-IP resources/processes.To finish up, I’ll use the following steps to create custom dashboards visualizing BIG-IP metrics and Advanced WAF, (formerly ASM) attack information. 1.From the Splunk Search page, I navigate to the Dashboards page by selecting ‘Dashboards’. 2.Select ‘Create New Dashboard’ from the Dashboards page. 3.Provide a name for the new dashboard and select ‘Create Dashboard’.The dashboard name, (ID will remain unchanged) will be updated in the next step where I replace the newly created dashboard’s XML source with one of the community-supported dashboard XML files here. 4.On the ‘Edit Dashboard' screen I select ‘Source’ to edit the dashboard XML.I replace the existing XML data with the contents of the ‘advWafInsights.xml’ file.Select ‘Save’ to install the new dashboard. 5.I’ll repeat steps 1-4 using ‘bigipSystemMetrics.xml’ to install the BIG-IP metrics dashboard, Additional Links ·F5 Telemetry Streaming ·Splunk Add-on for F5 BIG-IP ·Splunk Common Information Model ·F5 Automation ToolchainConfiguring the BIG-IP and PHP "Hack-It-Yourself" Auction Site
I've been writing a series of articles on the features and capabilities of the BIG-IP ASM...here are the links for your reading pleasure: What Is The ASM? Policy Building The Importance of File Types, Parameters, and URLs In those articles I've used a fictitious auction site to test the ASM. Several people have inquired about this auction site and they wanted to know how to configure it so that they could get their own hands dirty with testing and configuring their ASM. Well, this article outlines the steps needed to get your own BIG-IP ASM and PHP Auction site up and running. Then, it's up to you to try out all the cool features of the ASM. Required Software I used virtual machines to set up and configure the BIG-IP and the auction site. You can use whatever hypervisor you want, but for the purposes of this article, I'll show you how it's done using VMware workstation. So, here's the list of software you will need to make all this happen: VMware Workstation (free trial available here). I'm using a Windows PC, but you can use Linux if you want. BIG-IP Virtual Edition (make sure you have the ASM module). The PHP Auction Site VMware files. UPDATE: The Google Drive folder that has all the download files is here:https://drive.google.com/drive/folders/0B4xC7Uw8ziLER3RqZXZWTFZTcW8?resourcekey=0-shweDpAzbgSyDTAkHqy-iw&usp=sharing Caveat: This article assumes you already have a licensed and configured BIG-IP system, so I will just focus on the PHPAuction side of things. If you don't have the BIG-IP Virtual Edition, contact your sales rep and ask for a copy. Also, if you need help licensing and configuring the BIG-IP Virtual Edition, check out these solutions on AskF5: Licensing the BIG-IP system: http://support.f5.com/kb/en-us/solutions/public/7000/700/sol7752.html?sr=32111581 BIG-IP Virtual Edition Setup Guide for VMware: http://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/bigip-ve-setup-vmware-11-4-0.html Virtual Network Configuration Before loading up the PHP Auction site, you need to complete the following steps to configure the VMware Workstation Virtual Network Editor (shown in the picture below). If you have already set up your BIG-IP Virtual Edition, you may have already stepped through this configuration. If that's the case, you will want to make sure the settings for your VMnets are the same as the settings shown below. Delete any existing VMnets EXCEPT for VMnet0 Click the "Add Network" button, and add VMnet1, VMnet2, and VMnet3 VMnet0 Settings Since you kept the VMnet0 settings, you don't have to do anything else with that one. Here's a quick screenshot of the VMnet0 settings: VMnet1 Settings After you confirm the VMnet0 settings, select VMnet1 (this net will act as the out of band management) and configure it as follows: Select the Host-only (connect VMs internally in a private network) option Select the "Connect a host virtual adapter to this network" checkbox Clear the "Use local DHCP service to distribute IP address to VMs" checkbox For the Subnet IP, enter 10.128.1.0 and for the Subnet mask enter 255.255.255.0 VMnet2 Settings After you confirm the VMnet1 settings, select VMnet2 (this will act as the external VLAN for access to 0.0.0.0) and configure it as follows: Select the "NAT (shared host’s IP address with VMs)" option Select the "Connect a host virtual adapter to this network" checkbox Clear the "Use local DHCP service to distribute IP address to VMs" checkbox For the Subnet IP, enter 10.128.10.0 and for the Subnet mask enter 255.255.255.0 Click the "NAT Settings" button For the Gateway IP enter 10.128.10.1 and then click OK. VMnet3 Settings After you configure the VMnet2 settings, select VMnet3 (this will act as the internal VLAN where the server image exists) and configure it as follows: Host-only (connect VMs internally in a private network) option selected Connect a host virtual adapter to this network checkbox cleared Use local DHCP service to distribute IP address to VMs checkbox cleared For the Subnet IP enter 10.128.20.0 and for the Subnet mask enter 255.255.255.0 Ensure that the “Connect a host virtual adapter to this network” checkbox is cleared. This prevents your local workstation from having direct access to this network. This will avoid asymmetric routing issues and also allows you to demonstrate secure remote access and full proxy features. This concludes the network setup for VMware. Now you can install the PHP Auction virtual machine... Install the PHP Auction Image In VMware Workstation, select File >> Open and choose the "Open Virtual Machine Format" PHP Auction file (this will be a .ovf file). After you import the virtual machine (you may have to accept a few license agreements, etc), then you move on to configuring the virtual machine network settings. The following screenshot shows the settings for the PHP Auction virtual machine. You will need to customize the three Network Adapters to the settings shown in the picture below (you will use the three custom VMnets you just created), but you shouldn't need to configure anything else on this screen. After the virtual machine imports correctly, you can power it on and let it boot up. When the boot up completes, you will see the following login screen: The login is "root" and the password is "default". So, go ahead and login and then you can leave it alone...you won't need to touch it again. Configure the BIG-IP Now it's time to make sure the BIG-IP is set up correctly to protect the traffic flowing to/from the auction site. First, create a pool (I called it "auction_pool" but you can call it whatever you want) with node address 10.128.20.150 and service port 80. This pool only has one member. The following screenshot shows you the details for the auction pool. Next, you create the virtual server. I called it "auction_vs" but you can call it whatever you want...just don't call me Shirley. The IP address for the virtual server is 10.128.10.35, and I chose service port 443 so I could test out some crazy SSL capabilities. But, you can choose port 80 if you want. Keep in mind that if you choose port 443, you should create an SSL client profile and configure the appropriate certificate and key combo for the profile. The following screenshot shows the details of the virtual server configuration. The Final Touches OK...now that the BIG-IP is all set up and the PHP Auction virtual machine is running, you should be able to access the auction site from your browser of choice. One other optional thing you can do is add the virtual server IP address of the auction site to the "hosts" file in WIndows. You should be able to locate this file at: C: >> Windows >> System32 >> drivers >> etc. You can add the IP address to the file and then provide a host name for the auction site...the example below shows the name "auction.f5demo.com" but, again, you can choose something different if you want. So, the final check is to make sure all this works. Notice that the "auction.f5demo.com" worked as expected. You can also use the IP address "10.128.10.35" if you want. I hope this information is helpful. Feel free to comment and/or ask questions as needed. Also, be sure to check out the articles that dig deeper into the configurations and capabilities of the ASM!How BIG-IP can help secure your network against malware (Maze and Cloud Snooper)
In the F5 SIRT and F5 Support, we are often asked how F5 products can be used to defend against threats like ransomware, malware and rootkits. Threats like these can enter a target network through a multitude of vectors specific to the target endpoints – email spam campaigns, phishing and spear-phishing attacks, drive-by download vulnerabilities – but they can also enter the network through direct compromise of internet facing hosts. It’s tempting to look at F5 products and think they aren’t designed to detect a malicious binary being injected into your network, and while (with the exception of ICAP virus scanning of files uploaded to a webserver) that might be true, it is definitely not true to think that F5 products can’t be used as part of a defence-in-depth strategy to protect yourself from both compromise and post compromise exploitation. So let’s talk about a few ways you can bolster your network security using F5 products… The BIG-IP itself The first step is to understand that the BIG-IP is usually found sitting at the border of your network, therefore it is crucially important to protect the BIG-IP from compromise. If the BIG-IP is compromised then it doesn’t matter how good your Advanced WAF policies are or your AFM rulesets, the attacker has a neat jump-box which they can use to pivot into the rest of your network. Fortunately, protecting yourself against exploit here is as simple as following a few basic principles: Don’t expose the management interface to the Internet (if at all possible) Ensure Self IP addresses have appropriate Port Lockdown settings Use strong passwords and definitely don’t leave the passwords at their defaults (fortunately, changing the passwords at installation is now mandated in recent versions) Monitor log files for suspicious activity (like unexpected logins) If you absolutely must expose the management interface to the internet, be sure to set appropriate ACLs to restrict access to specific IP ranges. All of which is documented in our Ask F5 article K13092 Protecting your (other) assets Now that you’ve got the BIG-IP sorted, you can turn your attention to how the BIG-IP(s) in your infrastructure can help protect your internal assets. As we discussed earlier malware can enter your network in a number of different ways, and a BIG-IP can help secure a number of those: Ensuring that your assets can only be accessed via the BIG-IP on ports you control immediately ensures you aren’t accidentally exposing SSH on your webserver, ruling out the possibility of an attacker simply brute-forcing credentials and uploading malware directly. The BIG-IP is (usually!) a full proxy right down to Layer 4, so any exploits or C2 channels that rely on quirks of TCP or utilise unusual header fields to transfer data will fail when the BIG-IP proxies and applies its own optimisations to the server-side flows Placing Advanced WAF in front of your internet-facing web services, configured with an appropriate policy, significantly reduces the possibility of an attacker exploiting a weakness in your web applications to upload a malicious payload (such as SQL Injection or shell command injection) The AFM IPS (in AFM 13.1.0 and later with an IPS subscription license) allows you to scan non-HTTP traffic for potentially malicious activity and take action BIG-IP APM allows you to secure access to potentially vulnerable protocols like RDP, better still, APM layered with Advanced WAF allows you to protect your VPN and secure endpoints against brute force attacks Of course, it is true to say that BIG-IP products are not best placed to stop a user from clicking on a malicious link in an email, but the points above at least help close some of the vectors that malware can enter your network! But, don’t stop there. I’m sure that as part of your day to day, you are on the look-out for IOCs to watch for that might indicate a problem within your network? If you are inspecting your outbound traffic (with BIG-IP AFM or SSLO) then it’s a snap to add rules to those products that will alert you and/or block outbound C2 communication attempts from compromised clients! A practical example – Cloud Snooper Cloud Snooper first appeared in the news in February 2020 billed as “malware that sneaks into your Linux servers”. Reading the Sophos white-paper (links to a non-F5 resource) it becomes clear that the researchers aren’t sure what the original infection vector is here; they know that the key marker is a specific rootkit present on the hosts, but not how that rootkit got there in the first place. Since there isn’t a specific initial infection vector, like most malware, we know that we need to think about everything from the previous two sections: Protect the BIG-IP by ensuring access to the management interface and control-plane is not exposed to attackers Pass inbound traffic through inspection appropriate to the protocols the servers are exposing to the internet (Advanced WAF, AFM) to reduce the potential for malware to be injected into the network through compromise of an internet-facing application The white-paper tells us that the rootkit dropped initially is a local command-and-control system rather than the ultimate exploit; its job is to receive incoming C2 messages and arbitrate between those and malware subsequently installed. It does that by sniffing all of the network traffic arriving at the compromised host and inspecting the TCP source port of every incoming packet. If the source port of the packet matches one of the ‘magic’ numbers hard-coded in the rootkit then it is treated as a command, otherwise it is simply ignored, and the host processes it as normal. So we've a third item to add to our list above: Pass inbound traffic through the BIG-IP in order to block incoming C2 messages destined for the rootkit on any potentially compromised or at-risk host Good news! With even the default configuration in place there’s a good chance that the BIG-IP won’t preserve the TCP source port between the client side and the server side, especially on a system passing a reasonable amount of traffic. But you can go one step further – simply change the “Source Port” option on the Virtual Server from “Preserve” to “Change” and the BIG-IP will always try to change the source port! The C2 messages might arrive at the Virtual Server, but by the time they leave and head to the pool member they’ll be on a new port and the rootkit will completely ignore them. Want to go one step further and ensure that C2 traffic never reaches the server at all? BIG-IP AFM has you covered; just create a Network Firewall rule to deny all traffic with a source port of 1010, 2020, 6060, 7070, 8080 or 9999, assign the rule to a Rule List, assign the Rule List to a Network Firewall Policy and then assign that Network Firewall policy to any Virtual Servers you’re concerned about or even to all traffic passing through the BIG-IP. As the original white-paper notes, regular clients don’t normally use source ports below 32768 so legitimate traffic shouldn’t be affected – but simply by assigning a logging profile with Network Firewall logging enabled will let you see what’s being denied by the rule. See the following chapters in the manual (BIG-IP Network Firewall: Policies and Implementations) for your version for more information: Configuring BIG-IP Network Firewall Policies Local Logging with the Network Firewall If you don’t have AFM, we’ve still got you covered. A simple iRule attached to a Virtual Server provides a simple way to block incoming Cloud Snooper C2 communications: when CLIENT_ACCEPTED { switch -- [TCP::client_port] { 1010 - 2020 - 6060 - 7070 - 8080 - 9999 { # Uncomment the following to log locally, which should be used for debugging purposes only # log local0. "Reject possible Cloud Snooper C2 message from [IP::client_addr] with source port [TCP::client_port]" reject } default { # Not a Cloud Snooper C2 connection, continue as normal } } } Of course, an iRule doesn’t come with the many advantages of AFM's Network Firewall Policies – it’s more computationally expensive, there isn’t simple access to remote logging compatible with leading SIEM systems, and it isn’t as easily administered if needs change. But, still, it’s there and it’s an excellent option – the F5 SIRT makes regular use of iRules in order to stop the bleeding and get customers back up and running in a pinch! Cloud Snooper Conclusion In brief, to protect against Cloud Snooper: Protect the BIG-IP by ensuring access to the management interface and control-plane is not exposed to attackers Pass inbound traffic through inspection appropriate to the protocols the servers are exposing to the internet (Advanced WAF, AFM) to reduce the potential for malware to be injected into the network through compromise of an internet-facing application Pass inbound traffic through the BIG-IP in order to block incoming C2 messages destined for the rootkit on any potentially compromised or at-risk host, setting the "Source Port" option to "Change" on the appropriate Virtual Server or use a Network Firewall ruleset or iRule to block incoming connections with a TCP source port of 1010, 2020, 6060, 7070, 8080 or 9999 Example 2 – Maze The Maze ransomware has been in the news regularly since January 2020 and shares a lot of traits with previously identified ransomware – infection is likely to be through vectors such as phishing, brute force access to network infrastructure like RDP or compromising internet-facing hosts. One thing that is new (but not unique) with Maze is what it does with the data once it’s in; it doesn’t just encrypt files, it syphons them off, so the target is not only being coerced into paying to regain access to their data, but also to avoid their data being exposed on the internet. So, everything we discussed in the previous sections applies equally well to Maze as Cloud Snooper or any other malware infection, with the only difference being we aren't looking for inbound C2 messages but rather outbound data exfiltration: Protect the BIG-IP by ensuring access to the management interface and control-plane is not exposed to attackers Protect servers by placing them behind the BIG-IP and passing outbound traffic through inspection (AFM, SSLO) Pass inbound traffic through inspection appropriate to the protocols the servers are exposing to the internet (Advanced WAF, AFM) to reduce the potential for malware to be injected into the network through compromise of an internet-facing application It’s currently thought that Maze exfiltrates this data en-masse via outbound FTP connections, so protecting against that threat using F5 products is as simple as passing your outbound traffic through a Virtual Server and using Network Firewall policies or iRules to block all outbound FTP or to block or alert on outbound traffic destined to one of the published IOC IP addresses for Maze. Maze Conclusion In brief, to protect against Maze: Protect the BIG-IP by ensuring access to the management interface and control-plane is not exposed to attackers Protect servers by placing them behind the BIG-IP and passing outbound traffic through inspection (AFM, SSLO), blocking outbound FTP connections from any sensitive hosts to prevent Maze from exfiltrating data Pass inbound traffic through inspection appropriate to the protocols the servers are exposing to the internet (Advanced WAF, AFM) to reduce the potential for malware to be injected into the network through compromise of an internet-facing application Conclusion Admit it, you skipped right down to this section, didn’t you? It’s OK – we are all busy people, I understand! Here’s my five-bullet BIG-IP takeaway: Ensure you secure your BIG-IP first; failure to do so renders anything else you do moot. Follow the steps in K13092! Simply placing your servers behind a BIG-IP makes many attacks significantly harder to pull off, by virtue of the BIG-IPs full-proxy architecture If it serves HTTP, put Advanced WAF in front of it Put AFM in front of everything, including your internal clients accessing the internet! If you can’t use one of the products, iRules are your friend and can help you stop inbound and outbound threats And, finally, my general advice five-bullet takeaway – things that apply equally to F5 products and general-purpose computing: Never expose the management interface of anything directly to the internet; if you must, use ACLs to restrict access Visibility is king; you need visibility of traffic in and out of your network to watch for IOCs. If you can’t use F5 products (Advanced WAF, AFM and SSLO have excellent reporting capabilities) then use something and pipe all that data into an SIEM Patch in a timely manner Don’t use weak, insecure or previously leaked passwords Did I mention never exposing the management interface to the internet yet? Those ten things, the first five of which are covered in more detail in the article above, will go a long way to ensuring the security of your network and your assets. Of course, the BIG-IP and good security practices can't obviate the need for good endpoint security, MFA for your users and so on, but you'll at least have squashed a significant amount of attack surface.Machine learning is nothing new; the BIG-IP ASM system has been learning for more than a decade
One of the top trending topics in technology is how to use artificial intelligence (AI) and machine learning (ML) to resolve issues in various technologies. Prime among the technologies is application security (AppSec).AI and ML are poised to fix most issues that occur in AppSec. Specifically, ML Advanced Deep Learning Recurrent Neural Network algorithms are being developed to drastically improve the accuracy of threat detection. AI and ML predict attacks (including zero-day attacks) and stop the attacks before they reach your apps. Simply put, ML consists of training the system (the learning part of ML) to, on its own, detect an attack, and not rely on preconfigured signatures or anomaly-based logic to detect the attack. Thefoundational premiseis simple: the system (machine) is exposed to data sets and learns to detect threats from legitimate traffic. Application learning is not ML, but the premise is the same Enter BIG-IP ASM! For many years, the BIG-IP ASM system has had a learning engine (traffic learning). Like thefoundational premiseof ML, the BIG-IP ASM learning engine premise is simple: learn from the application traffic to automatically build or refine a security policy that detects threats. This method significantly reduces the complexity and number of signatures that you have to manually configure. In academia, this functionality is sometimes referred to as Adaptive Learning or Application Learning. The BIG-IP ASM learning game The BIG-IP ASM learning engine examines ingress and egress web traffic and intelligently learns the intended behavior of the application. That learningfeedsthe BIG-IP ASM Policy Builder, which can automatically configure the security controls required to protect your application. When using automatic learning, the system continuously learns as traffic is processed and can suggest new security controls as the application’s design or usage evolves.Tounderstandhow to leverage the BIG-IP ASM learning engine, refer toK75376155: Creating a security policy automatically (14.x - 15.x). Areas where AI and ML elevate the learning game Having the right data set, learning offline and protecting learning are essentialwhen detecting threatsand stopping attacks. The data set issue Just as it is difficult to learn Geography while being taught Chemistry, learning in AppSec requires the correct data set (consisting of both legitimate traffic and threats). Because of the agile nature of application development (the shorter release cycle) and the dynamic nature of threats, AI and ML can improve the data set quality in terms of scope, context, and retention period (the window of time before you stop analyzing a particular data set). Offline detection Offline detection (as opposed to real-time detection) refers to the process of analyzing and learning from a dataset outside of production traffic. AI and ML can learn from a large pool of datasets: internet traffic, honeypot systems, and partners sites, and AI and ML can also detect anomalies that identify attacks. Security controls can, therefore, be proactively developed and implemented on your WAF. Securing the learning engine Another thing that AI and ML tackle is protecting learning, itself. If attackerscouldfool or influence the way the ML-based system operates, they could nullify the core proposition of the AI and ML-based security systems, which is predicting and stopping attacks before they reach your apps. Overall, advances in AI and ML have exponentially empowered learning by addressing the data set issue and protecting the learning engine. These advances allow AI and ML systems to improve the speed and accuracy of threat detection (including zero-day attacks) while significantly reducing thefalsepositive rate.
