Javascript injecting systems effect on web application end users - a scenario review
Hello! ArvinF is back to share a scenario review where Javascript-injecting systems affected web application end users - web and mobile application. Problem Users are failing to login to a web application protected by BIG-IP ASM/Adv WAF and Shape Security Defense. The site owner notes that the authentication was failing for an unknown reason. There were ASM Support ID noted and an error informing to enable Javascript. Please enable JavaScript to view the page’s content. Your support ID is: xxxxxxxxxxxx Troubleshooting To understand the cause of the authentication failure, we gathered HTTP traffic through a HTTP sniffer. We used httpwatch and gathered HAR (HTTP Archive) files. The site was protected with both on-premise BIG-IP ASM/Adv WAF bot defense and back then, Shape Security Defense (now F5 Distributed Cloud Bot Defense). After the review of the HAR file in httpwatch, the following were noted: ASM blocks a request in a URL related to authentication with a Support ID in the response. There was also javascript code included and it references https[:]//s[.]go-mpulse[.]net/boomerang/. The authentication attempt failed with an error in the HTTP response: ...unable to process your request. Please try again later... BIG-IP ASM/Adv WAF related HTTP cookies from its various features such as Bot Defense Client Side challenges as TSPD_101* cookie was present and other TS cookies, which could also come from Bot defense and DoS profile and security policy configurations. There were also HTTP cookies coming from BIG-IP AVR - f5_cspm cookie was present. Application Visibility and Reporting (AVR) module provides detailed charts and graphs to give you more insight into the performance of web applications, with detailed views on HTTP and TCP stats, as well as system performance (CPU, memory, etc.). https://clouddocs.f5.com/training/community/analytics/html/index.html https://clouddocs.f5.com/api/irules/AVR_CSPM_INJECTION.html Seeing the javascript code referencing "/boomerang/" included in the ASM blocking response was interesting. Reviewing the HAR file, there were several instances of this "/boomerang/". This finding was inquired with the site owner and they noted that there is another system that is in the path between the end users and their web application - a CDN. The traffic flow is as follows: End user web browser / mobile application >>> CDN >>> FW >>> BIG-IP >>> web application On the BIG-IP Virtual Server that fronts the web application, F5 AVR profile, ASM/Adv WAF Bot defense, and security policy and Shape Security defense iRule are configured. From the F5 side, these were the products with features that may insert Javascript in the client-side response. As part of troubleshooting, to isolate the feature that might be causing the failing authentication for the web application, the bot defense profile was removed from the site's Virtual Server and the Shape Security iRule and AVR profile were left untouched. Site owner noted that the authentication works after this change. Shape Security Defense was implemented using an iRule to protect specific URIs. When the iRule was removed from the Virtual Server and the Bot defense and AVR profile were left on, the VS, Site owner noted that the authentication works after this change. But if both ASM/Adv WAF Bot defense and Shape Security Defense iRule is configured on the VS, the site's authentication fails. Per the site owner, there were no changes in the Bot Defense or Shape Security Defense iRule configurations prior to the incident and that these configurations were in place way before the incident. Site owners shared the findings with their respective internal teams for their review. Resolution Afterwards, Site owner shared that their site now works as expected and authentication works for the web application with no changes done on both ASM/Adv WAF Bot defense and Shape Security Defense iRule on the site's VS. The cause of the authentication failure was undetermined. A theory on the possible cause of the issue was perhaps, there was another system inserting Javascript code in the responses and it might have affected the authentication process of the web application by prevented that portion of the site from loading. Additional Troubleshooting Notes The data gathered during the troubleshooting were the qkview and HTTPWatch capture - HAR files. It would help if a packet capture was taken along with the HTTPWatch capture while the issue was happening to have a full view of the issue. Decrypt the packet capture to observe HTTP exchanges and to correlate it with HTTPWatch capture events. The corresponding BIG-IP ASM/Adv WAF application event logs, Bot Defense or DoS protection logs will also be helpful in the correlation. Having a visual idea on how the Security Policy, Bot Defense or DoS protection profile are configured is also helpful - so its good to have a screenshot of these. It helps in analysis when there is complete data. Gathering the asmqkview with report and traffic data and corresponding ASM and AVR db dumps helps in the analysis. asmqkview -s0 --add-request-log --include-traffic-data -f /var/tmp/`/bin/hostname`_asmqkview_`date +%Y%m%d%H%M%S`.tgz #mysqldump -uroot -p`perl -I/ts/packages -MPassCrypt -nle 'print PassCrypt::decrypt_password($_)' /var/db/mysqlpw` DCC | gzip -9 > /shared/tmp/dcc.dump.gz # mysqldump -uroot -p`perl -I/ts/packages -MPassCrypt -nle 'print PassCrypt::decrypt_password($_)' /var/db/mysqlpw` PLC | gzip -9 > /shared/tmp/plc.dump.gz # mysqldump -uroot -p`perl -I/ts/packages -MPassCrypt -nle 'print PassCrypt::decrypt_password($_)' /var/db/mysqlpw` PRX | gzip -9 > /shared/tmp/prx.dump.gz # mysqldump -uroot -p`perl -I/ts/packages -MPassCrypt -nle 'print PassCrypt::decrypt_password($_)' /var/db/mysqlpw` logdb | gzip -9 > /shared/tmp/logdb.dump.gz It would also help if the systems in the path of the web application are known and whether it has features that may interfere with the features of BIG-IP ASM/Adv WAF or Shape Security Defense. Per the findings, there was a CDN that was injecting javascript code in the HTTP response and it may have contributed to the authentication failure for the end users. Isolate potentially conflicting features by removing one of them one at a time and observe the HTTP responses. Per the reference configuration, BIG-IP ASM/Adv WAF, Shape Security Defense, and BIG-IP AVR worked well prior to the incident. boomerang The injected javascript code noted in the ASM blocking page response was loaded from https[:]//s[.]go-mpulse[.]net/boomerang/. Checking this reference, it was related to https://github.com/akamai/boomerang. boomerang is a JavaScript library that measures the page load time experienced by real users, commonly called RUM (Real User Measurement). It has the ability to send this data back to your server for further analysis. With boomerang, you find out exactly how fast your users think your site is. In BIG-IP, the similar product we have is BIG-IP AVR - Application Visibility and Reporting (AVR) - where it collects "performance of web applications, with detailed views on HTTP and TCP stats, as well as system performance (CPU, memory, etc.)." Organizations may have specific needs on data that they need to collect from their site/web application and using a customizable solution such as boomerang can help. That's It For Now I hope this scenario review on Javascript-injecting systems effect on web application end users will be helpful on your next troubleshooting and hopefully gives you guidance on what data to gather and look for and troubleshooting options. The F5 SIRT creates security-related content posted here in DevCentral, sharing the team’s security mindset and knowledge. Feel free to view the articles that are tagged with the following: F5 SIRT series-F5SIRT-this-week-in-security TWISSMTP Smugglers Blues
The SMTP protocol has been vulnerable to email smuggling for decades. Many of the mail servers out there have mitigations in place to handle this vulnerability but not all of them, especially the quick libraries and add-ons you can find on web sites. Protecting your server from these attacks is simple with F5 BIG-IP Advanced WAF and our SMTP Protocol Security profiles. Read to learn how to give those bad actors the “Smugglers Blues”
F5 BIG-IP Advanced WAF: OWASP Top 10 Application Security Risks 2021 Compliance Dashboard
Introduction The increase in vulnerabilities and application or API-related attacks exploiting those vulnerabilities has steadily risen. Vulnerabilities like Log4j, and the Log4Shell exploit are spawned and continue to impact many organizations even today. This is where a web application firewall (WAF) solution can protect your apps and APIs. One of the most respected authorities in web application security is the Open Web Application Security Project (OWASP). OWASP is an open-source project to improve web application security, a coalition of individual contributors and sponsor companies who come together to contribute resources to the project. One of the best-known resources the project delivers is the OWASP Top 10 List. Since web application vulnerability risks change frequently, becoming comparatively more or less critical over time, the OWASP Top 10 List is periodically updated to reflect these changes. The first version of the list was created in 2004, then updated in 2007, 2010, 2013, 2017, and again in 2021 (its most recent version). Figure 1: OWASP Top 10 Web Application Security Risks of 2021 F5 delivers a number of security solutions to help mitigate vulnerabilities in the OWASP categories, and the exploits that are produced from them. To ensure you’re compliant with the OWASP Top 10, F5 BIG-IP Advanced WAF offers a dedicated OWASP compliance dashboard that enables security admins to check how well their policy is set to defend against the OWASP Top 10 and allow organizations to easily reach 100% coverage. The solution makes it simple to modify policies to improve protection from exploit of vulnerabilities in the OWASP Top 10. The compliance dashboard provides a holistic and interactive view that shows the level of mitigation applied by SecOps team against the OWASP Top 10 vulnerability categories. It provides an overall assessment of the policies created and a percentage of how much the policies protect against the various vulnerability categories. The dashboard allows SecOps to increase/adjust the level of protection in real-time based on their needs by deploying pre-defined policies that mitigate the vulnerabilities and their associated exploits. This can be achieved directly from the BIG-IP Advanced WAF’s OWASP Top 10 2021 Dashboard, simplifying protection against known, unknown, and hidden vulnerabilities. Simple, quick, and easy vulnerability and exploit protection, from a single dashboard. Protection Overview Navigating to the OWASP Compliance screen, you can see the list of all the security policies. Clicking on a policy displays the OWASP compliance status for that policy and the coverage for each category. Figure 2: OWASP Compliance screen Expanding a category presents the compliance percentage, a description of that security risk, and the configuration required for full security coverage for this category. Each category is broken down into specific security protections, including positive and negative security controls that can be enabled, disabled, or ignored directly on the dashboard based on your organization’s requirements. Required Attack Signatures: Enforce all the relevant Attack Signatures for this attack type directly from the Dashboard. Required Policy Entities: Add protection configuration components such as Cookies and login Enforcement, data masking, Evasion techniques, detection, methods, URLs, and more relevant configurations for each attack type. In addition to WAF-specific security protections, the OWASP Compliance Dashboard also provides security Best Practices to follow in your processes, such as vulnerability scanning or using trusted repositories. Figure 3: OWASP category A03 Injection – protection and compliance The following video shows how to monitor the compliance coverage of security risks and how to quickly enhance an organization's security configuration directly from the dashboard to receive full compliance with protection from OWASP Top 10 vulnerabilities being actively exploited. Conclusion Web applications remain a top target for threats, such as automated attacks, data exfiltration, and vulnerabilities. But F5 can help! Not only can you check off regulatory compliance, but also be able to create reports via the security score relative to deployed policies that address the OWASP Top 10, enabling security admins to view each policy’s coverage status, improving protections if necessary, and even allowing security configuration to be performed directly from the dashboard. To learn more, please visit: How to deploy a basic OWASP Top 10 for 2021 compliant declarative WAF policy for BIG-IP K45215395: Guide introduction and contents | Secure against the OWASP Top 10 for 2021 K000135973: Guide Introduction and contents | APIs and the OWASP Top 10 guide (2023) Mitigating OWASP API Security risks using BIG-IP BIG-IP Advanced WAF Webpage Overview of BIG-IP
F5 powered API security and management
Editor's Note: The F5 Beacon capabilities referenced in this article hosted on F5 Cloud Services are planning a migration to a new SaaS Platform - Check out the latest here. Introduction Application Programming Interfaces (APIs) enable application delivery systems to communicate with each other. According to a survey conducted by IDC, security is the main impediment to delivery of API-based services. Research conducted by F5 Labs shows that APIs are highly susceptible to cyber-attacks. Access or injection attacks against the authentication surface of the API are launched first, followed by exploitation of excessive permissions to steal or alter data that is reachable via the API. Agile development practices, highly modular application architectures, and business pressures for rapid development contribute to security holes in both APIs exposed to the public and those used internally. API delivery programs must include the following elements : (1) Automated Publishing of APIs using Swagger files or OpenAPI files, (2) Authentication and Authorization of API calls, (3) Routing and rate limiting of API calls, (4) Security of API calls and finally (5) Metric collection and visualization of API calls. The reference architecture shown below offers a streamlined way of achieving each element of an API delivery program. F5 solution works with modern automation and orchestration tools, equipping developers with the ability to implement and verify security at strategic points within the API development pipeline. Security gets inserted into the CI/CD pipeline where it can be tested and attached to the runtime build, helping to reduce the attack surface of vulnerable APIs. Common Patterns Enterprises need to maintain and evolve their traditional APIs, while simultaneously developing new ones using modern architectures. These can be delivered with on-premises servers, from the cloud, or hybrid environments. APIs are difficult to categorize as they are used in delivering a variety of user experiences, each one potentially requiring a different set of security and compliance controls. In all of the patterns outlined below, NGINX Controller is used for API Management functions such as publishing the APIs, setting up authentication and authorization, and NGINX API Gateway forms the data path. Security controls are addressed based on the security requirements of the data and API delivery platform. 1. APIs for highly regulated business Business APIs that involve the exchange of sensitive or regulated information may require additional security controls to be in compliance with local regulations or industry mandates. Some examples are apps that deliver protected health information or sensitive financial information. Deep payload inspection at scale, and custom WAF rules become an important mechanism for protecting this type of API. F5 Advanced WAF is recommended for providing security in this scenario. 2. Multi-cloud distributed API Mobile App users who are dispersed around the world need to get a response from the API backend with low latency. This requires that the API endpoints be delivered from multiple geographies to optimize response time. F5 DNS Load Balancer Cloud Service (global server load balancing) is used to connect API clients to the endpoints closest to them. In this case, F5 Cloud Services Essential App protect is recommended to offer baseline security, and NGINX APP protect deployed closer to the API workload, should be used for granular security controls. Best practices for this pattern are described here. 3. API workload in Kubernetes F5 service mesh technology helps API delivery teams deal with the challenges of visibility and security when API endpoints are deployed in Kubernetes environment. NGINX Ingress Controller, running NGINX App Protect, offers seamless North-South connectivity for API calls. F5 Aspen Mesh is used to provide East-West visibility and mTLS-based security for workloads. The Kubernetes cluster can be on-premises or deployed in any of the major cloud provider infrastructures including Google’s GKE, Amazon’s EKS/Fargate, and Microsoft’s AKS. An example for implementing this pattern with NGINX per pod proxy is described here, and more examples are forthcoming in the API Security series. 4. API as Serverless Functions F5 cloud services Essential App Protect offering SaaS-based security or NGINX App Protect deployed in AWS Fargate can be used to inject protection in front of serverless API endpoints. Summary F5 solutions can be leveraged regardless of the architecture used to deliver APIs or infrastructure used to host them. In all patterns described above, metrics and logs are sent to one or many of the following: (1) F5 Beacon (2) SIEM of choice (3) ELK stack. Best practices for customizing API related views via any of these visibility solutions will be published in the following DevCentral series. DevOps can automate F5 products for integration into the API CI/CD pipeline. As a result, security is no longer a roadblock to delivering APIs at the speed of business. F5 solutions are future-proof, enabling development teams to confidently pivot from one architecture to another. To complement and extend the security of above solutions, organizations can leverage the power of F5 Silverline Managed Services to protect their infrastructure against volumetric, DNS, and higher-level denial of service attacks. The Shape bot protection solutions can also be coupled to detect and thwart bots, including securing mobile access with its mobile SDK.
