F5 Distributed Cloud Bot Defense Protecting AWS CloudFront Distributions
In this article, I will show you how to easily protect your AWS CloudFront distributions with F5 Distributed Cloud (XC) Bot Defense. We will take advantage of AWS Lambda@Edge and the AWS Serverless Application Repository (SAR) to integrate with the F5 XC Bot Defense API. Amazon CloudFront is a content delivery network (CDN) operated by Amazon Web Services. Content delivery networks provide a globally-distributed network of proxy servers that cache content, such as web videos or other bulky media, more locally to consumers, thus improving access speed for downloading the content. F5's Distributed Cloud Bot Defense combined with Amazon's CloudFront to protect your vital applications from malicious traffic is an effective and robust solution. General Overview of Architecture Create a new Bot Defense application for AWS CloudFront Log in to your F5 Distributed Cloud Console Go to the Dashboard page of XC console and click Bot Defense Verify you are in the correct Namespace. Click Add Application at the top-left of the page. Add a Name for the Application, and a Description. Select a region (US, EMEA, or APJC). For Connector Type, select AWS CloudFront. Once AWS CloudFront is selected, options appear to configure AWS reference details. Add AWS Reference Information Enter your AWS 12-digit Account Number. Specify your AWS Configuration and add your CloudFront distribution; a Distribution ID and/or a Distribution Tag. You can add one or more distributions. This information is needed to associate your newly created protected application to your AWS distribution(s). Add Protected Endpoints Click Configure to define your protected endpoints. Click Add Item Enter a name and a description to the specific endpoint. Specify the Domain Matcher. You can choose any domain or specify a specific host value. Specify the Path to the endpoint (such as /login). Choose the HTTP Methods for which request will be analyzed by Bot Defense. Multiple methods can be selected. Select the Client type that will access this endpoint (Web Client). Select the Mitigation action to be taken for this endpoint: Continue (request continues to origin) Redirect. Provide the appropriate Status Code and URI Block. Provide the Status Code, Content Type, and Response message When done configuring the endpoint, click Apply. To continue, click Apply at the bottom of the page. Define Continue Global Mitigation Action The Header Name for Continue Mitigation Action field is the header that is added to the request when the Continue mitigation action is selected and Add A Header was selected in the endpoint mitigation configuration screen. Define Web Client JavaScript Insertion Settings JS Location - Choose the location where to insert the JS in the code: Just After <head> tag. Just After </title> tag. Right Before <script> tag. Under Java Script Insertions. Select Configure. owing Javascript insertion menu Click Add Item Add the Web Client JavaScript Path. You should select paths to HTML pages that end users are likely to visit before they browse to any protected endpoint. Click Apply Click Save & Exit to save your protected application configuration. Download Config File and AWS Installer Tool In the Actions column of the table, click the 3 ellipses (…) on your application. Download both the Config File and the AWS Installer. Log in to your AWS Console Login to AWS Console home page. Select AWS Region Northern Virginia (US-EAST-1). Use the search to find Serverless Application Repository and click it Click Available Applications and search with "F5" Click the F5BotDefense tile This will take you to the Lambda page. Here you will be creating and deploying a Lambda Function Click Deploy to install the F5 Connector for CloudFront Deploying the F5 Connector creates a new Lambda Application in your AWS Account. AWS sets the name of the new Lambda Application to start with serverlessrepo-. The deployment can take some time. It is complete when you see the serverlessrepo-F5BotDefense-*of type Lambda Function. You can click on the nameto review contents of the installed Lambda Function. n details Switch to AWS CloudShell Configuration of the F5 Connector in AWS is best done via the F5 CLI tool. It is recommended to use the AWS CloudShell in us-east-1 region to avoid any issues. After starting AWS CloudShell, click Actions and Upload file. Upload the files you downloaded from the F5 XC Console, config.json and f5tool. (Only one file at a time can be uploaded) Run bash f5tool --install <config.json>. Installation can take up to 5 minutes. Note: Copy pasting the command may not work and so type it manually. The installation tool saves the previous configuration of each CloudFront Distribution in a file. You can use the F5 tool to restore a saved Distribution config (thus removing F5 Bot Defense). Note: Your F5 XC Bot Defense configuration, such as protected endpoints, is sensitive security info and is stored in AWS Secrets Manager. You should delete config.json after CLI installation. Validate CloudFront Distribution Functions Navigate to CloudFront > Distributions and select the distribution you are protecting. Then go to Behaviors Here under Behaviors are where you specify which request/response is forwarded to the Lambda@Edge Function to process with F5 XC Bot Defense. F5 XC Bot Defense requires us to leverage Viewer Request and Origin Request events. These events need to be available for user to use (IE they have not assigned other Functions) The AWS Installer tool that we downloaded from Distributed Cloud Console and ran in the AWS CloudShell configured this for us. AWS CloudWatch AWS CloudWatch contains logs for Lambda function deployed by F5BotDefense serverless application. The Log group name starts with /aws/lambda/us-east-1.serverlessrepo-F5BotDefense-F5BotDefense-*. The logs of lambda function can be found in the region closest to the location where the function executed. For troubleshooting, look for error messages contained in the links under Log steams. View Bot Traffic Now let’s return to F5 XC Console and show the monitoring page. Log in to your F5 Distributed Cloud Console Go to the Dashboard page of XC console and click Bot Defense. Make sure you are in the correct Namespace Under Overview click Monitor Here you can monitor and respond to events that are identified as Bot traffic. Conclusion That is all that is required to deploy F5 XC Bot Defense to protect your AWS Cloud Front distributions from mailicious bots protecting yourself from fraud and abuse. Related Articles: An overview of F5 Distributed Cloud Bot Defense How to easily protect your BIG-IP applications using F5's Distributed Cloud Bot Defense with iApps How to easily protect your BIG-IP applications using F5's Distributed Cloud Bot Defense, natively Related Video: Get Started: F5 Distributed Cloud Services F5 Distributed Cloud Bot Defense Brightboard Lesson
Use F5 Distributed Cloud to control Primary and Secondary DNS
Overview Domain Name Service (DNS); it's how humans and machines discover where to connect. DNS on the Internet is the universal directory of addresses to names. If you need to get support for the product Acme, you go to support.acme.com. Looking for the latest headlines in News, try www.aonn.com or www.npr.org. DNS is the underlying feature that nearly every service on the Internet depends on. Having a robust and reliable DNS provider is critical to keeping your organization online and working, and especially so during a DDoS attack. "Nature is a mutable cloud, which is always and never the same." - Ralph Waldo Emerson We might not wax that philosophically around here, but our heads are in the cloud nonetheless! Join the F5 Distributed Cloud user group today and learn more with your peers and other F5 experts. F5 Distributed Cloud DNS (F5 XC DNS) can function as both Primary or Secondary nameservers, and it natively includes DDoS protection. Using F5 XC DNS, it’s possible to provision and configure primary or secondary DNS securely in minutes. Additionally, the service uses a global anycast network and is built to scale automatically to respond to large query volumes. Dynamic security is included and adds automatic failover, DDoS protection, TSIG authentication support, and when used as a secondary DNS—DNSSEC support. F5 Distributed Cloud allows you to manage all of your sites as a single “logical cloud” providing: - A portable platform that spans multiple sites/clouds - A private backbone connects all sites - Connectivity to sites through its nodes (F5 Distributed Cloud Mesh and F5 Distributed Cloud App Stack) - Node flexibility, allowing it to be virtual machines, live on hardware within data centers, sites, or in cloud instances (e.g. EC2) - Nodes provide vK8s (virtual K8s), network and security services - Services managed through F5 Distributed Cloud’s SaaS base console Scenario 1 – F5 Distributed Cloud DNS: Primary Nameserver Consider the following; you're looking to improve the response time of your app with a geo-distributed solution, including DNS and app distribution. With F5 XC DNS configured as the primary nameserver, you’ll automatically get DNS DDoS protection, and will see an improvement in the response the time to resolve DNS just by using Anycast with F5’s global network’s regional point of presence. To configure F5 XC DNS to be the Primary nameserver for your domain, access the F5 XC Console, go to DNS Management, and then Add Zone. Alternately, if you're migrating from another DNS server or DNS service to F5 XC DNS, you can import this zone directly from your DNS server. Scenario 1.2 below illustrates how to import and migrate your existing DNS zones to F5 XC DNS. Here, you’ll write in the domain name (your DNS zone), and then View Configuration for the Primary DNS. On the next screen, you may change any of the default SOA parameters for the zone, and any type of resource record (RR) or record sets which the DNS server will use to respond to queries. For example, you may want to return more than one A record (IP address) for the frontend to your app when it has multiple points of presence. To do this, enter as many IP addresses of record type A as needed to send traffic to all the points of ingress to your app. Additional Resource Record Sets allows the DNS server to return more than a single type of RR. For example, the following configurations, returns two A (IPv4 address) records and one TXT record to the query of type ANY for “al.demo.internal”. Optionally, if your root DNS zone has been configured for DNSSEC, then enabling it for the zone is just a matter of toggling the default setting in the F5 XC Console. Scenario 1.2 - Import an Existing Primary Zone to Distributed Cloud using Zone Transfer (AXFR) F5 XC DNS can use AXFR DNS zone transfer to import an existing DNS zone. Navigate to DNS Management > DNS Zone Management, then click Import DNS Zone. Enter the zone name and the externally accessible IP of the primary DNS server. ➡️ Note: You'll need to configure your DNS server and any firewall policies to allow zone transfers from F5. A current list of public IP's that F5 uses can be found in the following F5 tech doc. Optionally, configure a transaction signature (TSIG) to secure the DNS zone transfer. When you save and exit, F5 XC DNS executes a secondary nameserver zone AXFR and then transitions itself to be the zone's primary DNS server. To finish the process, you'll need to change the NS records for the zone at your domain name registrar. In the registrar, change the name servers to the following F5 XC DNS servers: ns1.f5clouddns.com ns2.f5clouddns.com Scenario 1.3 - Import Existing (BIND format) Primary Zones directly to Distributed Cloud F5 XC DNS can directly import BIND formatted DNS zone files in the Console, for example, db.2-0-192.in-addr.arpa and db.foo.com. Enterprises often use BIND as their on-prem DNS service, importing these files to Distributed Cloud makes it easier to migrate existing DNS records. To import existing BIND db files, navigate to DNS Management > DNS Zone Management, click Import DNS Zone, then "BIND Import". Now click "Import from File" and upload a .zip with one or more BIND db zone files. The import wizard accepts all primary DNS zones and ignores other zones and files. After uploading a .zip file, the next screen reports any warnings and errors At this poing you can "Save and Exit" to import the new DNS zones or cancel to make any changes. For more complex zone configurations, including support for using $INCLUDE and $ORIGIN directives in BIND files, the following open source tool will convert BIND db files to JSON, which can then be copied directly to the F5 XC Console when configuring records for new and existing Primary DNS zones. BIND to XC-DNS Converter Scenario 2 - F5 Distributed Cloud DNS: Primary with Delegated Subdomains An enhanced capability when using Distributed Cloud (F5 XC) as the primary DNS server for your domains or subdomains, is to have services in F5 XC dynamically create their own DNS records, and this can be done either directly in the primary domain or the subdomains. Note thatbeforeJuly 2023, the delegated DNS feature in F5 XC required the exclusive use of subdomains to dynamically manage DNS records. As of July 2023, organizations are allowed to have both F5 XC managed and user-managed DNS resource records in the same domain or subdomain. When "Allow HTTP Load Balancer Managed Records" is checked, DNS records automatically added by F5 XC appear in a new RR set group called x-ves-io-managed which is read-only. In the following example, I've created an HTTP Load Balanacer with the domain "www.example.f5-cloud-demo.com" and F5 XC automatically created the A resource record (RR) in the group x-ves-io-managed. Scenario 3 – F5 Distributed Cloud DNS: Secondary Nameserver In this scenario, say you already have a primary DNS server in your on-prem datacenter, but due to security reasons, you don’t want it to be directly accessible to queries from the Internet. F5 XC DNS can be configured as a secondary DNS server and will both zone transfer (AXFR, IXFR) and receive (NOTIFY) updates from your primary DNS server as needed. To configure F5 XC DNS to be a secondary DNS server, go to Add Zone, then choose Secondary DNS Configuration. Next, View Configuration for it, and add your primary DNS server IP’s. To enhance the security of zone transfers and updates, F5 XC DNS supports TSIG encrypted transfers from the primary DNS server. To support TSIG, ensure your primary DNS server supports encryption, and enable it by entering the pre-shared key (PSK) name and its value. The PSK itself can be blindfold-encrypted in the F5 XC Console to prevent other Console admins from being able to view it. If encryption is desired, simply plug in the remaining details for your TSIG PSK and Apply. Once you’ve saved your new secondary DNS configuration, the F5 XC DNS will immediately transfer your zone details and begin resolving queries on the F5 XC Global Network with its pool of Anycast-reachable DNS servers. Conclusion You’ve just seen how to configure F5 XC DNS both as a primary DNS as well as a secondary DNS service. Ensure the reachability of your company with a robust, secure, and optimized DNS service by F5. A service that delivers the lowest resolution latency with its global Anycast network of nameservers, and one that automatically includes DDoS protection, DNSSEC, TSIG support for secondary DNS. Watch the following demo video to see how to configure F5 XC DNS for scenarios #1 and #3 above. Additional Resources For more information about using F5 Distributed Cloud DNS: https://www.f5.com/cloud/products/dns For technical documentation: https://docs.cloud.f5.com/docs/how-to/app-networking/manage-dns-zones DNS Management FAQ: https://f5cloud.zendesk.com/hc/en-us/sections/7057223802519-DNS-Management DNS Demo Guide and step-by-step walkthrough: https://github.com/f5devcentral/f5xc-dns BIND to XC-DNS Converter (open source tool):https://github.com/Mikej81/BINDtoXCDNSF5 Hybrid Security Architectures: One WAF Engine, Total Flexibility (Intro)
Layered security, we have been told for years that the most effective security strategy is composed of multiple, loosely coupled or independent layers of security controls. A WAF fits snuggly into the technical security controls area and has long been known as an essential piece of application security. What if we take this further and apply the layered approach directly to our WAF deployment? The F5 Hybrid Security Architectures explores this approach utilizing F5's best in class WAF products.F5 Distributed Cloud Bot Defense (Overview and Demo)
What is Distributed Cloud Bot Defense? Distributed Cloud Bot Defense protects your web properties from automated attacks by identifying and mitigating malicious bots. Bot Defense uses JavaScript and API calls to collect telemetry and mitigate malicious users within the context of the Distributed Cloud global network. Bot Defense can easily be integrated into existing applications in a number of ways. For applications already routing traffic through Distributed Cloud Mesh Service, Bot Defense is natively integrated into your Distributed Cloud Mesh HTTP load balancers. This integration allows you to configure the Bot Defense service through the HTTP load balancer's configuration in the Distributed Cloud Console. For other applications, connectors are available for several common insertion points that likely already exist in modern application architectures. Once Bot Defense is enabled and configured, you can view and filter traffic and transaction statistics on the Bot Defense dashboard in Distributed Cloud Console to see which users are malicious and how they’re being mitigated. F5 Distributed Cloud Bot Defense is an advanced add-on security feature included in the first launch of the F5 Web Application and API Protection (WAAP) service with seamless integration to protectyour web apps and APIs from a wide variety of attacks in real-time. High Level Distributed Cloud Security Architecture Bot Defense Demo: In this technical demonstration video we will walk through F5 Distributed Cloud Bot Defense, showing you how quick and easy it is to configure, the insights and visibility you have while demonstrating a couple of real attacks with Selenium and Python browser automation. "Nature is a mutable cloud, which is always and never the same." - Ralph Waldo Emerson We might not wax that philosophically around here, but our heads are in the cloud nonetheless! Join the F5 Distributed Cloud user group today and learn more with your peers and other F5 experts. Hope you enjoyed this Distributed Cloud Bot Defense Overview and Demo. If there are any comments or questions please feel free to reach us in the comments section. Thanks! Related Resources: Deploy Bot Defense on any Edge with F5 Distributed Cloud (SaaS Console, Automation) Protecting Your Web Applications Against Critical OWASP Automated Threats Making Mobile SDK Integration Ridiculously Easy with F5 XC Mobile SDK Integrator JavaScript Supply Chains, Magecart, and F5 XC Client-Side Defense (Demo) Bots, Fraud, and the OWASP Automated Threats Project (Overview) Protecting Your Native Mobile Apps with F5 XC Mobile App Shield Enabling F5 Distributed Cloud Client-Side Defense in BIG-IP 17.1 Bot Defense for Mobile Apps in XC WAAP Part 1: The Bot Defense Mobile SDK F5 Distributed Cloud WAAP Distributed Cloud Services Overview Enable and Configure Bot Defense - F5 Distributed Cloud ServiceUse F5 Distributed Cloud to service chain WAAP and CDN
The Content Delivery Network (CDN) market has become increasingly commoditized. Many providers have augmented their CDN capabilities with WAFs/WAAPs, DNS, load balancing, edge compute, and networking. Managing all these solutions together creates a web of operational complexity, which can be confusing. F5’s synergistic bundling of CDN with Web Application and API Protection (WAAP) benefits those looking for simplicity and ease of use. It provides a way around the complications and silos that many resource-strapped organizations face with their IT systems. This bundling also signifies how CDN has become a commodity product often not purchased independently anymore. This trend is encouraging many competitors to evolve their capabilities to include edge computing – a space where F5 has gained considerable experience in recent years. F5 is rapidly catching up to other providers’ CDNs. F5’s experience and leadership building the world’s best-of-breed Application Delivery Controller (ADC), the BIG-IP load balancer, put it in a unique position to offer the best application delivery and security services directly at the edge with many of its CDN points of presence. With robust regional edge capabilities and a global network, F5 has entered the CDN space with a complementary offering to an already compelling suite of features. This includes the ability to run microservices and Kubernetes workloads anywhere, with a complete range of services to support app infrastructure deployment, scale, and lifecycle management all within a single management console. With advancements made in the application security space at F5, WAAP capabilities are directly integrated into the Distributed Cloud Platform to protect both web apps and APIs. Features include (yet not limited to): Web Application Firewall: Signature + Behavioral WAF functionality Bot Defense: Detect client signals, determining if clients are human or automated DDoS Mitigation: Fully managed by F5 API Security: Continuous inspection and detection of shadow APIs Solution Combining the Distributed Cloud WAAP with CDN as a form of service chaining is a straightforward process. This not only gives you the best security protection for web apps and APIs, but also positions apps regionally to deliver them with low latency and minimal compute per request. In the following solution, we’ve combined Distributed Cloud WAAP and CDN to globally deliver an app protected by a WAF policy from the closest regional point of presence to the user. Follow along as I demonstrate how to configure the basic elements. Configuration Log in to the Distributed Cloud Console and navigate to the DNS Management service. Decide if you want Distributed Cloud to manage the DNS zone as a Primary DNS server or if you’d rather delegate the fully qualified domain name (FQDN) for your app to Distributed Cloud with a CNAME. While using Delegation or Managed DNS is optional, doing so makes it possible for Distributed Cloud to automatically create and manage the SSL certificates needed to securely publish your app. Next, in Distributed Cloud Console, navigate to the Web App and API Protection service, then go to App Firewall, then Add App Firewall. This is where you’ll create the security policy that we’ll later connect our HTTP LB. Let’s use the following basic WAF policy in YAML format, you can paste it directly in to the Console by changing the configuration view to JSON and then changing the format to YAML. Note: This uses the namespace “waap-cdn”, change this to match your individual tenant’s configuration. metadata: name: buytime-waf namespace: waap-cdn labels: {} annotations: {} disable: false spec: blocking: {} detection_settings: signature_selection_setting: default_attack_type_settings: {} high_medium_low_accuracy_signatures: {} enable_suppression: {} enable_threat_campaigns: {} default_violation_settings: {} bot_protection_setting: malicious_bot_action: BLOCK suspicious_bot_action: REPORT good_bot_action: REPORT allow_all_response_codes: {} default_anonymization: {} use_default_blocking_page: {} With the WAF policy saved, it’s time to configure the origin server. Navigate to Load Balancers > Origin Pools, then Add Origin Pool. The following YAML uses a FQDN DNS name reach the app server. Using an IP address for the server is possible as well. metadata: name: buytime-pool namespace: waap-cdn labels: {} annotations: {} disable: false spec: origin_servers: - public_name: dns_name: webserver.f5-cloud-demo.com labels: {} no_tls: {} port: 80 same_as_endpoint_port: {} healthcheck: [] loadbalancer_algorithm: LB_OVERRIDE endpoint_selection: LOCAL_PREFERRED With the supporting WAF and Origin Pool resources configured, it’s time to create the HTTP Load Balancer. Navigate to Load Balancers > HTTP Load Balancers, then create a new one. Use the following YAML to create the LB and use both resources created above. metadata: name: buytime-online namespace: waap-cdn labels: {} annotations: {} disable: false spec: domains: - buytime.waap.f5-cloud-demo.com https_auto_cert: http_redirect: true add_hsts: true port: 443 tls_config: default_security: {} no_mtls: {} default_header: {} enable_path_normalize: {} non_default_loadbalancer: {} header_transformation_type: default_header_transformation: {} advertise_on_public_default_vip: {} default_route_pools: - pool: tenant: your-tenant-uid namespace: waap-cdn name: buytime-pool kind: origin_pool weight: 1 priority: 1 endpoint_subsets: {} routes: [] app_firewall: tenant: your-tenant-uid namespace: waap-cdn name: buytime-waf kind: app_firewall add_location: true no_challenge: {} user_id_client_ip: {} disable_rate_limit: {} waf_exclusion_rules: [] data_guard_rules: [] blocked_clients: [] trusted_clients: [] ddos_mitigation_rules: [] service_policies_from_namespace: {} round_robin: {} disable_trust_client_ip_headers: {} disable_ddos_detection: {} disable_malicious_user_detection: {} disable_api_discovery: {} disable_bot_defense: {} disable_api_definition: {} disable_ip_reputation: {} disable_client_side_defense: {} resource_version: "517528014" With the HTTP LB successfully deployed, check that its status is ready on the status page. You can verify the LB is working by sending a basic request using the command line tool, curl. Confirm that the value of the HTTP header “Server” is “volt-adc”. da.potter@lab ~ % curl -I https://buytime.waap.f5-cloud-demo.com HTTP/2 200 date: Mon, 17 Oct 2022 23:23:55 GMT content-type: text/html; charset=UTF-8 content-length: 2200 vary: Origin access-control-allow-credentials: true accept-ranges: bytes cache-control: public, max-age=0 last-modified: Wed, 24 Feb 2021 11:06:36 GMT etag: W/"898-177d3b82260" x-envoy-upstream-service-time: 136 strict-transport-security: max-age=31536000 set-cookie: 1f945=1666049035840-557942247; Path=/; Domain=f5-cloud-demo.com; Expires=Sun, 17 Oct 2032 23:23:55 GMT set-cookie: 1f9403=viJrSNaAp766P6p6EKZK7nyhofjXCVawnskkzsrMBUZIoNQOEUqXFkyymBAGlYPNQXOUBOOYKFfs0ne+fKAT/ozN5PM4S5hmAIiHQ7JAh48P4AP47wwPqdvC22MSsSejQ0upD9oEhkQEeTG1Iro1N9sLh+w+CtFS7WiXmmJFV9FAl3E2; path=/ x-volterra-location: wes-sea server: volt-adc Now it’s time to configure the CDN Distribution and service chain it to the WAAP HTTP LB. Navigate to Content Delivery Network > Distributions, then Add Distribution. The following YAML creates a basic CDN configuration that uses the WAAP HTTP LB above. metadata: name: buytime-cdn namespace: waap-cdn labels: {} annotations: {} disable: false spec: domains: - buytime.f5-cloud-demo.com https_auto_cert: http_redirect: true add_hsts: true tls_config: tls_12_plus: {} add_location: false more_option: cache_ttl_options: cache_ttl_override: 1m origin_pool: public_name: dns_name: buytime.waap.f5-cloud-demo.com use_tls: use_host_header_as_sni: {} tls_config: default_security: {} volterra_trusted_ca: {} no_mtls: {} origin_servers: - public_name: dns_name: buytime.waap.f5-cloud-demo.com follow_origin_redirect: false resource_version: "518473853" After saving the configuration, verify that the status is “Active”. You can confirm the CDN deployment status for each individual region by going to the distribution’s action button “Show Global Status”, and scrolling down to each region to see that each region’s “site_status.status” value is “DEPLOYMENT_STATUS_DEPLOYED”. Verification With the CDN Distribution successfully deployed, it’s possible to confirm with the following basic request using curl. Take note of the two HTTP headers “Server” and “x-cache-status”. The Server value will now be “volt-cdn”, and the x-cache-status will be “MISS” for the first request. da.potter@lab ~ % curl -I https://buytime.f5-cloud-demo.com HTTP/2 200 date: Mon, 17 Oct 2022 23:24:04 GMT content-type: text/html; charset=UTF-8 content-length: 2200 vary: Origin access-control-allow-credentials: true accept-ranges: bytes cache-control: public, max-age=0 last-modified: Wed, 24 Feb 2021 11:06:36 GMT etag: W/"898-177d3b82260" x-envoy-upstream-service-time: 63 strict-transport-security: max-age=31536000 set-cookie: 1f945=1666049044863-471593352; Path=/; Domain=f5-cloud-demo.com; Expires=Sun, 17 Oct 2032 23:24:04 GMT set-cookie: 1f9403=aCNN1JINHqvWPwkVT5OH3c+OIl6+Ve9Xkjx/zfWxz5AaG24IkeYqZ+y6tQqE9CiFkNk+cnU7NP0EYtgGnxV0dLzuo3yHRi3dzVLT7PEUHpYA2YSXbHY6yTijHbj/rSafchaEEnzegqngS4dBwfe56pBZt52MMWsUU9x3P4yMzeeonxcr; path=/ x-volterra-location: dal3-dal server: volt-cdn x-cache-status: MISS strict-transport-security: max-age=31536000 To see a security violation detected by the WAF in real-time, you can simulate a simple XSS exploit with the following curl: da.potter@lab ~ % curl -Gv "https://buytime.f5-cloud-demo.com?<script>('alert:XSS')</script>" * Trying x.x.x.x:443... * Connected to buytime.f5-cloud-demo.com (x.x.x.x) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem * CApath: none * (304) (OUT), TLS handshake, Client hello (1): * (304) (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-ECDSA-AES256-GCM-SHA384 * ALPN, server accepted to use h2 * Server certificate: * subject: CN=buytime.f5-cloud-demo.com * start date: Oct 14 23:51:02 2022 GMT * expire date: Jan 12 23:51:01 2023 GMT * subjectAltName: host "buytime.f5-cloud-demo.com" matched cert's "buytime.f5-cloud-demo.com" * issuer: C=US; O=Let's Encrypt; CN=R3 * SSL certificate verify ok. * Using HTTP2, server supports multiplexing * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x14f010000) > GET /?<script>('alert:XSS')</script> HTTP/2 > Host: buytime.f5-cloud-demo.com > user-agent: curl/7.79.1 > accept: */* > * Connection state changed (MAX_CONCURRENT_STREAMS == 128)! < HTTP/2 200 < date: Sat, 22 Oct 2022 01:04:39 GMT < content-type: text/html; charset=UTF-8 < content-length: 269 < cache-control: no-cache < pragma: no-cache < set-cookie: 1f945=1666400679155-452898837; Path=/; Domain=f5-cloud-demo.com; Expires=Fri, 22 Oct 2032 01:04:39 GMT < set-cookie: 1f9403=/1b+W13c7xNShbbe6zE3KKUDNPCGbxRMVhI64uZny+HFXxpkJMsCKmDWaihBD4KWm82reTlVsS8MumTYQW6ktFQqXeFvrMDFMSKdNSAbVT+IqQfSuVfVRfrtgRkvgzbDEX9TUIhp3xJV3R1jdbUuAAaj9Dhgdsven8FlCaADENYuIlBE; path=/ < x-volterra-location: dal3-dal < server: volt-cdn < x-cache-status: MISS < strict-transport-security: max-age=31536000 < <html><head><title>Request Rejected</title></head> <body>The requested URL was rejected. Please consult with your administrator.<br/><br/> Your support ID is 85281693-eb72-4891-9099-928ffe00869c<br/><br/><a href='javascript:history.back();'>[Go Back]</a></body></html> * Connection #0 to host buytime.f5-cloud-demo.com left intact Notice that the above request intentionally by-passes the CDN cache and is sent to the HTTP LB for the WAF policy to inspect. With this request rejected, you can confirm the attack by navigating to the WAAP HTTP LB Security page under the WAAP Security section within Apps & APIs. After refreshing the page, you’ll see the security violation under the “Top Attacked” panel. Demo To see all of this in action, watch my video below. This uses all of the configuration details above to make a WAAP + CDN service chain in Distributed Cloud. Additional Guides Virtually deploy this solution in our product simulator, or hands-on with step-by-step comprehensive demo guide. The demo guide includes all the steps, including those that are needed prior to deployment, so that once deployed, the solution works end-to-end without any tweaks to local DNS. The demo guide steps can also be automated with Ansible, in case you'd either like to replicate it or simply want to jump to the end and work your way back. Conclusion This shows just how simple it can be to use the Distributed Cloud CDN to frontend your web app protected by a WAF, all natively within the F5 Distributed Cloud’s regional edge POPs. The advantage of this solution should now be clear – the Distributed Cloud CDN is cloud-agnostic, flexible, agile, and you can enforce security policies anywhere, regardless of whether your web app lives on-prem, in and across clouds, or even at the edge. For more information about Distributed Cloud WAAP and Distributed Cloud CDN, visit the following resources: Product website: https://www.f5.com/cloud/products/cdn Distributed Cloud CDN & WAAP Demo Guide: https://github.com/f5devcentral/xcwaapcdnguide Video: https://youtu.be/OUD8R6j5Q8o Simulator: https://simulator.f5.com/s/waap-cdn Demo Guide: https://github.com/f5devcentral/xcwaapcdnguideF5 BIG-IP Access Policy Manager (APM) - Google Authenticator and Microsoft Authenticator
Introduction Lab guide Phase 1: Token Generation Phase 2: Token verification Related Content Introduction In our walkthrough we are refreshing an existing time-based one-time password (TOTP) deployment Two-Factor Authentication With Google Authenticator And APM In our walkthrough we are following the below assumptions, Secret key is generated outside of F5 and saved to Active Directory (AD) user attribute. F5 APM should be able to query AD user attribute (for example, in our case it's called serialNumber). We have two separate portals, One portal for Token generation and QR scanning. One portal for Application access. Lab guide Phase 1: Token Generation User logs in to the token generation portal and authenticates with AD credentials. F5 APM authenticates the user with AD and query the attribute for the secret key. F5 APM presnets the QR code for the user to be scanned whether by Google or Microsoft Authenticators. Phase 2: Token verification User Access the application and authenticates with AD credentials. Once user is successfully authenticated, the user is prompted to provide the one-time password (OTP). Once F5 verifies the provided OTP, the user is allowed to access the application. Verification iRule when ACCESS_POLICY_AGENT_EVENT { if { [ACCESS::policy agent_id] eq "ga_code_verify" } { ### Google Authenticator verification settings ### # lock the user out after x attempts for a period of x seconds set static::lockout_attempts 3 set static::lockout_period 30 # logon page session variable name for code attempt form field set static::ga_code_form_field "ga_code_attempt" # key (shared secret) storage method: ldap, ad, or datagroup set static::ga_key_storage "ad" # LDAP attribute for key if storing in LDAP (optional) set static::ga_key_ldap_attr "google_auth_key" # Active Directory attribute for key if storing in AD (optional) set static::ga_key_ad_attr [ACCESS::session data get "session.ad.last.attr.serialNumber"] # datagroup name if storing key in a datagroup (optional) set static::ga_key_dg "google_auth_keys" ##################################### ### DO NOT MODIFY BELOW THIS LINE ### ##################################### # set lockout table set static::lockout_state_table "[virtual name]_lockout_status" # set variables from APM logon page set username [ACCESS::session data get session.logon.last.username] set ga_code_attempt [ACCESS::session data get session.logon.last.$static::ga_code_form_field] # retrieve key from specified storage set ga_key "" switch $static::ga_key_storage { ldap { set ga_key [ACCESS::session data get session.ldap.last.attr.$static::ga_key_ldap_attr] } ad { set ga_key [ACCESS::session data get "session.ad.last.attr.serialNumber"] } datagroup { set ga_key [class lookup $username $static::ga_key_dg] } } # increment the number of login attempts for the user set prev_attempts [table incr -notouch -subtable $static::lockout_state_table $username] table timeout -subtable $static::lockout_state_table $username $static::lockout_period # verification result value: # 0 = successful # 1 = failed # 2 = no key found # 3 = invalid key length # 4 = user locked out # make sure that the user isn't locked out before calculating GA code if { $prev_attempts <= $static::lockout_attempts } { # check that a valid key was retrieved, then proceed #Update the key length based on the organization requirements if { [string length $ga_key] == 16 } { # begin - Base32 decode to binary # Base32 alphabet (see RFC 4648) array set static::b32_alphabet { A 0 B 1 C 2 D 3 E 4 F 5 G 6 H 7 I 8 J 9 K 10 L 11 M 12 N 13 O 14 P 15 Q 16 R 17 S 18 T 19 U 20 V 21 W 22 X 23 Y 24 Z 25 2 26 3 27 4 28 5 29 6 30 7 31 } set ga_key [string toupper $ga_key] set l [string length $ga_key] set n 0 set j 0 set ga_key_bin "" for { set i 0 } { $i < $l } { incr i } { set n [expr $n << 5] set n [expr $n + $static::b32_alphabet([string index $ga_key $i])] set j [incr j 5] if { $j >= 8 } { set j [incr j -8] append ga_key_bin [format %c [expr ($n & (0xFF << $j)) >> $j]] } } # end - Base32 decode to binary # begin - HMAC-SHA1 calculation of Google Auth token set time [binary format W* [expr [clock seconds] / 30]] set ipad "" set opad "" for { set j 0 } { $j < [string length $ga_key_bin] } { incr j } { binary scan $ga_key_bin @${j}H2 k set o [expr 0x$k ^ 0x5C] set i [expr 0x$k ^ 0x36] append ipad [format %c $i] append opad [format %c $o] } while { $j < 64 } { append ipad 6 append opad \\ incr j } binary scan [sha1 $opad[sha1 ${ipad}${time}]] H* token # end - HMAC-SHA1 calculation of Google Auth hex token # begin - extract code from Google Auth hex token set offset [expr ([scan [string index $token end] %x] & 0x0F) << 1] set ga_code [expr (0x[string range $token $offset [expr $offset + 7]] & 0x7FFFFFFF) % 1000000] set ga_code [format %06d $ga_code] # end - extract code from Google Auth hex token if { $ga_code_attempt eq $ga_code } { # code verification successful set ga_result 0 } else { # code verification failed set ga_result 1 } } elseif { [string length $ga_key] > 0 } { # invalid key length, greater than 0, but not length not equal to 16 chars set ga_result 3 } else { # could not retrieve user's key set ga_result 2 } } else { # user locked out due to too many failed attempts set ga_result 4 } # set code verification result in session variable ACCESS::session data set session.custom.ga_result $ga_result } } Related Content Two-Factor Authentication With Google Authenticator And APM Demystifying Time-based OTP Google Authenticator Token Verification iRule For APM APM Google Authenticator HTTP API Google Authenticator Verification iRule (TMOS v11.1+ optimized) UDF Lab Configuring MFA OTP Note, Scan the Article photo for more BIG-IP Access Policy Manager (APM) info.Introduction to OWASP API Security Top 10 2023
Introduction to API An Application Programming Interface (API) is a component that enables communication between two different systems by following certain rules. It also adds a layer of abstraction between the two systems where the requester does not know how the other system has derived the result and responded back. Over the past few years, developers have started relying more on APIs as it helps them to meet the needs of today’s rapid application deployment model. As the APIs started getting a wider acceptance it is highly critical to safeguard them by thoroughly testing their behavior and following best securitypractices. Learn API Security Best Practices. Overview of OWASP API Security The OWASP API Security project aims to help the organizations by providing a guide with a list of the latest top 10 most critical API vulnerabilities and steps to mitigate them. As part of updating the old OWASP API Security risk categories of 2019, recently OWASP API Security Top 10 2023 is released. What’s new in OWASP API Sec 2023? List of vulnerabilities: API1:2023 Broken Object Level Authorization Broken Object Level Authorization (BOLA) is a vulnerability that occurs when there is a failure in validation of user’s permissions to perform a specific task over an object which may eventually lead to leakage, updation or destruction of data. To prevent this vulnerability,proper authorization mechanism should be followed, proper checks should be made to validate user’s action on a certain record and security tests should be performedbefore deploying any production grade changes. API2:2023 Broken Authentication Broken Authentication is a critical vulnerability that occurs when application’s authentication endpoints fail to detect attackers impersonating someone else’s identity and allow partial or full control over the account. To prevent this vulnerability,observability and understanding of all possible authentication API endpoints is needed, re-authentication should be performed for any confidential changes, multi-factor authentication, captcha-challenge and effective security solutions should be appliedto detect &mitigate credential stuffing, dictionary and brute force type of attacks. API3:2023 Broken Object Property Level Authorization Broken Object Property Level Authorization is one of the new risk categories of OWASP API Security Top 10 2023 RC. This vulnerability occurs when a user is allowed to access an object’s property without validating his access permissions. Excessive Data Exposure and Mass Assignment which were a part of OWASP APISec 2019 are now part of this new vulnerability. To prevent this vulnerability, access privileges of users requesting for a specific object's propertyshould be scrutinized before exposureby the API endpoints. Use of generic methods &automatically binding client inputs to internal objects or code variables should be avoided and schema-based validation should be enforced. API4:2023 Unrestricted Resource Consumption Unrestricted Resource Consumption vulnerability occurs when the system’s resources are being unnecessarily consumed which could eventually lead to degradation of services and performance latency issues.Although the name has changed,the vulnerability is still the same asthat of Lack of Resources & Rate Limiting. To prevent this vulnerability, rate-limiting, maximum size forinput payload/parameters and server-side validations of requests should be enforced. API5:2023 Broken Function Level Authorization Broken Function Level Authorization occurs when vulnerable API endpoints allow normal users to perform administrative actions or user from one group is allowed to access a function specific to users of another group. To prevent this vulnerability, access control policies and administrative authorization checks based on user’s group/roles should be implemented. API6:2023Unrestricted Access to Sensitive Business Flows Unrestricted Access to Sensitive Business Flows is also a new addition to the list of API vulnerabilities. While writing API endpoints it is extremely critical for the developers to have a clear understanding of the business flows getting exposed by it. To avoid exposing any sensitive business flow and limit its excessive usage which if not considered, might eventually lead to exploitation by the attackers and cause some serious harm to the business. This also includes securing and limiting access to B2B APIs that are consumed directly and often integrated with minimal protection mechanism. By keeping automation to work, now-a-days attackers can bypass traditional protection mechanisms. APIs inefficiency in detecting automated bot attacks not only causes business loss but also it can adversely impact the services for real users as well. To overcome this vulnerability, enterprises need to have a platform to identify whether the request is from a real user or an automated tool by analyzing and tracking patterns of usage. Device fingerprinting, Integrating Captcha solution, blocking Tor requests, are a few methods which can help to minimize the impact of such automated attacks. For more details on automated threats, you can visit OWASP Automated Threats to Web Applications Note: Although the vulnerability is new but it contains some references ofAPI10:2019 Insufficient Logging & Monitoring API7:2023 Server-Side Request Forgery After finding a place in OWASP Top 10 web application vulnerabilities of 2021, SSRF has now been included in OWASP API Security Top 10 2023 RC list as well, showing the severity of this vulnerability. Server-Side Request Forgery (SSRF) vulnerability occurs when an API fetches an internal server resource without validating the URL from the user. Attackers exploit this vulnerability by manipulating the URL, which in turn helps them to retrieve sensitive data from the internal servers. To overcome this vulnerability, Input data validations should be implemented to ensure that the client supplied input dataobeys the expected format. Allow lists should be maintained so thatonly trusted requests/calls will be processed, andHTTP redirections should be disabled. API8:2023 Security Misconfiguration Security Misconfiguration is a vulnerability that may arise when security best practices are overlooked. Unwanted exposure of debug logs, unnecessary enabled HTTP Verbs, unapplied latest security patches, missing repeatable security hardening process, improper implementation of CORS policy etc. are a few examples of security misconfiguration. To prevent this vulnerability, systems and entire API stack should be maintained up to date without missing any security patches. Continuous security hardening and configurations tracking process should be carried out. Make sure all API communications take place over a secure channel (TLS) and all servers in HTTP server chain process incoming requests. Cross-Origin Resource Sharing (CORS) policy should be set up properly. Unnecessary HTTP verbs should be disabled. API9:2023 Improper Inventory Management Improper Inventory Management vulnerability occurs when organizations don’t have much clarity on their own APIs as well as third-party APIs that they use and lack proper documentation. Unawareness with regards to current API version, environment, access control policies, data shared with the third-party etc. can lead to serious business repercussions. Clear understanding and proper documentation arethe keyto overcome this vulnerability.All the details related to API hosts, API environment, Network access, API version, Integrated services, redirections, rate limiting, CORS policy should be documented correctly and maintained up to date.Documenting every minor detail is advisable and authorized access should be given to these documents. Exposed API versions should be secured along with the production version. A risk analysis is recommended whenever newer versions of APIs are available. API10:2023 Unsafe Consumption of APIs Unsafe Consumption of APIs is again a newly added vulnerability covering a portion of API8:2019 Injectionvulnerability. This occurs when developers tend to apply very little or no sanitization on the data received from third-party APIs. To overcome this, we should make sure that API interactions take place over an encrypted channel. API data evaluation and sanitization should be carried out before using the data further. Precautionary actions should be taken to avoid unnecessary redirections by using Allow lists. How F5 XC can help? F5 Distributed Cloud (F5 XC) has a wide range of solutions for deploying, managing and securing application deployments in different environments. XC WAAP is a F5 SaaS offering. The 4 key components of WAAP are Web Application Firewall, API Security, Bot Defense, DDoS Mitigation. All these solutions are powered on top of the XC platform. In addition to WAAP, F5 XC has other solutions to offer such as Fraud and Abuse, AIP, CDN, MCN, DNS and so on. API security in XC WAAP simplifies operations with automated discovery of API transactions using AI/ML Engine along with insights of performance. It also provides API protection features like Rate Limiting, PII safeguard along with comprehensive security monitoring GUI dashboard. API security provides feasibility to import the inventory file in the form of swagger which helps to know exactly what endpoints, methods and payloads are valid, and this tightens security against abuse. F5 XC management console helps the customers to leverage the benefit of monitoring, managing, and maintaining their application’s traffic from a single place irrespective of its platform on which it is hosted, it could be multi-cloud, on prem or edge. Note: This is an initialarticle covering the overview of proposed most critical API vulnerabilities from OWASP API Security community for 2023. More articles covering detailed insight of each vulnerability and their mitigation steps using F5 XC platform will follow this article in coming days. Meanwhile, you can refer to overview article for OWASP API Security Top 10 2019 which contains link to detailed articles covering API vulnerabilities of 2019 and how F5 XC can help to mitigate them. Related OWASP API Security article series: Broken Authentication Excessive Data Exposure Mass Assignment Lack of Resources & Rate limiting Security Misconfiguration Improper Assets Management Unsafe consumption of APIs Server-Side Request Forgery Unrestricted Access to Sensitive Business Flows OWASP API Security Top 10 - 2019Introduction to OWASP Top 10 API Security Risks - 2019 and F5 Distributed Cloud WAAP
Introduction to API: An application programming interface (API) is a combination of protocols, functions, etc. which we can utilize to get details about resources, services and features.APIs are fast, lightweight and reliablebut they expose sensitive data and so they have become the targets of hackers. Overview of OWASP API Security: The simplicity of APIs has given hackers a chance to infiltrate them in plethora of ways tosteal personal and sensitive details. Increase in demand of API security caused a need for a project to keep track of latest API vulnerabilities andsecurity procedures calledOWASP API Security Top 10. As per the above project below are the top ten issues and their overview in API security as of 2019. API1:2019 Broken Object Level Authorization APIs expose endpoints that manage objects using unique identifiers, providing hackers a chanceto bypass access controls. To prevent this attacks authorized checks like credentials and APItokenshould always be kept in place in the code ifthere is a request using a user input. API2:2019 Broken User Authentication Authentication mechanisms are sometimes implemented with less security, allowing attackers to compromise authentication tokensto take over other user's identities. API3:2019 Excessive Data Exposure In most of the recent attacks it was observed developers are exposing unnecessary and sensitive object properties providing illegal users a way to exploit them.For more information about this vulnerability, demonstration scenario and prevention steps using F5 XC refer to thearticle. API4:2019 Lack of Resources & Rate Limiting APIs do not have any restrictions on the size or number of resources that can be requested by the end user. Above mentioned scenarios sometimes lead to poor API server performance, Denial of Service (DoS) and brute force attacks.For more information about this vulnerability, demonstration scenario and prevention steps using F5 XC refer to thearticle. API5:2019 Broken Function Level Authorization Most applications are composed of different groups, users and roles. If configurations like access control are not applied, it will lead to authorization flaws allowing one user to access theresources of other users. API6:2019 Mass Assignment Code sanity should always be performed in response data, binding client data into code variables without filtering gives hackers a chance to guess object's properties by exploring the API endpoints, documentations, etc. API7:2019 Security Misconfiguration This attack is mostly caused because of misconfigured HTTP headers, unnecessary HTTP methods, permissive Cross-Origin resource sharing (CORS), and verbose error messages in logs containing sensitive information like usernames, PIN, IP addresses, etc.For more information about this vulnerability, demonstration scenario and prevention steps using F5 XC refer to thearticle. API8:2019 Injection OS commands, SQL, Command Injection, etc., occur if there are no restrictions on user requested schema as part of filter query. The malicious request can sometimes bypass these validations to execute unintended commands providing attackers access to sensitive information.For more information about this vulnerability, demonstration scenario and prevention steps using F5 XC refer to thearticle. API9:2019 Improper Assets Management A modern web application typically hosts thousands of requests. It is critical to update the documentation/swagger as per the latest changes and include information about newly implemented APIs. If they are not regularly updated hackers can explore and find any deprecated API which may sometimes expose debug endpoints.For more information about this vulnerability, demonstration scenario and prevention steps using F5 XC refer to thearticle. API10:2019 Insufficient Logging & Monitoring Any issues in logging and monitoring services will give attackers more ways to attack systems without being recognized. It’s always advised to configure the best monitoring solutions to keep track of all logs and to configure email alerts. Sometimes it’s the best practice to keep logging details in a different location to avoid malicious user activity erasing their log trails.For more information refer to the article. Overview of F5 Distributed Cloud WAAP: Web Application and API protection (WAAP) is a SAAS offering provided by F5 Distributed Cloud Services to protect applications and published APIs using Web Application Firewall (WAF), bot protection, API security, and DDoS mitigation. Once WAAP policy is applied on the load balancer, these service engines protect web applications and API endpoints with the latest automatic detection of WAF, Bot and DOS attack signatures. One of the key sections of Distributed Cloud WAAP is API security which focuses primarily on securing the API’s using different configurations like OpenAPI ingestion, automatic API discovery, service policies, rate limiting, Allow/Denied URLs, etc. Below diagram shows how Distributed Cloud WAAP protects APIs: Whenever there is a request originating from end users Distributed Cloud WAAP analyses the request metadata details like URL, filter parameters, Headers, etc. to find whether it’s a legitimate request. Once the request is screened, validated and approved then only the request is forwarded to the back-end servers. Back-end servers then return the requested details to the end user. If for any reason Distributed Cloud WAAP finds the request has discrepancies or is not valid the request will be blocked, and a security event will be generated in dashboard. Users or administrators can analyze the captured request details and can modify the existing Distributed Cloud WAAP configurations if needed to reach the business goals. Articles on OWASP API Security: Excessive Data Exposure Lack of Resources & Rate Limiting Security Misconfiguration Injection Improper Assets Management Insufficient Logging & Monitoring Note: Articles on remaining OWASP API Security Top 10 2019 vulnerabilities are in pipeline and will get published shortly, stay tuned for the update New edition of OWASP API Security Top 10 risks - 2023 is released and you can check this link for more details Related Links: F5 Distributed Cloud WAAP F5 Distributed Cloud Services
