Use F5 Distributed Cloud to service chain WAAP and CDN
The Content Delivery Network (CDN) market has become increasingly commoditized. Many providers have augmented their CDN capabilities with WAFs/WAAPs, DNS, load balancing, edge compute, and networking. Managing all these solutions together creates a web of operational complexity, which can be confusing. F5’s synergistic bundling of CDN with Web Application and API Protection (WAAP) benefits those looking for simplicity and ease of use. It provides a way around the complications and silos that many resource-strapped organizations face with their IT systems. This bundling also signifies how CDN has become a commodity product often not purchased independently anymore. This trend is encouraging many competitors to evolve their capabilities to include edge computing – a space where F5 has gained considerable experience in recent years. F5 is rapidly catching up to other providers’ CDNs. F5’s experience and leadership building the world’s best-of-breed Application Delivery Controller (ADC), the BIG-IP load balancer, put it in a unique position to offer the best application delivery and security services directly at the edge with many of its CDN points of presence. With robust regional edge capabilities and a global network, F5 has entered the CDN space with a complementary offering to an already compelling suite of features. This includes the ability to run microservices and Kubernetes workloads anywhere, with a complete range of services to support app infrastructure deployment, scale, and lifecycle management all within a single management console. With advancements made in the application security space at F5, WAAP capabilities are directly integrated into the Distributed Cloud Platform to protect both web apps and APIs. Features include (yet not limited to): Web Application Firewall: Signature + Behavioral WAF functionality Bot Defense: Detect client signals, determining if clients are human or automated DDoS Mitigation: Fully managed by F5 API Security: Continuous inspection and detection of shadow APIs Solution Combining the Distributed Cloud WAAP with CDN as a form of service chaining is a straightforward process. This not only gives you the best security protection for web apps and APIs, but also positions apps regionally to deliver them with low latency and minimal compute per request. In the following solution, we’ve combined Distributed Cloud WAAP and CDN to globally deliver an app protected by a WAF policy from the closest regional point of presence to the user. Follow along as I demonstrate how to configure the basic elements. Configuration Log in to the Distributed Cloud Console and navigate to the DNS Management service. Decide if you want Distributed Cloud to manage the DNS zone as a Primary DNS server or if you’d rather delegate the fully qualified domain name (FQDN) for your app to Distributed Cloud with a CNAME. While using Delegation or Managed DNS is optional, doing so makes it possible for Distributed Cloud to automatically create and manage the SSL certificates needed to securely publish your app. Next, in Distributed Cloud Console, navigate to the Web App and API Protection service, then go to App Firewall, then Add App Firewall. This is where you’ll create the security policy that we’ll later connect our HTTP LB. Let’s use the following basic WAF policy in YAML format, you can paste it directly in to the Console by changing the configuration view to JSON and then changing the format to YAML. Note: This uses the namespace “waap-cdn”, change this to match your individual tenant’s configuration. metadata: name: buytime-waf namespace: waap-cdn labels: {} annotations: {} disable: false spec: blocking: {} detection_settings: signature_selection_setting: default_attack_type_settings: {} high_medium_low_accuracy_signatures: {} enable_suppression: {} enable_threat_campaigns: {} default_violation_settings: {} bot_protection_setting: malicious_bot_action: BLOCK suspicious_bot_action: REPORT good_bot_action: REPORT allow_all_response_codes: {} default_anonymization: {} use_default_blocking_page: {} With the WAF policy saved, it’s time to configure the origin server. Navigate to Load Balancers > Origin Pools, then Add Origin Pool. The following YAML uses a FQDN DNS name reach the app server. Using an IP address for the server is possible as well. metadata: name: buytime-pool namespace: waap-cdn labels: {} annotations: {} disable: false spec: origin_servers: - public_name: dns_name: webserver.f5-cloud-demo.com labels: {} no_tls: {} port: 80 same_as_endpoint_port: {} healthcheck: [] loadbalancer_algorithm: LB_OVERRIDE endpoint_selection: LOCAL_PREFERRED With the supporting WAF and Origin Pool resources configured, it’s time to create the HTTP Load Balancer. Navigate to Load Balancers > HTTP Load Balancers, then create a new one. Use the following YAML to create the LB and use both resources created above. metadata: name: buytime-online namespace: waap-cdn labels: {} annotations: {} disable: false spec: domains: - buytime.waap.f5-cloud-demo.com https_auto_cert: http_redirect: true add_hsts: true port: 443 tls_config: default_security: {} no_mtls: {} default_header: {} enable_path_normalize: {} non_default_loadbalancer: {} header_transformation_type: default_header_transformation: {} advertise_on_public_default_vip: {} default_route_pools: - pool: tenant: your-tenant-uid namespace: waap-cdn name: buytime-pool kind: origin_pool weight: 1 priority: 1 endpoint_subsets: {} routes: [] app_firewall: tenant: your-tenant-uid namespace: waap-cdn name: buytime-waf kind: app_firewall add_location: true no_challenge: {} user_id_client_ip: {} disable_rate_limit: {} waf_exclusion_rules: [] data_guard_rules: [] blocked_clients: [] trusted_clients: [] ddos_mitigation_rules: [] service_policies_from_namespace: {} round_robin: {} disable_trust_client_ip_headers: {} disable_ddos_detection: {} disable_malicious_user_detection: {} disable_api_discovery: {} disable_bot_defense: {} disable_api_definition: {} disable_ip_reputation: {} disable_client_side_defense: {} resource_version: "517528014" With the HTTP LB successfully deployed, check that its status is ready on the status page. You can verify the LB is working by sending a basic request using the command line tool, curl. Confirm that the value of the HTTP header “Server” is “volt-adc”. da.potter@lab ~ % curl -I https://buytime.waap.f5-cloud-demo.com HTTP/2 200 date: Mon, 17 Oct 2022 23:23:55 GMT content-type: text/html; charset=UTF-8 content-length: 2200 vary: Origin access-control-allow-credentials: true accept-ranges: bytes cache-control: public, max-age=0 last-modified: Wed, 24 Feb 2021 11:06:36 GMT etag: W/"898-177d3b82260" x-envoy-upstream-service-time: 136 strict-transport-security: max-age=31536000 set-cookie: 1f945=1666049035840-557942247; Path=/; Domain=f5-cloud-demo.com; Expires=Sun, 17 Oct 2032 23:23:55 GMT set-cookie: 1f9403=viJrSNaAp766P6p6EKZK7nyhofjXCVawnskkzsrMBUZIoNQOEUqXFkyymBAGlYPNQXOUBOOYKFfs0ne+fKAT/ozN5PM4S5hmAIiHQ7JAh48P4AP47wwPqdvC22MSsSejQ0upD9oEhkQEeTG1Iro1N9sLh+w+CtFS7WiXmmJFV9FAl3E2; path=/ x-volterra-location: wes-sea server: volt-adc Now it’s time to configure the CDN Distribution and service chain it to the WAAP HTTP LB. Navigate to Content Delivery Network > Distributions, then Add Distribution. The following YAML creates a basic CDN configuration that uses the WAAP HTTP LB above. metadata: name: buytime-cdn namespace: waap-cdn labels: {} annotations: {} disable: false spec: domains: - buytime.f5-cloud-demo.com https_auto_cert: http_redirect: true add_hsts: true tls_config: tls_12_plus: {} add_location: false more_option: cache_ttl_options: cache_ttl_override: 1m origin_pool: public_name: dns_name: buytime.waap.f5-cloud-demo.com use_tls: use_host_header_as_sni: {} tls_config: default_security: {} volterra_trusted_ca: {} no_mtls: {} origin_servers: - public_name: dns_name: buytime.waap.f5-cloud-demo.com follow_origin_redirect: false resource_version: "518473853" After saving the configuration, verify that the status is “Active”. You can confirm the CDN deployment status for each individual region by going to the distribution’s action button “Show Global Status”, and scrolling down to each region to see that each region’s “site_status.status” value is “DEPLOYMENT_STATUS_DEPLOYED”. Verification With the CDN Distribution successfully deployed, it’s possible to confirm with the following basic request using curl. Take note of the two HTTP headers “Server” and “x-cache-status”. The Server value will now be “volt-cdn”, and the x-cache-status will be “MISS” for the first request. da.potter@lab ~ % curl -I https://buytime.f5-cloud-demo.com HTTP/2 200 date: Mon, 17 Oct 2022 23:24:04 GMT content-type: text/html; charset=UTF-8 content-length: 2200 vary: Origin access-control-allow-credentials: true accept-ranges: bytes cache-control: public, max-age=0 last-modified: Wed, 24 Feb 2021 11:06:36 GMT etag: W/"898-177d3b82260" x-envoy-upstream-service-time: 63 strict-transport-security: max-age=31536000 set-cookie: 1f945=1666049044863-471593352; Path=/; Domain=f5-cloud-demo.com; Expires=Sun, 17 Oct 2032 23:24:04 GMT set-cookie: 1f9403=aCNN1JINHqvWPwkVT5OH3c+OIl6+Ve9Xkjx/zfWxz5AaG24IkeYqZ+y6tQqE9CiFkNk+cnU7NP0EYtgGnxV0dLzuo3yHRi3dzVLT7PEUHpYA2YSXbHY6yTijHbj/rSafchaEEnzegqngS4dBwfe56pBZt52MMWsUU9x3P4yMzeeonxcr; path=/ x-volterra-location: dal3-dal server: volt-cdn x-cache-status: MISS strict-transport-security: max-age=31536000 To see a security violation detected by the WAF in real-time, you can simulate a simple XSS exploit with the following curl: da.potter@lab ~ % curl -Gv "https://buytime.f5-cloud-demo.com?<script>('alert:XSS')</script>" * Trying x.x.x.x:443... * Connected to buytime.f5-cloud-demo.com (x.x.x.x) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem * CApath: none * (304) (OUT), TLS handshake, Client hello (1): * (304) (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-ECDSA-AES256-GCM-SHA384 * ALPN, server accepted to use h2 * Server certificate: * subject: CN=buytime.f5-cloud-demo.com * start date: Oct 14 23:51:02 2022 GMT * expire date: Jan 12 23:51:01 2023 GMT * subjectAltName: host "buytime.f5-cloud-demo.com" matched cert's "buytime.f5-cloud-demo.com" * issuer: C=US; O=Let's Encrypt; CN=R3 * SSL certificate verify ok. * Using HTTP2, server supports multiplexing * Connection state changed (HTTP/2 confirmed) * Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0 * Using Stream ID: 1 (easy handle 0x14f010000) > GET /?<script>('alert:XSS')</script> HTTP/2 > Host: buytime.f5-cloud-demo.com > user-agent: curl/7.79.1 > accept: */* > * Connection state changed (MAX_CONCURRENT_STREAMS == 128)! < HTTP/2 200 < date: Sat, 22 Oct 2022 01:04:39 GMT < content-type: text/html; charset=UTF-8 < content-length: 269 < cache-control: no-cache < pragma: no-cache < set-cookie: 1f945=1666400679155-452898837; Path=/; Domain=f5-cloud-demo.com; Expires=Fri, 22 Oct 2032 01:04:39 GMT < set-cookie: 1f9403=/1b+W13c7xNShbbe6zE3KKUDNPCGbxRMVhI64uZny+HFXxpkJMsCKmDWaihBD4KWm82reTlVsS8MumTYQW6ktFQqXeFvrMDFMSKdNSAbVT+IqQfSuVfVRfrtgRkvgzbDEX9TUIhp3xJV3R1jdbUuAAaj9Dhgdsven8FlCaADENYuIlBE; path=/ < x-volterra-location: dal3-dal < server: volt-cdn < x-cache-status: MISS < strict-transport-security: max-age=31536000 < <html><head><title>Request Rejected</title></head> <body>The requested URL was rejected. Please consult with your administrator.<br/><br/> Your support ID is 85281693-eb72-4891-9099-928ffe00869c<br/><br/><a href='javascript:history.back();'>[Go Back]</a></body></html> * Connection #0 to host buytime.f5-cloud-demo.com left intact Notice that the above request intentionally by-passes the CDN cache and is sent to the HTTP LB for the WAF policy to inspect. With this request rejected, you can confirm the attack by navigating to the WAAP HTTP LB Security page under the WAAP Security section within Apps & APIs. After refreshing the page, you’ll see the security violation under the “Top Attacked” panel. Demo To see all of this in action, watch my video below. This uses all of the configuration details above to make a WAAP + CDN service chain in Distributed Cloud. Additional Guides Virtually deploy this solution in our product simulator, or hands-on with step-by-step comprehensive demo guide. The demo guide includes all the steps, including those that are needed prior to deployment, so that once deployed, the solution works end-to-end without any tweaks to local DNS. The demo guide steps can also be automated with Ansible, in case you'd either like to replicate it or simply want to jump to the end and work your way back. Conclusion This shows just how simple it can be to use the Distributed Cloud CDN to frontend your web app protected by a WAF, all natively within the F5 Distributed Cloud’s regional edge POPs. The advantage of this solution should now be clear – the Distributed Cloud CDN is cloud-agnostic, flexible, agile, and you can enforce security policies anywhere, regardless of whether your web app lives on-prem, in and across clouds, or even at the edge. For more information about Distributed Cloud WAAP and Distributed Cloud CDN, visit the following resources: Product website: https://www.f5.com/cloud/products/cdn Distributed Cloud CDN & WAAP Demo Guide: https://github.com/f5devcentral/xcwaapcdnguide Video: https://youtu.be/OUD8R6j5Q8o Simulator: https://simulator.f5.com/s/waap-cdn Demo Guide: https://github.com/f5devcentral/xcwaapcdnguideEnhance your Application Security using Client-side signals
In this two-part post, I’m describing the concept of Client-side signals and how they can be used to enhance your web and mobile application security strategy. As is well known, API and application security are currently at the forefront of today's CISO agenda. These are real concerns, especially due to the accelerated digital transformation that has forced all companies to adapt and quickly build new applications to stay competitive and keep their relevance through the stormy times we have lived these past 3 years. According to the World Bank, over 40% of adults who made merchant in-store and online payments using a card, phone, or the internet did so for the first time since the start of the pandemic. This is an attestation of how much more people are consuming online applications. Delving further into the “adapt and quickly build” of new applications, what transpired was the creation of numerous new APIs, Web and Mobile applications, and an entire ecosystem facilitating deep interdependency between them. When you visit any online application today, be it Web or Mobile, it is highly likely that this application is constructed using APIs and is leveraging a broad array of 3rd-party integrations in the form of APIs or other web components. Below, you will find a simple example of a Web page calling external entities as it loads in a browser: Figure 1. Waterfall The figure above illustrates what we could characterize as an ecosystem comprising internal and external web components. These components provide the necessary services to enable the application to fulfill its intended purpose. A browser loading components of web applications isn’t something new; in fact, browsers are created to perform exactly this function; what is new is the fact that this application requires a whole set of external components that are fetched directly or indirectly by the browser, and, despite these external dependencies being transparent for the end-user, they represent a real challenge from the application security strategy point of view. At F5, we embody what I personally refer to as the “web proxy mindset”. For the past 27 years, a substantial part of our work, of course, in an oversimplified way, has to do with proxying network communications between a client and a server; thereby, proxying these communications gives us the ability to guarantee that all requirements for an application or API are met, whether related to security or connectivity. Figure 2. Typical client-server communication If you search online for the definition of a proxy server, you will find that it is defined as “a server application that acts as an intermediary between a client requesting a resource and the server providing the resource, and it improves privacy, security, and performance in the process.” The paradigmshift for protecting modern applications To fight the new generation of attackers and fraudsters, the ones that usually sell the attack as-a-Service on the dark forums, F5 has enhanced its solutions by integrating our web proxy approach with the client-side signals. We will get into deeper details later in this post, but it could be seen as having a magnified glass to closely monitor the activities within the browser as a person interacts with the application prior to having the actual request sent to the application server. Combining both techniques is a strong strategy to enhance your application security. When discussing the concept of client-side signals with CISOs and their teams, I often find that this methodology is not widely known and is largely underutilized by companies seeking to bolster their security detection capabilities. What are the Client-side signals? Client-side signals are the telemetry that can be pulled from the browser or mobile app while a person or an entity interacts with the application. These signals can typically be categorized into three main groups: a) human interaction signals, b) device environment signals, and c) network signals. Human Interaction Signals This category of signals will allow us to determine if what is using an application is a person or not and if this person is bad or good intended. But how and why is this relevant to your security strategy? Is this a person or not? When this question is raised, how confidently can you answer that? It is well-known in the application security industry that bots are a real problem. These software entities are created for several different purposes; one of them is to “mimic” a person's behavior and, by doing that, imitate the steps and behaviors a human would do while interacting with a page or app. Distinguishing if an entity visiting your application is a Human or a Bot before having your application servers process an HTTP request will automatically improve the overall user experience, as you will offload the burden of processing noise from your servers. It is important to emphasize that Bots are not typically created to overload your servers; they’re created to “mimic” humans with the intention of abusing a legit business logic your application provides and do not get caught as they do that. The examples below were extracted from a real Login application and can give a visual perspective of how Bots can “mimic” humans: Figure 3. - Bot Figure 4. - Human Is this person good or bad intended? I must admit that this is a very tricky question to answer; however, answering it at an initial stage of interaction can significantly enhance your security strategy. Determining intention is a typical methodology used by fraud prevention solutions, but having a glimpse of how a person behaves while interacting with the application and using it to either permit or deny a request is also applicable for the application security teams as it can help detect the initial stages of an attack, which usually involves reconnaissance techniques. What distinguishes a human performing an attack reconnaissance technique from a regular human using your application? There are several factors at play here, and employing AI strategies can probabilistically determine the typical behavior of well-intended users on your application by analyzing things like: How common is having users often leave the active browser tab while interacting with the application? How typical is having users press unusual keys while filling out forms? What is the average time a user spends on each step of your application? How frequently does a user change or use different devices to access the application? Is the mouse moving like an untrained human? Is this person actively submitting any data while using the application or sitting there and clicking randomly, or, maybe, apparently doing nothing? …and several others. Device Environment Signals This category of signals is based on fingerprinting techniques. Device fingerprinting isn’t something new, and there are known ways to get around fingerprinting. However, the goal here isn’t only to fingerprint a device but also to check if the collected signals are contradicting somehow, thus giving the signs that something has been spoofed to avoid real identification. Remember, we are looking for lies! Application security teams should be looking for signals like: Screen Size Fonts Plugins/Extensions CPU/GPU capabilities Graphic rendering capabilities Canvas configurations Browser configurations Time zones …and hundreds of others. Figure 5. Emoji rendering Examples of questions to help identify spoofed devices using these signals could be: Why is this browser saying it is a certain version of Firefox for MacOS, but some properties are only found on Windows OS? Why is this browser rendering a certain emoji as if it were for a different browser or OS? Why is this mobile app showing properties as if it is an emulator? Why is this session coming from a remote desktop? Why is this a Virtual Machine? …and several other questions that can lead to identifying a spoofed device and eventually trigger a security policy. Network Signals This category of signals will provide insights into where this request is coming from. Typically, security solutions rely on IP addresses to determine if a request is permitted or denied, but today, relying only on IP addresses is a very ineffective way of determining the real source of a request. Numerous companies provide VPN and forward proxy services, commonly found in the attacker's toolkit, but the real danger lives on the Botnets. Today’s Botnets are comprised of regular end-users who willingly participate in the network and compromised devices acting as internet proxies without the owner's knowledge. In such cases, relying on a Geo-blocking or IP reputation database may not flag these requests as malicious since the source IP address is often associated with a regular ADSL or LTE/5G mobile network and is not necessarily engaged in malicious activities. Additional effort is required to enhance your ability to identify the true source of a request. Collecting network-based signals, including IP addresses, HTTP headers, TLS fingerprints, and a portion of the request's payload, and combining these signals with device environment and human interaction signals can provide robust insights into the malicious or non-malicious nature of a request. It is important to note that these alone are not a solution to detect a malicious source. More advanced techniques like TCP handshake timing, TCP-related variances and others can also come into play and enhance the overall detection. Also, every browser has its own way of crafting an HTTP request, so the goal here is to detect inconsistencies in the HTTP requests and enhance anomaly detection capabilities. Now that you know some capabilities of the Client-side signals, the next questions might be: How to collect Client-side Signals? How can you guarantee the signals are not altered or faked while they are collected? Making decisions based on bad data leads to false positives and false negatives. How can you confidently say that after collecting the signals, they are sent to your backend in a protected way and are not altered while in transit? What happens when no signals are collected? What practical improvements will you get when you start using the client-side signals? What if you have a compromised 3rd-partycomponent? How can you detect a potential supply-chain issue? These are all interesting questions, and the answers are coming in the second part of this article. Please stay tuned, and thank you for your interest and time to read this article. References: World Bank https://www.worldbank.org/en/news/press-release/2022/06/29/covid-19-drives-global-surge-in-use-of-digital-payments Proxy Server https://en.wikipedia.org/wiki/Proxy_serverProtecting APIs with Access Policy Manager and custom iRules
The problem: Unprotected API - Vulnerable to Overload Without Rate-Limiting Enforcement Our customer in the B2B sector is encountering a challenge with their public API. Despite having implemented a custom method for generating long-lived API keys, they find themselves unable to enforce rate-limiting effectively. This absence of rate-limiting mechanisms poses significant challenges, potentially resulting in the overloading of their system due to excessive requests or the exploitation of their API by unauthorized users. Without proper rate-limiting controls in place, the customer faces risks to both the performance and security of their API infrastructure, necessitating a solution to mitigate these concerns and ensure the smooth operation of their services for their clients. Our customers wants to offer two tiers of service level agreements (SLAs) - gold and standard. Complicating matters further, the API key, integral to authentication, is transmitted via a custom HTTP header. The solution: BIG-IP APM and Custom iRules for Effective Rate-Limiting My solution involves leveraging the API Protection feature of BIG-IP APM in conjunction with a custom iRule. By utilizing this combination, our customer can effectively extract the API Keys from HTTP requests and enforce rate limiting on specific API endpoints. As for now they only want to enforce rate limiting on the POST endpoints. This approach empowers the customer to secure their API while efficiently managing and controlling access to critical endpoints, ensuring optimal performance and safeguarding against abuse or overload. With this iRule we can to extract the API key from the HTTP Requests and store it in a variable, that can later be used by the API Protection feature of the APM. API Keys and the associated SLA level are stored in a Data Groupof the typestring. # Enable (1) or disable (0) logging globally when RULE_INIT { set static::debug 1 } # Access and analyze the HTTP header data for SLA value when HTTP_REQUEST { set sla [class lookup [HTTP::header value apikey] dg_apikeys] if { $static::debug } {log local0. "Made it to HTTP_REQUEST event with SLA value $sla."} } # Evaluate SLA value during per-request access policy execution when ACCESS_PER_REQUEST_AGENT_EVENT { set id [ACCESS::perflow get perflow.irule_agent_id] if { $id eq "read-sla" } { if { $static::debug } {log local0. "Made it to iRule agent in perrequest policy with SLA value $sla."} ACCESS::perflow set perflow.custom "$sla" } } And this is how the Per Request Policy in the API Protection profile looks. It uses the value of the API Key (extracted with the help of the the iRule) and the Source IP of the client to enforce Rate Limiting on the POST endpoints, using two different SLAs. In the APM log you should see the following message, once the client exceeds his quota defined in the SLA. Apr 28 20:12:42 ltm-apm-16.mylab.local notice tmm[11094]: 01870075:5: (null):/Common: API request with weight (1) violated the quota in rate limiting config(/Common/demo_api_ratelimiting_auto_rate_limiting_standard). Apr 28 20:12:42 ltm-apm-16.mylab.local notice tmm[11094]: 0187008d:5: /Common/demo_api_ratelimiting_ap:Common:6600283561834577940: Execution of per request access policy (/Common/demo_api_ratelimiting_prp) done with ending type (Reject) Further reading: You can find a more detailed write-up on my GitHub page: https://github.com/webserverdude/f5_APM_API_Protection There you can find the Per Request Policy explained in all details. The Data Group with for the iRule. A demo API for testing my solution. A Postman Collection for working with my demo API.F5 Distributed Cloud (XC) API Security in Out-of-Band Mode using BIG-IP
Introduction There are API Security use cases, which need to be deployed in out-of-band mode. An example of such use cases is an existing very high-critical API deployment, which cannot risk a new inline point. Another example is API security visibility deployment with so many deployment points, which renders new inline/proxy points becoming too expensive. F5 Distributed Cloud (XC) API Security solution by default, requires an inline deployment to be the most effective solution. It needs a different approach if out-of-band deployment is required. Out-of-band mode in this context, means that F5 XC is deployed outside of the API traffic path flow. This implies that F5 XC is in monitoring mode by receiving the API traffic from mirroring devices or API logging systems. While in monitoring mode, it cannot provide any protection or enforcement without any integration with existing security solutions; inline or F5 XC deployment is changed to be inline mode. The best deployment position for F5 XC API Security remains the inline mode. Architecture F5 XC is designed to be an inline solution. It receives the API request traffic, analyzes it, and forwards it to the backend/origin servers. Receives the API response traffic, analyzes it, and forwards it to the clients. F5 XC is not designed to handle API logs or mirrored traffic. Hence, it requires a separate component to receive the API logs or mirrored traffic and translate them into API request and response traffic, and send them to F5 XC. The architecture below uses BIG-IP as the separate component carrying out the needed functions. The above architecture takes an example of Apigee API Gateway platform, which sends an API trace in XML format containing the request and response traffic of an API call passing through it to a BIG-IP VE (Virtual Edition) deployed in AWS. The BIG-IP VE is licensed with LTM and configured with 3 virtual servers and 3 iRules to carry out the required functions. Step (1): BIG-IP VE receives the uploaded API trace in XML format via vs_log_receiver VS attached with an iRule 01_incoming_xml_to_log_converter. The iRule parses the trace, captures the API request and response traffic, create+save the simulated API response in the cache, create a unique link ID, create a simulated API request, and send the simulated API request to vs_http_to_https using Sideband iRule. Step (2): The vs_http_to_https VS receives the simulated API request traffic and forwards it as an HTTPS request to F5 Distributed Cloud (XC). F5 XC receives the simulated API request and forwards it back to the BIG-IP VE. Step (3): The BIG-IP VE receives the simulated API request from F5 XC, finds the corresponding API response from the cache using the unique link ID inside the request header, and sends the simulated API response to F5 XC. F5 XC then forwards the simulated API response to the BIG-IP VE completing the traffic flow cycle. With those steps, they mimic the situation where F5 XC receives normal API request and response traffic. The difference here is that the client and the server are the simulated one, not the real client and server. F5 XC processes the API traffic according to the API security configuration in a more passive role instead of an active role. The iRules are available at this link. Benefits The benefits of this architecture are: Deploy F5 XC API Security in out-of-band mode without disturbing/changing the existing API traffic flow, which makes the deployment faster and less intrusive. Get the API Discovery from F5 XC to discover API traffic endpoints and provide the analysis. Verify the API Authentication in each API request traffic. Conduct API Security posture management. Display the API security visibility in centralized manner without changing the API environment. Create the right justification cases for the higher-management approval to deploy the F5 XC API Security solution in inline mode. Getfull support for the solution from F5 ecosystem because the solution uses standard F5 solution components. Scaling Ifsignificant capacity is required to process the mirrored API traffic, it is possible to deploy multiple BIG-IP VE instances and load-balance them using AWS Network Load Balancer. A specific iRule to synchronize the response cache entries between BIG-IP VE instances needs to be developed. I leave this scope for further details on implementation. Let me know your thoughts by leaving a comment or two below 🙂Mitigation of OWASP API Security Risk: Unrestricted Access to Sensitive Business Flows using F5 XC
We have already covered different OWASP API risks in our previous articles (check reference section for more details). OWASP continuously analysed API threats in the last few years and has identified new types of risks which are not part of API Security Top 10 - 2019 edition. So, they added these new ones in the 2nd edition of OWASP API Security Top 10 2023 list and this article will cover the nuances of the newly added risk: Unrestricted Access to Sensitive Business Flows. Introduction: API owners should be very cautious of all the API endpoint’s exposed to users and they should identify each endpoint’s business justification. When developing an API Endpoint, we shall understand API use case and its intended scope of user action. Some business flows need to be monitored, restricted or blocked depending on the sensitivity of endpoint data. If any sensitive business flow is not protected, attackers can exploit them and cause some serious damage to the business. Using wide variety of automated tools available in market, hackers can automate the manual processthereby adversely impacting the genuine business workflows. That’s all the theory I have !!. Let’s plunge into a demo application use case and discover how F5 Distributed Cloud Platform (XC) can detect and guard our API application endpoints against this vulnerability. Use case: As part of testing, I was exploring the options available in one of the demo application “F5AIR” which is used for booking some dummy flight tickets and as a promotion this application is also offering 200$ as account balance after every user signup. In the 3rd tab we observed that this balance can be used to create gift cards which can be redeemed by users. After doing thorough research we have identified there are no restrictions on this workflow and it can be exploited using automated tools. Automated tools can be used to create multiple users, generate gift cards from each user and then redeem them into a single valid account to further book flight tickets without paying anything. Because of this risk, businesses can incur losses and so this is marked as a sensitive business flow. Artificial Intelligence is a truly disruptive technology spreading like wildfire and so for the purpose of today’s demo, I am using AXIOM.AI browser extension to automate the above manual workflow steps. It just took me around 30 minutes to understand how it works and was able to automate the above exploited manual steps. After 10 user creations and redeeming their gift cards valid main user will have around 2000$ which can be used to book flight tickets. Note: To showcase how AI tools can be leveraged to exploit modern applications we are using axiom ai tool and intended only for educational purposes. Mitigation Steps: A straightforward one-point solution may not be appropriate for different types of these vulnerabilities. Secops team should dig deeper into their incoming application traffic, differentiate genuine & malicious security data and then identify the API endpoints which are sensitive to their business flows. Once they have analyzed the traffic then they can apply below solutions as per their requirements Configure API Discovery to detect different API vulnerabilities like sensitive Data, API Attributes like Login page, Zombie API, security Posture, etc. You can find more details in this article Configure rate limiting on the sensitive business end points to keep a limit on number of requests - check here for more details on rate limiting Configure API Protection rules for these business API’s to restrict access to applications – check here for more details on API rules Configure Bot Defense to prevent bot attacks – check here for more details on bot protection As an example, let’s consider the above demonstrated AI tool example, to block any botsfrom accessing demo application we can apply bot defense configurations in root folder location “/” as shown below after which bot AI exploit requests can be mitigated. Note: Above config is for this article’s use case, but users must understand the API endpoint’s which should be protected and apply configs appropriately. We can also try other automation tools like postman which may also be blocked as below In F5 XC console if we navigate to this load balancer security events and bot defense dashboards, we can see these requests are blocked. Conclusion: In this article we explored some insights on this newly added OWASP API Security Top 10 risk, then we shed some light on how AI tools have opened floodgates to a new approach of application threats. Finally, we also revealed the final puzzle of how F5 XC Bot defense can become our elixir in identifying and protecting against this OWASP API risk along with novel AI threats. For more information or to get started check links below: OWASP API Security Top 10 2023 OWASP API Security Top 10 - 2019 F5 Distributed Cloud WAAP
OWASP Tactical Access Defense Series: Broken Authentication and BIG-IP APM
The threat of broken authentication poses a significant risk to organizations, potentially leading to unauthorized access and data breaches. In the face of this formidable challenge, F5's Access Policy Manager (APM) emerges as a robust and indispensable solution. By seamlessly integrating advanced authentication mechanisms and comprehensive access controls, F5 BIG-IP APM stands as a stalwart guardian against the vulnerabilities associated with broken authentication. This article explores the pivotal role played by BIG-IP APM in fortifying authentication protocols, mitigating risks, and ensuring a resilient defense against unauthorized access, ultimately safeguarding the integrity and security of sensitive data in today's dynamic digital environment. Broken Authentication Broken Authentication Examples BIG-IP APM and Broken Authentication Related Content Broken Authentication Authentication mechanism is an exposed target due to the nature of this function, as authentication is the first point of entry to any platform. The difficulty to exploit authentication weaknesses differs based on how the authentication platform is secured. In the current digital era the security perimeters are very fluid, and so are the trust boundries for our authentication platforms those require more cautions from the developers and security architects regarding authentication flows. Not only we need to protect authentication endpoints and flows, but also some overlooked items like forget and reset password endpoints. How can we consider endpoint to be vulnerable? Credential stuffing. Brute force attacks targetting users' accounts. Weak Passwords. Sensitive details in the URL (passwords, Tokens). Allow users sensitive actions without confirmation. No validation for the tokens authenticity. Accept unsigned or weak jwt tokens. No validation for jwt expiration. Use of plain-text, non-encrypted or non-hashed passwords. Use of weak encryption algorithms. Endpoint can access each other without proper authentication. Use weak or predictable tokens for intra-endpoint authentication. Broken Authentication Examples Making use of GraphQL query patching to bypass API ratelimiting and brute force user's login. POST /graphql [ {"query":"mutation{login(username:\"victim\",password:\"password\"){token}}"}, {"query":"mutation{login(username:\"victim\",password:\"123456\"){token}}"}, {"query":"mutation{login(username:\"victim\",password:\"qwerty\"){token}}"}, ... {"query":"mutation{login(username:\"victim\",password:\"123\"){token}}"}, ] Update / modify user's sensitive information without API authorization token. PUT /account Authorization: Bearer <token> { "newpassword": "<new_password>" } BIG-IP APM and Broken Authentication We start with creating our Per-Request policy, this policy works in a different way than the per-session policy, as the flow will be evaluted on a per-request basis, making sure to consider variations throught the session life-time. Below are some of the key benefits: Wide range of Authentication, SSO, and MFA mechanisms to properly identify the initiating machine or user. Ability to integrate with 3rd parties to provide additional enforcement decisions based on the organization's policy. Ability to apply endpoint checks on the client side before session initiation. This goes to BIG-IP in general, the ability to apply custom traffic control on both of the traffic sides, Client and Server. The ability to create whitelist / blacklist for API Access tokens, JSON Web Tokens ID (JTI) or a different element based on the used authentication method, below example steps for JWT: Extract JTI value from Access token. Add JTI value to whether Allow/Block lists. Related Content F5 BIG-IP Access Policy Manager | F5 Introduction to OWASP API Security Top 10 2023 OWASP Top 10 API Security Risks – 2023 - OWASP API Security Top 10 API Protection Concepts OWASP Tactical Access Defense Series: How BIG-IP APM Strengthens Defenses Against OWASP Top 10 OWASP Tactical Access Defense Series: Broken Object Level Authorization and BIG-IP APM F5 Hybrid Security Architectures (Part 5 - F5 XC, BIG-IP APM, CIS, and NGINX Ingress Controller)
F5 powered API security and management
Editor's Note:The F5 Beacon capabilities referenced in this article hosted on F5 Cloud Services are planning a migration to a new SaaS Platform - Check out the latesthere. Introduction Application Programming Interfaces (APIs) enable application delivery systems to communicate with each other. According to a survey conducted by IDC, security is the main impediment to delivery of API-based services.Research conducted by F5 Labs shows that APIs are highly susceptible to cyber-attacks. Access or injection attacks against the authentication surface of the API are launched first, followed by exploitation of excessive permissions to steal or alter data that is reachable via the API.Agile development practices, highly modular application architectures, and business pressures for rapid development contribute to security holes in both APIs exposed to the public and those used internally. API delivery programs must include the following elements : (1) Automated Publishing of APIs using Swagger files or OpenAPI files, (2) Authentication and Authorization of API calls, (3) Routing and rate limiting of API calls, (4) Security of API calls and finally (5) Metric collection and visualization of API calls.The reference architecture shown below offers a streamlined way of achieving each element of an API delivery program. F5 solution works with modern automation and orchestration tools, equipping developers with the ability to implement and verify security at strategic points within the API development pipeline. Security gets inserted into the CI/CD pipeline where it can be tested and attached to the runtime build, helping to reduce the attack surface of vulnerable APIs. Common Patterns Enterprises need to maintain and evolve their traditional APIs, while simultaneously developing new ones using modern architectures. These can be delivered with on-premises servers, from the cloud, or hybrid environments. APIs are difficult to categorize as they are used in delivering a variety of user experiences, each one potentially requiring a different set of security and compliance controls. In all of the patterns outlined below, NGINX Controller is used for API Management functions such as publishing the APIs, setting up authentication and authorization, and NGINX API Gateway forms the data path.Security controls are addressed based on the security requirements of the data and API delivery platform. 1.APIs for highly regulated business Business APIs that involve the exchange of sensitive or regulated information may require additional security controls to be in compliance with local regulations or industry mandates.Some examples are apps that deliver protected health information or sensitive financial information.Deep payload inspection at scale, and custom WAF rules become an important mechanism for protecting this type of API. F5 Advanced WAF is recommended for providing security in this scenario. 2.Multi-cloud distributed API Mobile App users who are dispersed around the world need to get a response from the API backend with low latency.This requires that the API endpoints be delivered from multiple geographies to optimize response time.F5 DNS Load Balancer Cloud Service (global server load balancing) is used to connect API clients to the endpoints closest to them.In this case, F5 Cloud Services Essential App protect is recommended to offer baseline security, and NGINX APP protect deployed closer to the API workload, should be used for granular security controls. Best practices for this pattern are described here. 3.API workload in Kubernetes F5 service mesh technology helps API delivery teams deal with the challenges of visibility and security when API endpoints are deployed in Kubernetes environment. NGINX Ingress Controller, running NGINX App Protect, offers seamless North-South connectivity for API calls. F5 Aspen Mesh is used to provide East-West visibility and mTLS-based security for workloads.The Kubernetes cluster can be on-premises or deployed in any of the major cloud provider infrastructures including Google’s GKE, Amazon’s EKS/Fargate, and Microsoft’s AKS. An example for implementing this pattern with NGINX per pod proxy is described here, and more examples are forthcoming in the API Security series. 4.API as Serverless Functions F5 cloud services Essential App Protect offering SaaS-based security or NGINX App Protect deployed in AWS Fargate can be used to inject protection in front of serverless API endpoints. Summary F5 solutions can be leveraged regardless of the architecture used to deliver APIs or infrastructure used to host them.In all patterns described above, metrics and logs are sent to one or many of the following: (1) F5 Beacon (2) SIEM of choice (3) ELK stack.Best practices for customizing API related views via any of these visibility solutions will be published in the following DevCentral series. DevOps can automate F5 products for integration into the API CI/CD pipeline.As a result, security is no longer a roadblock to delivering APIs at the speed of business. F5 solutions are future-proof, enabling development teams to confidently pivot from one architecture to another. To complement and extend the security of above solutions, organizations can leverage the power of F5 Silverline Managed Services to protect their infrastructure against volumetric, DNS, and higher-level denial of service attacks.The Shape bot protection solutions can also be coupled to detect and thwart bots, including securing mobile access with its mobile SDK.OWASP Tactical Access Defense Series: Broken Object Property Level Authorization and BIG-IP APM
AUTHOR NOTE: Unauthorized access to private/sensitive object properties may result in data disclosure, data loss, or data corruption. Under certain circumstances, unauthorized access to object properties can lead to privilege escalation or partial/full account takeover. In this article we are going through API3 item from OWASP top 10 API Security risks exploring BIG-IP Access Policy Manager (APM) role in our arsenal. Identifying Vulnerable APIs In order to identify the API endpoint is vulnerable to Broken Object Property Level Authorization, Sensitive properties exposure of certain object for non-intended user (Excessive Data Exposure). import requests # Assuming the API endpoint for retrieving user data is /api/users api_endpoint = "https://example.com/api/users" # Sending a GET request to the API endpoint response = requests.get(api_endpoint) # Checking if the request was successful (status code 200) if response.status_code == 200: # Printing the response content (which could contain excessive data) print(response.json()) else: print("Failed to retrieve data from the API") API allow to change, add or delete sensitive object property for non-intended user (Mass assignment). import requests # Assuming the API endpoint for updating user information is /api/users api_endpoint = "https://example.com/api/users" # Malicious payload containing additional fields malicious_payload = { "username": "malicious_user", "password": "password123", "isAdmin": True # Malicious user attempts to elevate privileges } # Sending a POST request with the malicious payload response = requests.post(api_endpoint, json=malicious_payload) # Checking if the request was successful (status code 200) if response.status_code == 200: print("User information updated successfully") else: print("Failed to update user information") Object Property Level Authorization involves controlling access to specific properties or attributes of an object within a system. Instead of granting blanket access to an entire object, this approach enables fine-grained control, allowing administrators to restrict or permit access to individual properties based on user roles or permissions. While implementing protection against such security risk involves different aspects, one is making sure the user is authorized to access object property, and here BIG-IP APM plays crucial role. Mitigating Risks with BIG-IP APM BIG-IP APM per-request granularity. With per-request granularity, organizations can dynamically enforce access policies based on various factors such as user identity, device characteristics, and contextual information. This enables organizations to implement fine-grained access controls at the API level, mitigating the risks associated with Broken Object Property Level Authorization. Key Features: Dynamic Access Control Policies: BIG-IP APM empowers organizations to define dynamic access control policies that adapt to changing conditions in real-time. By evaluating each API request against these policies, BIG-IP APM ensures that only authorized users can access specific resources and perform permitted actions. Granular Authorization Rules: BIG-IP APM enables organizations to define granular authorization rules that govern access to individual objects or resources within the API ecosystem. By enforcing strict authorization checks at the object level, F5 APM prevents unauthorized users from tampering with sensitive data or performing unauthorized actions. Conclusion In conclusion, BIG-IP APM per-request granularity is a powerful tool for defending against Broken Object-Level Authorization vulnerabilities in APIs. By enforcing fine-grained access controls at the API level, organizations can mitigate the risks associated with unauthorized access to sensitive data. Additionally, proactive security assessments and vulnerability scans are essential for identifying and addressing vulnerabilities in APIs, thereby strengthening overall security posture in the digital ecosystem. Related Content F5 BIG-IP Access Policy Manager | F5 Introduction to OWASP API Security Top 10 2023 OWASP Top 10 API Security Risks – 2023 - OWASP API Security Top 10 API Protection Concepts OWASP Tactical Access Defense Series: How BIG-IP APM Strengthens Defenses Against OWASP Top 10 OWASP Tactical Access Defense Series: Broken Object Level Authorization and BIG-IP APM F5 Hybrid Security Architectures (Part 5 - F5 XC, BIG-IP APM, CIS, and NGINX Ingress Controller) OWASP Tactical Access Defense Series: Broken Authentication and BIG-IP APMIntegrating NGINX Controller API Management with PingFederate to secure financial services API transactions
Introduction The previous article in the "Securing financial services APIs" series, "Using NGINX Controller API Management Module and NGINX App Protect to secure financial services API transactions", described a setup where NGINX Controller APIm, acting as an OAuth Resource Server, was using F5's APM as an OIDC IdP / OAuth Autorization Server in an OAuth/OIDC authentication flow. The current article explores the integration of NGINX Controller APIm with PingFederate, one of the market leading identity management solutions, in a similar setup. Ping Identity has partnered with OBIE (Open Banking Implementation Entity) the body responsible for UK Open Banking implementation as a response to EU's PSD2 directive and, as such, it acquired a front seat in the development of Open Banking initiative, one of the most mature examples of financial service API. Ping Identity technology is also Financial-Grade API (FAPI) compliant, supporting the features critical in ensuring higher security for financial API transactions, while maintaining seamless user experience and ease of configuration. Ping Identity's PDS2 & Open Banking technical solution guide can be found here, while this article focusses primarily on the ease of configuration of NGINX Controller APIm to interact with PingFederate solution in a basic financial services API scenario. Setup For demo purposes, as a backend banking application we used a server stub generated from UK Open Banking's OpenAPI spec deployed in a Kubernetes environment, having NGINX App Protect deployed on Kubernetes Ingress controller as an API WAF. The API Gateway and API Management function is implemented by NGINX API Gateway and NGINX Controller APIm, placed in front of the Kubernetes environment. The configuration of all the above (backend server, NAP/KIC and NGINX APIm) is managed through a CI/CD pipeline configured in Gitlab, simulating a modern application development environment. Authentication and API flow This demo is implementing the Authorization Code flow to enable a "domestic payment" transaction. Summarising the steps of the authentication and API flow (refer to the setup diagram above): 1. The user logs into the Third Party Provider application ("client") and creates a new funds transfer 2. The TPP application redirects the user to the OAuth Authorization Server / OIDC IdP - PingFederate 3. The user provides its credentials to PingFederate and gets access to the consent management screen where the required "payments" scope will be listed 4. If the user agrees to give consent to the TPP client to make payments out of his/her account, PingFederate will generate an authorization code (and an ID Token) and redirect the user to the TPP client 5. The TPP client exchanges the authorization code for an access token and attaches it as a bearer token to the /domestic-payments call sent to the API gateway 6. The API Gateway authenticates the access token by downloading the JSON Web Keys from PingFederate and grants conditional access to the backend application 7. The Kubernetes Ingress receives the API call and performs WAF security checks via NGINX App Protect 8. The API call is forwarded to the backend server pod NGINX APIm configuration In this scenario, NGINX APIm is performing the Resource Server OAuth role, where it downloads the JWKs from the OAuth Authorization Server / OIDC IdP (PingFederate) and checks the authenticity of the access token presented in the API call. Additionally, it may apply further checks to conditionally grant access to the application - in this demo it will check for the presence of the "payments" scope. The NGINX APIm configuration is straightforward and consists of two steps: 1. Configuring the IdP Go to Services => Identity Providers and click on Create identity Providers. Fill in the mandatory parameters Name, Environment and Type (JWT). Enter the JWKs URL location and the caching duration. 2. Configuring the OAuth authorization and conditional access criteria Go to Services =>APIs , select an API Definition and edit the associated Published API. Navigate to Routing and edit the Component to be protected, navigating to Security/Authentication. Select the previously created Identity Provider and optionally enable conditional access. As an example, access is granted if "payments" is one of the scopes found in the access token. Conclusion NGINX APIm offers a very simple yet granular way of configuring NGINX API Gateway as an OAuth Resource Server and allows the integration with an industry-leading IAM solution, PingFederate, to protect financial services API transactions. Links UDF lab environment link.
Guarding privacy and security using API Gateways: Implement the Phantom Token Flow using NGINX and the Curity Identity Server
ByDamian Curry, NGINX Business Development Technical Director, F5,andMichal Trojanowski, Product Marketing Engineer,Curity ---- In today’s world, APIs are ubiquitous, either in communication between-backend services or from front ends to back ends. They serve all kinds of purposes and come in different flavors and can return data in various formats. The possibilities are countless. Still, they all share one common trait – an API needs to be secure. Secure access to an API should be paramount for any company exposing them, especially if the APIs are available externally and consumed by third-party clients. To help organizations address the critical topic of API security, OWASP has provided guidelines to ensure safety. In 2019, OWASP released a “top 10” compilation of the most common security vulnerabilities in APIs. OAuth and JWT as a Sign of Mature API Security Modern, mature APIs are secured using access tokens issued according to the OAuth standard or ones that build on top of it. Although OAuth does not require the use of JSON Web Token (JWT) format for access tokens, per se, it is common practice to use them. Signed JWTs (JWTs signed using JSON Web Signing, JWS) have become a popular solution because they include built-in mechanisms that protect their integrity. Although signed JWTs are a reliable mechanism when the integrity of the data is considered, they are not as goodin regards toprivacy. A signed JWT (JWS) protects integrity of the data, while an encrypted one (JWE) protects also the privacy. A JSON Web Token consists of claims which can carry information about the resource owner – the user who has granted access to their resources or even about the API itself. Anyone in possession of a signed JWT can decode it and read the claims inside. This can produce various issues: You have to be really careful when putting claims about the user or your API in the token. If any Personally Identifiable Information (PII) ends up in there, this data essentially becomes public. Striving to keep PII private should be the goal of your organization. What is more, in some countries, privacy is protected by laws like GDPR or CCPA, and it can be an offense not to keep PII confidential. Developers of apps consuming your access tokens can start depending on the contents of your tokens. This can lead to a situation where making updates to the contents of your access tokens breaks the integrations with your API. Such a situation would be an inconvenience both for your company and your partners. An access token should be opaque to the consumers of your API, but this can be hard to enforce outside of your organization. By-reference Tokens and the Phantom Token Flow When there is a problem, there must be a solution. In fact, there are two ways to address these aforementioned problems: Encryption Use by-reference (AKA handle) tokens To protect the contents of JWTs from being visible, the token can be encrypted (thus creating a JWE inside the JWS). Encrypted content will be safe as long as you use a strong encryption algorithm and safeguard the keys used for encryption. To use JWEs, though, you need a Public Key Infrastructure (PKI), and setting one up can be a big deal. If you do not use PKI, you need another mechanism to exchange keys between the Authorization Server and the client. As JWEs also need to be signed, your APIs have to be able to access over TLS the keys used to verify and decrypt tokens. These requirements add considerable complexity to the whole system. Another solution is to use by-reference tokens – an opaque string that serves as a reference to the actual data kept securely by the Authorization Server. Using opaque tokens protects the privacy of the data normally kept in tokens, but now your services, which process requests, need some other way to get the information that was earlier available in the JWT’s claims. Here,the Token Introspectionstandard can prove useful. Using a standardized HTTP call to the Authorization service, the API can exchange the opaque token for a set of claims in JSON format. Exchanging an opaque token for a set of claims comes to the API just as easily as extracting the data from decoding a JWT. Because the API connects securely to the Authentication Server and trusts it, the response does not have to be signed, as is the case with JWTs. Still, in a world dominated by microservices, a problem may arise with the proposed solution. Have a look at this diagram below. Note that every service which processes the request needs to perform the introspection. When there are many services processing one request, the introspection must be done repeatedly or else the APIs would have to pass unsigned JSONs and trust the caller with the authorization data – which can lead to security issues. Situation in which every service calls the Authorization Server to introspect the token can put excess load on the Authorization Server, which will have to return the same set of claims many times. It can also overload the network between the services and the Authorization Server. If your API consists of numerous microservices, there is probably some kind of a reverse proxy or an API gateway standing in front of all of them, e.g., an instance of NGINX. Such a gatewayis capable of performing the introspection flow for you. This kind of setup is called a Phantom Token Flow. The Authorization Server issues opaque tokens and serves them to the client. The client uses the token as any other token – by appending it to the request in theAuthorizationheader. When the gateway receives a request, it extracts the opaque token from the header and performs the introspection flow, but it is a bit different from the flow performed by APIs. According to the specification, when the API introspects a token, it receives a JSON with the claims that would usually end up in a JWT. For an API, it is enough, as it trusts the Authorization Server and just needs the data associated with the access token, the claims do not have to be signed. For the API gateway, though, it’s not enough. The gateway needs to exchange the opaque token for the set of data but will be passing this data to the downstream services together with the request. The APIs must be sure that the claims they received from the gateway have not been tampered with. That’s why some Authorization Servers allow returning a JWT from the introspection endpoint. The API gateway can introspect the opaque token, but in the response, it gets a JWT that corresponds to the access token. This JWT can then be added to the request’sAuthorizationheader, and the downstream APIs can use it as any other JWT. Thanks to this solution, APIs can securely passthe access token between themselves and at the same time have simple access to the claims, without the need of contacting the Authorization Server. Using the Phantom Token Flow allows you to leverage the power of JWTs inside your infrastructure while at the same time keeping high levels of security and privacy outside of your organization. The network workload is greatly reduced because now only the gateway needs to contact the Authorization Server - the introspection is done only once for every request. Also, the amount of work the gateway needs to do is limited – it does not have to parse the body or decode the JWT in order to decide whether the token is valid or has not expired. The gateway can depend only on the status code of the response from the Authorization Server. An OK response means that the token is valid and has not expired. The amount of network traffic can be reduced even further. The API gateway can cache the Authorization Server’s response for as long as the server tells it to (which will usually be the lifetime of the access token). If the opaque reference token is globally unique, this works as an ideal cache key, and the mapping in the cache can even be shared across any API. Implementing the Phantom Token Flow using NGINX and theCurityIdentity Server AtCurity, we have created anopen-source modulefor NGINX to facilitate implementing the Phantom Token flow. The module takes care of performing the introspection and keeping the result in the cache (if caching is enabled). As values of the opaque token are globally unique, so the cache can be as well; there is no need to keep a reference on a per-client basis. The NGINX module is easy to use as all parameters can be set using standard NGINX configuration directives. The module can be built from source, but there are binaries available for a few different Linux distributions and NGINX versions. You can find them in thereleasessection on GitHub. Installing and Configuring the Module Once downloaded, the module needs to be enabled in your instance of NGINX unless you want to create your own build of NGINX with all your modules embedded. To enable the module, add this directive in the main context of your configuration: load_modulemodules/ngx_curity_http_phantom_token_module.so; Then you can apply the phantom token flow to chosen servers and locations. Here’s an example configuration that enables the Phantom Token Flow for an/apiendpoint. The configuration assumes that: Your NGINX instance runs on a different host (nginx.example.com) than yourCurityIdentity Server instance (curity.example.com). The API that will eventually process the request listens onexample.com/api. TheCurityIdentity Server’s introspection endpoint is/oauth/v2/introspection. Responses from introspection are cached by NGINX. http { proxy_cache_path/path/to/cache/cache levels=1:2keys_zone=my_cache:10mmax_size=10g inactive=60muse_temp_path=off; server { server_namenginx.example.com; location /api{ proxy_passhttps://example.com/api; phantom_tokenon; phantom_token_client_credential"client_id" "client_secret"; phantom_token_introspection_endpointcurity; } locationcurity{ proxy_pass"https://curity.example.com/oauth/v2/introspection"; proxy_cache_methodsPOST; proxy_cachemy_cache; proxy_cache_key$request_body; proxy_ignore_headersSet-Cookie; } } } Now, your NGINX gateway is capable of performing the Phantom Token Flow! Conclusion The Phantom Token Flow is a recommended practice for dealing with access tokens that are available publicly. The flow is especially effective when there are many microservices processing one request, which would otherwise have to introspect the tokens on their own. Using Phantom Tokens enhances your security and the privacy of your users, which can never be underestimated. With the help of NGINX as your gateway and theCurityIdentity Server as the Authorization Server implementing the Phantom Token Flow is as simple as loading a module in your NGINX server! Sign up now for a free API Gateway Webinar to learn more about guarding privacy and security with Curity and NGINX
