NGINX WAF

27 Topics

What is Message Queue Telemetry Transport (MQTT)? How to secure MQTT?
MQTT is a messaging protocol broadly used in IoT and connected services, very lightweight and reliable even over poor quality networks. It is designed lightweight so it can work on constrained devices but, even in its latest version MQTTv5, the attack surface is very large.
Fouad_Chmainy
Jan 30, 2025 Place Technical Articles
306Views
0likes
1Comment
Mitigate OWASP LLM Security Risk: Sensitive Information Disclosure Using F5 NGINX App Protect
This short WAF security article covered the critical security gaps present in current generative AI applications, emphasizing the urgent need for robust protection measures in LLM design deployments. Finally we also demonstrated how F5 Nginx App Protect v5 offers an effective solution to mitigate the OWASP LLM Top 10 risks.
Janibasha
Oct 09, 2024 Place Technical Articles
328Views
2likes
0Comments
Safeguard your Apps and APIs using NGINX App Protect WAF 5.0

Akash_Ananthanarayan
Jul 01, 2024 Place Technical Articles
282Views
0likes
0Comments
Dashboards for NGINX App Protect
Introduction NGINX App Protect is a new generation WAF from F5 which is built in accordance with UNIX philosophy such that it does one thing well everything else comes from integrations. NGINX App Protect is extremely good at HTTP traffic security. It inherited a powerful WAF engine from BIG-IP and light footprint and high performance from NGINX. Therefore NGINX App Protect brings fine-grained security to all kinds of insertion points where NGINX use to be either on-premises or cloud-based. Therefore NGINX App Protect is a powerful and flexible security tool but without any visualization capabilities which are essential for a good security product. As mentioned above everything besides primary functionality comes from integrations. In order to introduce visualization capabilities, I've developed an integration between NGINX App Protect and ELK stack (Elasticsearch, Logstash, Kibana) as one of the most adopted stacks for log collection and visualization. Based on logs from NGINX App Protect ELK generates dashboards to clearly visualize what WAF is doing. Overview Dashboard Currently, there are two dashboards available. "Overview" dashboard provides a high-level view of the current situation and also allows to discover historical trends. You are free to select any time period of interest and filter data simply by clicking on values right on the dashboard. Table at the bottom of the dashboard lists all requests within a time frame and allows to see how exactly a request looked like. False Positives Dashboard Another useful dashboard called "False Positives" helps to identify false positives and adjust WAF policy based on findings. For example, the chart below shows the number of unique IPs that hit a signature. Under normal conditions when traffic is mostly legitimate "per signature" graphs should fluctuate around zero because legitimate users are not supposed to hit any of signatures. Therefore if there is a spike and amount of unique IPs which hit a signature is close to the total amount of sources then likely there is a false positive and policy needs to be adjusted. Conclusion This is an open-source and community-driven project. The more people contribute the better it becomes. Feel free to use it for your projects and contribute code or ideas directly to the repo. The plan is to make these dashboards suitable for all kinds of F5 WAF flavors including AWAF and EAP. This should be simple because it only requires logstash pipeline adjustment to unify logs format stored in elasticsearch index. If you have a project for AWAF or EAP going on and would like to use dashboards please feel free to develop and create a pull request with an adjusted logstash pipeline to normalize logs from other WAFs. Github repo: https://github.com/464d41/f5-waf-elk-dashboards Feel free to reach me with questions. Good luck!
Mikhail_Fedoro1
Sep 26, 2023 Place Technical Articles
2.1KViews
4likes
1Comment
Implementing BIG-IP WAF logging and visibility with ELK
Scope This technical article is useful for BIG-IP users familiar with web application security and the implementation and use of the Elastic Stack. This includes, application security professionals, infrastructure management operators and SecDevOps/DevSecOps practitioners. The focus is for WAF logs exclusively. Firewall, Bot, or DoS mitigation logging into the Elastic Stack is the subject of a future article. Introduction This article focusses on the required configuration for sending Web Application Firewall (WAF) logs from the BIG-IP Advanced WAF (or BIG-IP ASM) module to an Elastic Stack (a.k.a. Elasticsearch-Logstash-Kibana or ELK). First, this article goes over the configuration of BIG-IP. It is configured with a security policy and a logging profile attached to the virtual server that is being protected. This can be configured via the BIG-IP user interface (TMUI) or through the BIG-IP declarative interface (AS3). The configuration of the Elastic Strack is discussed next. The configuration of filters adapted to processing BIP-IP WAF logs. Finally, the article provides some initial guidance to the metrics that can be taken into consideration for visibility. It discusses the use of dashboards and provides some recommendations with regards to the potentially useful visualizations. Pre-requisites and Initial Premise For the purposes of this article and to follow the steps outlined below, the user will need to have at least one BIG-IP Adv. WAF running TMOS version 15.1 or above (note that this may work with previous version but has not been tested). The target BIG-IP is already configured with: A virtual Server A WAF policy An operational Elastic Stack is also required. The administrator will need to have configuration and administrative privileges on both the BIG-IP and Elastic Stack infrastructure. They will also need to be familiar with the network topology linking the BIG-IP with the Elastic Search cluster/infrastructure. It is assumed that you want to use your Elastic Search (ELK) logging infrastructure to gain visibility into BIG-IP WAF events. Logging Profile Configuration An essential part of getting WAF logs to the proper destination(s) is the Logging Profile. The following will go over the configuration of the Logging Profile that sends data to the Elastic Stack. Overview of the steps: Create Logging Profile Associate Logging Profile with the Virtual Server After following the procedure below On the wire, logs lines sent from the BIG-IP are comma separated value pairs that look something like the sample below: Aug 25 03:07:19 localhost.localdomainASM:unit_hostname="bigip1",management_ip_address="192.168.41.200",management_ip_address_2="N/A",http_class_name="/Common/log_to_elk_policy",web_application_name="/Common/log_to_elk_policy",policy_name="/Common/log_to_elk_policy",policy_apply_date="2020-08-10 06:50:39",violations="HTTP protocol compliance failed",support_id="5666478231990524056",request_status="blocked",response_code="0",ip_client="10.43.0.86",route_domain="0",method="GET",protocol="HTTP",query_string="name='",x_forwarded_for_header_value="N/A",sig_ids="N/A",sig_names="N/A",date_time="2020-08-25 03:07:19",severity="Error",attack_type="Non-browser Client,HTTP Parser Attack",geo_location="N/A",ip_address_intelligence="N/A",username="N/A",session_id="0",src_port="39348",dest_port="80",dest_ip="10.43.0.201",sub_violations="HTTP protocol compliance failed:Bad HTTP version",virus_name="N/A",violation_rating="5",websocket_direction="N/A",websocket_message_type="N/A",device_id="N/A",staged_sig_ids="",staged_sig_names="",threat_campaign_names="N/A",staged_threat_campaign_names="N/A",blocking_exception_reason="N/A",captcha_result="not_received",microservice="N/A",tap_event_id="N/A",tap_vid="N/A",vs_name="/Common/adv_waf_vs",sig_cves="N/A",staged_sig_cves="N/A",uri="/random",fragment="",request="GET /random?name=' or 1 = 1' HTTP/1.1\r\n",response="Response logging disabled" Please choose one of the methods below. The configuration can be done through the web-based user interface (TMUI), the command line interface (TMSH), directly with a declarative AS3 REST API call, or with the BIG-IP native REST API. This last option is not discussed herein. TMUI Steps: Create Profile Connect to the BIG-IP web UI and login with administrative rights Navigate to Security >> Event Logs >> Logging Profiles Select “Create” Fill out the configuration fields as follows: Profile Name (mandatory) Enable Application Security Set Storage Destination to Remote Storage Set Logging Format to Key-Value Pairs (Splunk) In the Server Addresses field, enter an IP Address and Port then click on Add as shown below: Click on Create Add Logging Profile to virtual server with the policy Select target virtual server and click on the Security tab (Local Traffic >> Virtual Servers : Virtual Server List >> [target virtualserver] ) Highlight the Log Profile from the Available column and put it in the Selected column as shown in the example below (log profile is “log_all_to_elk”): Click on Update At this time the BIG-IP will forward logs Elastic Stack. TMSH Steps: Create profile ssh into the BIG-IP command line interface (CLI) from the tmsh prompt enter the following: create security log profile [name_of_profile] application add { [name_of_profile] { logger-type remote remote-storage splunk servers add { [IP_address_for_ELK]:[TCP_Port_for_ELK] { } } } } For example: create security log profile dc_show_creation_elk application add { dc_show_creation_elk { logger-type remote remote-storage splunk servers add { 10.45.0.79:5244 { } } } } 3. ensure that the changes are saved: save sys config partitions all Add Logging Profile to virtual server with the policy 1. From the tmsh prompt (assuming you are still logged in) enter the following: modify ltm virtual [VS_name] security-log-profiles add { [name_of_profile] } For example: modify ltm virtual adv_waf_vs security-log-profiles add { dc_show_creation_elk } 2. ensure that the changes are saved: save sys config partitions all At this time the BIG-IP sends logs to the Elastic Stack. AS3 Application Services 3 (AS3) is a BIG-IP configuration API endpoint that allows the user to create an application from the ground up. For more information on F5’s AS3, refer to link. In order to attach a security policy to a virtual server, the AS3 declaration can either refer to a policy present on the BIG-IP or refer to a policy stored in XML format and available via HTTP to the BIG-IP (ref. link). The logging profile can be created and associated to the virtual server directly as part of the AS3 declaration. For more information on the creation of a WAF logging profile, refer to the documentation found here. The following is an example of a pa rt of an AS3 declaration that will create security log profile that can be used to log to Elastic Stack: "secLogRemote": { "class": "Security_Log_Profile", "application": { "localStorage": false, "maxEntryLength": "10k", "protocol": "tcp", "remoteStorage": "splunk", "reportAnomaliesEnabled": true, "servers": [ { "address": "10.45.0.79", "port": "5244" } ] } In the sample above, the ELK stack IP address is 10.45.0.79 and listens on port 5244 for BIG-IP WAF logs. Note that the log format used in this instance is “Splunk”. There are no declared filters and thus, only the illegal requests will get logged to the Elastic Stack. A sample AS3 declaration can be found here. ELK Configuration The Elastic Stack configuration consists of creating a new input on Logstash. This is achieved by adding an input/filter/ output configuration to the Logstash configuration file. Optionally, the Logstash administrator might want to create a separate pipeline – for more information, refer to this link. The following is a Logstash configuration known to work with WAF logs coming from BIG-IP: input { syslog { port => 5244 } } filter { grok { match => { "message" => [ "attack_type=\"%{DATA:attack_type}\"", ",blocking_exception_reason=\"%{DATA:blocking_exception_reason}\"", ",date_time=\"%{DATA:date_time}\"", ",dest_port=\"%{DATA:dest_port}\"", ",ip_client=\"%{DATA:ip_client}\"", ",is_truncated=\"%{DATA:is_truncated}\"", ",method=\"%{DATA:method}\"", ",policy_name=\"%{DATA:policy_name}\"", ",protocol=\"%{DATA:protocol}\"", ",request_status=\"%{DATA:request_status}\"", ",response_code=\"%{DATA:response_code}\"", ",severity=\"%{DATA:severity}\"", ",sig_cves=\"%{DATA:sig_cves}\"", ",sig_ids=\"%{DATA:sig_ids}\"", ",sig_names=\"%{DATA:sig_names}\"", ",sig_set_names=\"%{DATA:sig_set_names}\"", ",src_port=\"%{DATA:src_port}\"", ",sub_violations=\"%{DATA:sub_violations}\"", ",support_id=\"%{DATA:support_id}\"", "unit_hostname=\"%{DATA:unit_hostname}\"", ",uri=\"%{DATA:uri}\"", ",violation_rating=\"%{DATA:violation_rating}\"", ",vs_name=\"%{DATA:vs_name}\"", ",x_forwarded_for_header_value=\"%{DATA:x_forwarded_for_header_value}\"", ",outcome=\"%{DATA:outcome}\"", ",outcome_reason=\"%{DATA:outcome_reason}\"", ",violations=\"%{DATA:violations}\"", ",violation_details=\"%{DATA:violation_details}\"", ",request=\"%{DATA:request}\"" ] } break_on_match => false } mutate { split => { "attack_type" => "," } split => { "sig_ids" => "," } split => { "sig_names" => "," } split => { "sig_cves" => "," } split => { "staged_sig_ids" => "," } split => { "staged_sig_names" => "," } split => { "staged_sig_cves" => "," } split => { "sig_set_names" => "," } split => { "threat_campaign_names" => "," } split => { "staged_threat_campaign_names" => "," } split => { "violations" => "," } split => { "sub_violations" => "," } } if [x_forwarded_for_header_value] != "N/A" { mutate { add_field => { "source_host" => "%{x_forwarded_for_header_value}"}} } else { mutate { add_field => { "source_host" => "%{ip_client}"}} } geoip { source => "source_host" } } output { elasticsearch { hosts => ['localhost:9200'] index => "big_ip-waf-logs-%{+YYY.MM.dd}" } } After adding the configuration above to the Logstash parameters, you will need to restart the Logstash instance to take the new logs into configuration. The sample above is also available here. The Elastic Stack is now ready to process the incoming logs. You can start sending traffic to your policy and start seeing logs populating the Elastic Stack. If you are looking for a test tool to generate traffic to your Virtual Server, F5 provides a simple WAF tester tool that can be found here. At this point, you can start creating dashboards on the Elastic Stack that will satisfy your operational needs with the following overall steps: · Ensure that the log index is being created (Stack Management >> Index Management) · Create a Kibana Index Pattern (Stack Management>>Index patterns) · You can now peruse the logs from the Kibana discover menu (Discover) · And start creating visualizations that will be included in your Dashboards (Dashboards >> Editing Simple WAF Dashboard) A complete Elastic Stack configuration can be found here – note that this can be used with both BIG-IP WAF and NGINX App Protect. Conclusion You can now leverage the widely available Elastic Stack to log and visualize BIG-IP WAF logs. From dashboard perspective it may be useful to track the following metrics: -Request Rate -Response codes -The distribution of requests in term of clean, blocked or alerted status -Identify the top talkers making requests -Track the top URL’s being accessed -Top violator source IP An example or the dashboard might look like the following:
Romain
May 09, 2023 Place Technical Articles
13KViews
5likes
6Comments
L7 DoS Protection with NGINX App Protect DoS
Intro NGINX security modules ecosystem becomes more and more solid. Current App Protect WAF offering is now extended by App Protect DoS protection module. App Protect DoS inherits and extends the state-of-the-art behavioral L7 DoS protection that was initially implemented on BIG-IP and now protects thousands of workloads around the world. In this article, I’ll give a brief explanation of underlying ML-based DDoS prevention technology and demonstrate few examples of how precisely it stops various L7 DoS attacks. Technology It is important to emphasize the difference between the general volumetric-based protection approach that most of the market uses and ML-based technology that powers App Protect DoS. Volumetric-based DDoS protection is an old and well-known mechanism to prevent DDoS attacks. As the name says, such a mechanism counts the number of requests sharing the same source or destination, then simply drops or applies rate-limiting after some threshold crossed. For instance, requests sourcing the same IP are dropped after 100 RPS, requests going to the same URL after 200 RPS, and rate-limiting kicks in after 500 RPS for the entire site. Obviously, the major drawback of such an approach is that the selection criterion is too rough. It can cause erroneous drops of valid user requests and overall service degradation. The phenomenon when a security measure blocks good requests is called a “false positive”. App Protect DoS implements much more intelligent techniques to detect and fight off DDoS attacks. At a high level, it monitors all ongoing traffic and builds a statistical model in other words a baseline in a process called “learning”. The learning process almost never stops, therefore a baseline automatically adjusts to the current web application layout, a pattern of use, and traffic intensity. This is important because it drastically reduces maintenance cost and reaction speed for the solution. There is no more need to manually customize protection configuration for every application or traffic change. Infinite learning produces a legitimate question. Why can’t the system learn attack traffic as a baseline and how does it detect an attack then? To answer this question let us define what a DDoS attack is. A DDoS attack is a traffic stream that intends to deny or degrade access to a service. Note, the definition above doesn’t focus on the amount of traffic. ‘Low and slow’ DDoS attacks can hurt a service as severely as volumetric do. Traffic is only considered malicious when a service level degrades. So, this means that attack traffic can become a baseline, but it is not a big deal since protected service doesn’t suffer. Now only the “service degradation” term separates us from the answer. How does the App Protect DoS measure a service degradation? As humans, we usually measure the quality of a web service in delays. The longer it takes to get a response the more we swear. App Protect DoS mimics human behavior by measuring latency for every single transaction and calculates the level of stress for a service. If overall stress crosses a threshold App Protect DoS declares an attack. Think of it; a service degradation triggers an attack signal, not a traffic volume. Volume is harmless if an application server manages to respond quickly. Nice! The attack is detected for a solid reason. What happens next? First of all, the learning process stops and rolls back to a moment when the stress level was low. The statistical model of the traffic that was collected during peacetime becomes a baseline for anomaly detection. App Protect DoS keeps monitoring the traffic during an attack and uses machine learning to identify the exact request pattern that causes a service degradation. Opposed to old-school volumetric techniques it doesn’t use just a single parameter like source IP or URL, but actually builds as accurate as possible signature of entire request that causes harm. The overall number of parameters that App Protect DoS extracts from every request is in the dozens. A signature usually contains about a dozen including source IP, method, path, headers, payload content structure, and others. Now you can see that App Protect DoS accuracy level is insane comparing to volumetric vectors. The last part is mitigation. App Protect DoS has a whole inventory of mitigation tools including accurate signatures, bad actor detection, rate-limiting or even slowing down traffic across the board, which it uses to return service. The strategy of using those is convoluted but the main objective is to be as accurate as possible and make no harm to valid users. In most cases, App Protect DoS only mitigates requests that match specific signatures and only when the stress threshold for a service is crossed. Therefore, the probability of false positives is vanishingly low. The additional beauty of this technology is that it almost doesn’t require any configuration. Once enabled on a virtual server it does all the job "automagically" and reports back to your security operation center. The following lines present a couple of usage examples. Demo Demo topology is straightforward. On one end I have a couple of VMs. One of them continuously generates steady traffic flow simulating legitimate users. The second one is supposed to generate various L7 DoS attacks pretending to be an attacker. On the other end, one VM hosts a demo application and another one hosts NGINX with App Protect DoS as a protection tool. VM on a side runs ELK cluster to visualize App Protect DoS activity. Workflow of a demo aims to showcase a basic deployment example and overall App Protect DoS protection technology. First, I’ll configure NGINX to forward traffic to a demo application and App Protect DoS to apply for DDoS protection. Then a VM that simulates good users will send continuous traffic flow to App Protect DoS to let it learn a baseline. Once a baseline is established attacker VM will hit a demo app with various DoS attacks. While all this battle is going on our objective is to learn how App Protect DoS behaves, and that good user's experience remains unaffected. Similar to App Protect WAF App Protect DoS is implemented as a separate module for NGINX. It installs to a system as an apt/yum package. Then hooks into NGINX configuration via standard “load_module” directive. load_module modules/ngx_http_app_protect_dos_module.so; Once loaded protection enables under either HTTP, server, or location sections. Depending on what would you like to protect. app_protect_dos_enable [on|off] By default, App Protect DoS takes a protection configuration from a local policy file “/etc/nginx/BADOSDefaultPolicy.json” { "mitigation_mode" : "standard", "use_automation_tools_detection": "on", "signatures" : "on", "bad_actors" : "on" } As I mentioned before App Protect DoS doesn’t require complex config and only takes four parameters. Moreover, default policy covers most of the use cases therefore, a user only needs to enable App Protect DoS on a protected object. The next step is to simulate good users’ traffic to let App Protect DoS learn a good traffic pattern. I use a custom bash script that generates about 6-8 requests per second like an average surfing activity. While inspecting traffic and building a statistical model of good traffic App Protect DoS sends logs and metrics to Elasticsearch so we can monitor all its activity. The dashboard above represents traffic before/after App Protect DoS, degree of application stress, and mitigations in place. Note that the rate of client-side transactions matches the rate of server-side transactions. Meaning that all requests are passing through App Protect DoS and there are not any mitigations applied. Stress value remains steady since the backend easily handles the current rate and latency does not increase. Now I am launching an HTTP flood attack. It generates several thousands of requests per second that can easily overwhelm an unprotected web server. My server has App Protect DoS in front applying all its’ intelligence to fight off the DoS attack. After a few minutes of running the attack traffic, the dashboard shows the following situation. The attack tool generated roughly 1000 RPS. Two charts on the left-hand side show that all transactions went through App Protect DoS and were reaching a demo app for a couple of minutes causing service degradation. Right after service stress has reached a threshold an attack was declared (vertical red line on all charts). As soon as the attack has been declared App Protect DoS starts to apply mitigations to resume the service back to life. As I mentioned before App Protect DoS tries its best not to harm legitimate traffic. Therefore, it iterates from less invasive mitigations to more invasive. During the first several seconds when App Protect DoS just detected an attack and specific anomaly signature is not calculated yet. App Protect DoS applies an HTTP redirect to all requests across the board. Such measure only adds a tiny bit of latency for a web browser but allows it to quickly filter out all not-so-intelligent attack tools that can’t follow redirects. In less than a minute specific anomaly signature gets generated. Note how detailed it is. The signature contains 11 attributes that cover all aspects: method, path, headers, and a payload. Such a level of granularity and reaction time is not feasible neither for volumetric vectors nor a SOC operator armed with a regex engine. Once a signature is generated App Protect DoS reduces the scope of mitigation to only requests that match the signature. It eliminates a chance to affect good traffic at all. Matching traffic receives a redirect and then a challenge in case if an attacker is smart enough to follow redirects. After few minutes of observation App Protect DoS identifies bad actors since most of the requests come from the same IP addresses (right-bottom chart). Then switches mitigation to bad actor challenge. Despite this measure hits all the same traffic it allows App Protect DoS to protect itself. It takes much fewer CPU cycles to identify a target by IP address than match requests against the signature with 11 attributes. From now on App Protect DoS continues with the most efficient protection until attack traffic stops and server stress goes away. The technology overview and the demo above expose only a tiny bit of App Protect DoS protection logic. A whole lot of it engages for more complicated attacks. However, the results look impressive. None of the volumetric protection mechanisms or even a human SOC operator can provide such accurate mitigation within such a short reaction time. It is only possible when a machine fights a machine.
Mikhail_Fedoro1
Dec 16, 2022 Place Technical Articles
4.5KViews
1like
3Comments
WAF Policy Editor - a Web-Based Tool to Configure a Declarative WAF Policy
Introduction The declarative policy is a great step forward to unify WAF configuration across F5 WAFs. However, the policy format is JSON; easy for machines to deal with but not so easy for humans. Currently, the process of putting together a declarative WAF policy requires a human to carefully read through tons of online documentation, figure out what features need to be enabled, and how each feature works. Once understood, the typical human will read one more time to examine configuration examples and then type out the JSON structure, without typos. Any typo may cause the WAF engine to reject a policy. Not a terribly user-friendly procedure. The following project exists to address this kind of complexity. The Project WAF Policy Editor is a web-based tool that implements a UI to put together a declarative WAF policy. The basic concept is simple. Everything you configure in the UI will translate into a JSON file automatically and vice versa. The following screenshot gives an overview of the UI. The menu ribbon at the very top lists all of the supported features. Input fields in the middle configure a policy. The text area at the bottom represents the current policy state. The user is free to modify a policy via either input fields or directly in the JSON. Both representations are synchronized. Another important aspect is that the policy editor continuously verifies policy validity and notifies a user if the configuration doesn't comply with a policy schema. For instance, the screenshot below informs a user that an application language field can only contain specific values. The Workflows There are few workflows for which this tool is designed. The basic workflow is when a user configures a policy from scratch using the UI to set up desired features. Once done a policy can be simply either copied or downloaded as a file. The second approach allows modifying an existing policy. It is similar to previous, however, once a user pastes an existing policy JSON to the text area it gets automatically translated to UI so a user can make desired modifications. The last workflow allows to modify the existing policy as well but instead of pasting a policy a user can simply reference it as a query string parameter "ref". This is handy when a policy is located in a repository. Example link Even though it is just a small community team working on this project it actively evolving. Every week new features arrive. If you find the project useful please give it a try and leave your feedback right to GitHub issues. Your feedback is critical in order to steer as many efforts as possible to the most voted features.
Mikhail_Fedoro1
Sep 09, 2021 Place Technical Articles
1.6KViews
1like
0Comments
Cloud Template for App Protect WAF
Introduction Everybody needs a WAF. However, when it gets to a deployment stage a team usually realizes that production-grade deployment going to be far more complex than a demo environment. In the case of a cloud deployment VPC networking, infrastructure security, VM images, auto-scaling, logging, visibility, automation, and many more topics require detailed analysis. Usually, it takes at least a few weeks for an average team to design and implement a production-grade WAF in a cloud. That is the one side of the problem. Additionally, cloud deployment best practices are the same for everyone, therefore most of well-made WAF deployments follow a similar path and become similar at the end. The statements above bring us to an obvious conclusion that proper WAF deployment can be templatized. So a team doesn’t spend time on deployment and maintenance but starts to use a WAF from day zero. The following paragraphs introduce a project that implements a Cloud Formation template to deploy production-grade WAF in AWS cloud just in a few clicks. Project (GitHub) On a high level, the project implements a Cloud Formation template that automatically deploys a production-grade WAF to AWS cloud. The template aims to follow cloud deployment best practices to set up a complete solution that is fully automated, requires minimum to no infrastructure management, therefore, allows a team to focus on application security. The following picture represents the overall solution structure. The solution includes a definition of three main components. Auto-scaling data plane based on official NGINX App Protect AWS AMI images. Git repository as the source of data plane and security configuration. Visibility dashboards displaying the WAF health and security data. Therefore it becomes a complete and easy-to-use solution to protect applications whether they run in AWS or in any other location. Data Plane: Data plane auto-scales based on the amount of incoming traffic and uses official NGINX App Protect AWS AMIs to spin up new VM instances. That removes the operational headache and optimizes costs since WAF dynamically adjusts the amount of computing resources and charges a user on an as-you-go basis. Configuration Plane: Solution configuration follows GitOps principles. The template creates the AWS CodeCommit git repository as a source of forwarding and security configuration. AWS CodeDeploy pipeline automatically delivers a configuration across all data plane VMs. Visibility: Alongside the data plane and configuration repository the template sets up a set of visibility dashboards in AWS CloudWatch. Data plane VMs send logs and metrics to CloudWatch service that visualizes incoming data as a set of charts and tables showing WAF health and security violations. Therefore these three components form a complete WAF solution that is easy to deploy, doesn't impose any operational headache and provides handy interfaces for WAF configuration and visibility right out of the box. Demo As mentioned above, one of the main advantages of this project is the ease of WAF deployment. It only requires downloading the AWS CloudFormation template file from the project repository and deploy it whether via AWS Console or AWS CLI. Template requests a number of parameters, however, all they are optional. As soon as stack creation is complete WAF is ready to use. Template outputs contain WAF URL and pointer to configuration repository. By default the WAF responds with static page. As a next step, I'll put this cloud WAF instance in front of a web application. Similar to any other NGINX instance, I'll configure it to forward traffic to the app and inspect all requests with App Protect WAF. As mentioned before, all config lives in a git repo that resides in the AWS CodeCommit service. I'm adjusting the NGINX configuration to forward traffic to the protected application. Once committed to the repo, a pipeline delivers the change to all data plane VMs. Therefore all traffic redirects to a protected application (screenshot below is not of a real company, and used for demo purposes only). Similar to NGINX configuration App Protect policy resides in the same repository. Similarly, all changes reflect running VMs. Once the configuration is complete, a user can observe system health and security-related data via pre-configured AWS CloudWatch dashboards. Outline As you can see, the use of a template to deploy a cloud WAF allows to significantly reduce time spent on WAF deployment and maintenance. Handy interfaces for configuration and visibility turn this project into a boxed solution allowing a user to easily operate a WAF and focus on application security. Please comment if you find useful to have this kind of solution in major public clouds marketplaces. It is a community project so far, and we need as much feedback as possible to steer one properly. Feel free to give it a try and leave feedback here or at the project's git repository. P.S.: Take a look to another community project that contributes to F5 WAF ecosystem: WAF Policy Editor
Mikhail_Fedoro1
Sep 07, 2021 Place Technical Articles
717Views
3likes
0Comments
Deploying NGINXplus with AppProtect in Tanzu Kubernetes Grid
Introduction Tanzu Kubernetes Grid (aka TKG) is VMware's main Kubernetes offering. Although Tanzu Kubernetes Grid is a certified conformant Kubernetes offering the different Kubernetes offerings can be customized in different ways. In the case of TKG a remarkable feature is the use of Pod Security Policies by default. TKG clusters are very easily spin-up in either public or private clouds by means of creating a single declaration YAML file such as the following: apiVersion: run.tanzu.vmware.com/v1alpha1 kind: TanzuKubernetesCluster metadata: name: tkg1 namespace: tkg1 spec: distribution: version: v1.18.15+vmware.1-tkg.1.600e412 topology: controlPlane: count: 1 class: best-effort-medium storageClass: vsan-default-storage-policy workers: count: 1 class: best-effort-medium storageClass: vsan-default-storage-policy As you can see from the schema a TKG cluster is deployed just as another Kubernetes resource. How does it work? In the case of public clouds, these TanzuKubernetesCluster resources are instantiated from a bootstrap cluster named "management Kubernetes cluster" whilst when using vSphere with Tanzu, the TanzuKubernetesCluster resources are instantiated from vSphere with Tanzu's supervisor cluster. In this blog post it will be shown an example from start to end: Creating wildcard certificate from custom CA with easy-rsa. Enabling Harbor registry. Installing NGINXplus with AppProtect. Using NGINXplus Ingress Controller without AppProtect. Adding AppProtect to an Ingress resource. AppProtect is an NGINXplus module for WAF and Bot protection based on market leading F5 BIG-IP's ASM. AppProtect provides enhanced capabilities and performance for those who require more than what mod_auth provides. We will finish with two relevant considerations: Updating NGINXplus Ingress controller using Helm. This is used for example for scaling-out NGINXplus hence improving the overall performance. Using NGINXplus alongside with other Ingress Controllers (such as Contour). Prerequisites You need an NGINXplus license which can be retrieved from https://www.nginx.com/free-trial-request-nginx-ingress-controller/. This license is in practice a cert/key pair with the file names nginx-repo.{crt,key} referenced later on. The following software needs to be present in your in your machine: Docker v18.09+ GNU Make git Helm3 OpenSSL https://github.com/OpenVPN/easy-rsa.git Create a wildcard certificate with easy-rsa In the next steps it will be created a Certificate Authority (CA) and from it a wildcard certificate/key pair which will be loaded into Kubernetes as a TLS secret. This wildcard certificate will be used by all the services which will expose through Ingress. Retrieve easy-rsa and initialize a CA (output summarized): $ git clone https://github.com/OpenVPN/easy-rsa.git $ cd easyrsa3/ $ ./easyrsa init-pki $ ./easyrsa build-ca Generate the wildcard key/cert pair (output summarized): $ ./easyrsa gen-req wildcard Common Name (eg: your user, host, or server name) [wildcard]:*.tkg.bd.f5.com Keypair and certificate request completed. Your files are: req: /Users/alonsocamaro/Documents/VMware-Tanzu/tanzu/easy-rsa/easyrsa3/pki/reqs/wildcard.req key: /Users/alonsocamaro/Documents/VMware-Tanzu/tanzu/easy-rsa/easyrsa3/pki/private/wildcard.key $ ./easyrsa sign-req server wildcard Request subject, to be signed as a server certificate for 825 days: subject= commonName = *.tkg.bd.f5.com The Subject's Distinguished Name is as follows commonName :ASN.1 12:'*.tkg.bd.f5.com' Certificate is to be certified until Aug 21 15:58:24 2023 GMT (825 days) Write out database with 1 new entries Data Base Updated Certificate created at: /Users/alonsocamaro/Documents/VMware-Tanzu/tanzu/easy-rsa/easyrsa3/pki/issued/wildcard.crt The certificate is stored in ./pki/issued/wildcard.crt and the key is stored encrypted in pki/private/wildcard.key. Import these into a Kubernetes secret using the next steps: $ openssl rsa -in ./pki/private/wildcard.key -out ./pki/private/wildcard-unencrypted.key Enter pass phrase for ./pki/private/wildcard.key: writing RSA key $ kubectl create ns ingress-nginx $ kubectl create -n ingress-nginx secret tls wildcard-tls --key ./pki/private/wildcard-unencrypted.key --cert ./pki/issued/wildcard.crt secret/wildcard-tls created $ rm ./pki/private/wildcard-unencrypted.key As you might have noticed the secret is loaded in the namespace ingress-nginx where NGINXplus Ingress Controller will be installed. Enable your image registry in Tanzu You need an image registry. When using vSphere with Tanzu this comes with Harbor. In this case you have to follow the next steps: Enable Harbor by following https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-AE24CF79-3C74-4CCD-B7C7-757AD082D86A.html. Trust Harbor's self-signed certificate: In the case of using vSphere with Tanzu Update 2 (U2) follow https://tanzu.vmware.com/content/blog/how-to-set-up-harbor-registry-self-signed-certificates-tanzu-kubernetes-clusters. If using a previous version follow https://cormachogan.com/2020/06/23/integrating-embedded-vsphere-with-kubernetes-harbor-registry-with-tkg-guest-clusters/ with the caveat that you will need to re-apply the changes if you upgrade the TKG cluster. Alternatively, you could also use your own certificates and follow https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.3/vmware-tanzu-kubernetes-grid-13/GUID-cluster-lifecycle-secrets.html#trust-custom-ca-certificates-on-cluster-nodes-3. Installing NGINXplus Ingress Controller This blog shows step by step everything that needs to be done to create an AppProtect-secured Ingress Controller with a wildcard certificate that will created as well. This blog only assumes that the TKG cluster is up and running. If you want to perform further customizations you can check https://docs.nginx.com/nginx-ingress-controller and https://docs.nginx.com/nginx-app-protect/configuration. In this blog it has been used TKG 1.3 in vSphere with Tanzu with NSX-T. The steps are similar when using any other supported TKG environment. Build the NGINXplus DOCKER image Define the registry endpoint and namespace where the TKG cluster will be deployed: $ REGISTRY=<registry IP or FQDN> $ NS=<your namespace> Log in the registry $ docker login $REGISTRY Username: <your user> Password: <your password> Login Succeeded Retrieve NGINXplus $ git clone https://github.com/nginxinc/kubernetes-ingress/ $ cd kubernetes-ingress $ git checkout v1.11.1 Copy the license files into base folder of NGINXplus $ cp $LICDIR/nginx-repo.{crt,key} . Build the image $ make debian-image-nap-plus PREFIX=$REGISTRY/$NS/nginx-plus-ingress TARGET=container Docker version 19.03.8, build afacb8b docker build --build-arg IC_VERSION=1.11.1-32745366 --build-arg GIT_COMMIT=32745366 --build-arg VERSION=1.11.1 --target container -f build/Dockerfile -t 10.105.210.67/tkg1/nginx-plus-ingress:1.11.1 . --build-arg BUILD_OS=debian-plus-ap --build-arg PLUS=-plus --secret id=nginx-repo.crt,src=nginx-repo.crt --secret id=nginx-repo.key,src=nginx-repo.key [+] Building 4.9s (24/24) FINISHED After this we can verify the image is ready in our local docker: $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE $REGISTRY/$NS/nginx-plus-ingress 1.11.1 70113ec38914 35 minutes ago 626MB If we wanted to push it into another namespace we would perform an image tag operation as follows: docker image tag 70113ec38914 $REGISTRY/$ANOTHERNS/nginx-plus-ingress:1.11.1 Upload the image into the repository: make push PREFIX=$REGISTRY/$NS/nginx-plus-ingress Configure NGINXplus installation Switch to the helm chart folder cd deployments/helm-chart Make a backup of the default nginx-plus config file cp values-plus.yaml values-plus.yaml.orig We will edit the file values-plus.yaml as follows in order to: Enable AppProtect Allow to specify a wildcard TLS certificate that we will use for all the services. Expose NGINXplus using a LoadBalancer with a Cluster externalTrafficPolicy. Exposing NGINXplus (or any other Ingress Controller such as Contour) using Cluster externalTrafficPolicy is required given that the NSX-T native load balancer doesn't perform any health checking when creating a Service of Type LoadBalancer. We will see how to improve this in future blogs with the use of BIG-IP. controller: nginxplus: true image: repository: nginx-plus-ingress tag: "1.11.1" service: externalTrafficPolicy: Cluster appprotect: ## Enable the App Protect module in the Ingress Controller. enable: true wildcardTLS: ## The base64-encoded TLS certificate for every Ingress host that has TLS enabled but no secret specified. ## If the parameter is not set, for such Ingress hosts NGINX will break any attempt to establish a TLS connection. cert: "" ## The base64-encoded TLS key for every Ingress host that has TLS enabled but no secret specified. ## If the parameter is not set, for such Ingress hosts NGINX will break any attempt to establish a TLS connection. key: "" ## The secret with a TLS certificate and key for every Ingress host that has TLS enabled but no secret specified. ## The value must follow the following format: `<namespace>/<name>`. ## Used as an alternative to specifying a certificate and key using `controller.wildcardTLS.cert` and `controller.wildcardTLS.key` parameters. ## Format: <namespace>/<secret_name> secret: ingress-nginx/wildcard-tls This file can also be found in https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/values-plus.yaml Apply the required PodSecurityPolicy before NGINXplus installation The next step creates a PodSecurityPolicy which is required by Tanzu Kubernetes Grid and it is bound to the Service Account ingress-nginx used in the regular NGINXplus install. $ kubectl apply -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/nginx-psp.yaml podsecuritypolicy.policy/ingress-nginx created clusterrole.rbac.authorization.k8s.io/ingress-nginx-psp created clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-psp created Install NGINXplus Ingress controller using Helm From the deployments/helm-chart directory of the downloaded NGINXplus, it is just needed to run the next command: $ helm -n ingress-nginx install ingress-nginx -f values-plus.yaml . NAME: ingress-nginx LAST DEPLOYED: Mon May 17 15:16:20 2021 NAMESPACE: ingress-nginx STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: The NGINX Ingress Controller has been installed. Checking the resulting installation When checking the resulting resources we can see that by default a single POD is created. We can scale up/down this as required by using Helm. This will be shown later on. Note also that by default the NGINXplus Ingress Controller is automatically exposed using the Service Type LoadBalancer resource which configures an external load balancer. In this case the external load balancer is NSX-T's native LB as shown in the screenshot. below When using vSphere networking this would have been HAproxy by default. In next blogs we will show how to use F5 BIG-IP in TKG clusters instead. $ kubectl -n ingress-nginx get all NAME READY STATUS RESTARTS AGE pod/ingress-nginx-nginx-ingress-7d4587b44c-n9b8l 1/1 Running 0 16h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/ingress-nginx-nginx-ingress LoadBalancer 100.69.127.66 10.105.210.68 80:30527/TCP,443:31185/TCP 16h NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/ingress-nginx-nginx-ingress 1/1 1 1 16h NAME DESIRED CURRENT READY AGE replicaset.apps/ingress-nginx-nginx-ingress-7d4587b44c 1 1 1 16h In the next screenshots we can see the resulting configuration in NSX-T: Both pools for port 80 and port 443 point to the worker's node addresses. This means that the traffic flow will be NSX-T LB -> ClusterIP -> Ingress Controller's POD address (in the same or in another node). This is the case of any regular Ingress Controller including TKG's provided Contour. In next blogs it will be shown how these many layers of indirection can be bypassed using F5 BIG-IP. Using NGINXplus Ingress Controller without AppProtect Creating a regular Ingress resource In this initial example we will create two services (coffee and tea) which will be exposed with an Ingress resource called cafe-ingress. This will expose the services in the URL https://cafe.tkg.bd.f5.com/coffee and https://cafe.tkg.bd.f5.com/tea using the previously created wildcard certificate for *.tkg.bd.f5.com as depicted in the next diagram. To create this setup run the following commands: $ kubectl create ns test $ kubectl -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/cafe-rbac.yaml $ kubectl -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/cafe.yaml $ kubectl -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/cafe-ingress.yaml This is the same example provided in the official NGINXplus documentation but we add the cafe-rbac.yaml declaration which creates the necessary PodSecurity policies and bindings for TKG. To verify the result first we will check the Ingress resource itself: $ kubectl -n test get ingress NAME CLASS HOSTS ADDRESS PORTS AGE cafe-ingress nginx cafe.tkg.bd.f5.com 10.105.210.68 80, 443 3m44s where we can observe that the IP address is the one of the external loadbalancer seen before. To verify it is all working as expected we will use curl as follows: $ curl --cacert ca-tkg.bd.f5.com.crt --resolve cafe.tkg.bd.f5.com:443:10.105.210.68 https://cafe.tkg.bd.f5.com/coffee Server address: 100.96.1.16:8080 Server name: coffee-86954d94fd-pnvpq Date: 18/May/2021:16:18:59 +0000 URI: /coffee Request ID: 63964930a2d1038af5f204ef8fbe91fc which has the following key parameters: Use --cacert to specify our CA crt file previously created Use --resolve to allow curl resolve the FQDN of the request Adding AppProtect to an Ingress resource Additional configuration Our deployed NGINXplus has AppProtect built-in. It is up to the user of the Ingress resource if it wants to enable it, on a per Ingress basis. In our example we will apply the AppProtect security policies in the user namespace "test". We will also create a syslog store in the ingress-nginx namespace. All these can be customized. Ultimately, the user just needs to add the following annotations in order to secure the cafe site: annotations: appprotect.f5.com/app-protect-policy: "test/dataguard-alarm" appprotect.f5.com/app-protect-enable: "True" appprotect.f5.com/app-protect-security-log-enable: "True" appprotect.f5.com/app-protect-security-log: "test/logconf" appprotect.f5.com/app-protect-security-log-destination: "syslog:server=100.70.175.24:514" The custom AppProtect policy used in this example contains DataGuard protection for Credit Card Number, US Social Security number leaks and a custom signature. It is also defined where to send the AppProtect logs. These are sent in SYSLOG/TCP mode independently of the regular logs generated by NGINXplus. To make all these happen first we will create the syslog server: kubectl apply -n ingress-nginx -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/syslog-rbac.yaml kubectl apply -n ingress-nginx -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/syslog.yaml Next, we will create the AppProtect policies: kubectl apply -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/ap-apple-uds.yaml kubectl apply -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/ap-dataguard-alarm-policy.yaml kubectl apply -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/ap-logconf.yaml Finally we will add the above annotations to the Ingress resource. For that, we will need to get SYSLOG's POD address and replace it in the cafe-ingress-ap.yaml definition. curl -O https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/cafe-ingress-ap.yaml SYSLOG_IP=<IP address of syslog's POD> sed -e "s/SYSLOG/$SYSLOG_IP/" cafe-ingress-ap.yaml > cafe-ingress-ap-syslog.yaml kubectl apply -n test -f cafe-ingress-ap-syslog.yaml Note: it might take few seconds to make the AppProtect configuration effective. Verifying AppProtect Run the following command to watch the requests live as handled by AppProtect: kubectl -n ingress-nginx exec -it <SYSLOG POD NAME> -- tail -f /var/log/messages Send a request that triggers the custom signature: curl --cacert ca-tkg.bd.f5.com.crt --resolve cafe.tkg.bd.f5.com:443:10.105.210.69 "https://cafe.tkg.bd.f5.com/coffee/" -X POST -d "apple" You should see a log similar to the following one in the syslog logs: May 24 13:43:23 ingress-nginx-nginx-ingress-7d4587b44c-wvrxs ASM:attack_type="Non-browser Client,Brute Force Attack",blocking_exception_reason="N/A",date_time="2021-05-24 13:43:23",dest_port="443",ip_client="10.105.210.69",is_truncated="false",method="POST",policy_name="dataguard-alarm",protocol="HTTPS",request_status="blocked",response_code="0",severity="Critical",sig_cves="N/A",sig_ids="300000000",sig_names="Apple_medium_acc [Fruits]",sig_set_names="{apple_sigs}",src_port="4096",sub_violations="N/A",support_id="15704273797572010868",threat_campaign_names="N/A",unit_hostname="ingress-nginx-nginx-ingress-7d4587b44c-wvrxs",uri="/coffee/",violation_rating="3",vs_name="24-cafe.tkg.bd.f5.com:9-/coffee",x_forwarded_for_header_value="N/A",outcome="REJECTED",outcome_reason="SECURITY_WAF_VIOLATION",violations="Attack signature detected,Bot Client Detected",violation_details="<?xml version='1.0' encoding='UTF-8'?><BAD_MSG><violation_masks><block>10000000200c00-3030430000070</block><alarm>2477f0ffcbbd0fea-8003f35cb000007c</alarm><learn>200000-20</learn><staging>0-0</staging></violation_masks><request-violations><violation><viol_index>42</viol_index><viol_name>VIOL_ATTACK_SIGNATURE</viol_name><context>request</context><sig_data><sig_id>300000000</sig_id><blocking_mask>7</blocking_mask><kw_data><buffer>YXBwbGU=</buffer><offset>0</offset><length>5</length></kw_data></sig_data></violation></request-violations></BAD_MSG>",bot_signature_name="curl",bot_category="HTTP Library",bot_anomalies="N/A",enforced_bot_anomalies="N/A",client_class="Untrusted Bot",request="POST /coffee/ HTTP/1.1\r\nHost: cafe.tkg.bd.f5.com\r\nUser-Agent: curl/7.64.1\r\nAccept: */*\r\nContent-Length: 5\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\napple" Updating NGINXplus Ingress controller using Helm By default a single NGINXplus instance is created, if you want to increase the performance of it, scaling-out is as simple as editing the values-plus.yaml file and setting a replicaCount parameter with the desired value: controller: replicaCount: 4 And running helm upgrade as follows $ helm -n ingress-nginx upgrade ingress-nginx -f values-plus.yaml . Release "ingress-nginx" has been upgraded. Happy Helming! NAME: ingress-nginx LAST DEPLOYED: Thu May 20 14:20:38 2021 NAMESPACE: ingress-nginx STATUS: deployed REVISION: 2 TEST SUITE: None NOTES: The NGINX Ingress Controller has been installed. Using NGINXplus alongside with other Ingress Controllers (such as Contour). NGINXplus does support Ingress/v1 resource version available in Kubernetes 1.18+ as well as previous Ingress/v1beta1 API resource version for backwards compatibility. Contour Ingress Controller is provided in TKG by VMware as an add-on which is not installed by default. If installed, you have to be aware that Contour, at time of this writting (May 2021), only supports the older Ingress/v1beta1 API resource version. This means that when defining Ingress resources you have to specify the Ingress Controller to use by means of adding the following annotation: kubernetes.io/ingress.class: <ingress conroller name> where <ingress controller name> could be nginx or contour. For further details on this topic you can check https://docs.nginx.com/nginx-ingress-controller/installation/running-multiple-ingress-controllers/ Conclusion In this blog post we have gone through all the steps required to install and use NGINXplus with AppProtect in Tanzu Kubernetes Grid with a real world example. Overall, the installation is the same as in any Kubernetes but the following two items need to be taken into account: Before deploying, make sure that the appropriate PodSecurityPolicies are in place for either NGINXplus or the workloads. PodSecurityPolicies are not enabled by default in many Kubernetes distributions so this represents a change from the usual practice. If deploying NGINXplus alongside another Ingress Controller make sure that the Ingress resources are defined appropriately in order to select the right Ingress Controller for the corresponding Ingress resource. In this blog we used NGINXplus in a TKG cluster deployed in an on-premises infrastructure (vSphere with Tanzu) with the Antrea CNI and NSX-T networking. The steps would have been the same if it had been used vSphere networking or Calico CNI. The only difference could come when exposing it through the external load balancer. If the external load balancer performed health checking it would be preferable to use local externalTrafficPolicy since this avoids a hop and allows keeping the client source address. In future blogs we will post how to expose NGINXplus more effectively and in a cloud-agnostic manner by using BIG-IP as external load balancer.
Ulises_Alonso
Jun 02, 2021 Place Technical Articles
1.3KViews
1like
0Comments
Mitigating New Gadget Leveraging JNDI Injection into Remote Code Execution Using Advanced WAF
Recently a new gadget that bypass existing Java restrictions for JNDI injection was published via Tweet made by a security researcher identified by the twitter handle @PewGrand. JNDI – Java Naming and Directory Interface is a Java API that allows the application developer to retrieve objects based on a given name. JNDI supports different implementations like Remote Method Invocation (RMI), Lightweight Directory Access Protocol (LDAP) and others. The JNDI Injection vulnerability class was first presented in Black Hat 2016 by Alvaro Munoz and Oleksandr Mirosh who demonstrated the different ways where applications utilizing JNDI can be exploited into Remote Code Execution. Applications are vulnerable to JNDI Injections when user-controlled input is being passed as argument to methods like “lookup” of the “InitialContext” Java class, which is part of the Java Naming API. This method is responsible for resolving a name it received into an object identified by the supplied name. Figure 1: Example of vulnerable endpoint in a Spring application The name that the “lookup” method receives can also be in the form of LDAP or RMI URLs and therefore vulnerable applications may be forced to fetch a malicious object from attacker controlled RMI or LDAP server. In the last few years Oracle has applied several restrictions that aimed to prevent attackers from exploiting JNDI Injection vulnerabilities. One example of such restriction is the “trustURLCodebase” property which was introduced in Java Development Kit 8 – Update 121. This property prevents vulnerable applications from loading malicious objects from remote RMI repositories. Later, similar restriction was added also to cover LDAP repositories. Since those restrictions were added, exploiting JNDI Injection vulnerabilities now depends on existing gadgets, which means the classes used in the exploit must reside in the vulnerable application class path for the exploit to work. The new gadget discovered by @PewGrand uses similar approach as another previously disclosed gadget. They both start a rogue LDAP server that hosts an LDAP entry which specifies the “javaClassName” and “javaSerializedData” attributes. Those attributes can be used by Java LDAP implementations to reference Java objects in LDAP entries. Once the vulnerable application will fetch this entry from the attacker-controlled LDAP server the object referenced by the “javaSerializedData” attribute will be deserialized. Another point of similarity to the previously disclosed gadget is that both gadgets specify the BeanFactory class of Apache Tomcat as the object factory of the serialized object stored in the LDAP entry. The factory class is responsible for creating an instance of the class specified in the LDAP entry. The reason that the BeanFactory is a good pick for both gadgets is because it allows the rogue LDAP server to also specify a method that receives a string as a parameter which should be invoked after creating the instance from the specified class. This is achieved by specifying the “forceString” JNDI Reference. This is also where the difference between the two gadgets relies - the previous gadget was pointing to the “ELProcessor” class and specified the “eval” method as the “forceString” JNDI Reference, which allowed the gadget to execute arbitrary Java EL Expression template and thus achieve arbitrary code execution. Figure 2: Previous gadget creating instance of the ELProcessor class and invoking its “eval” method. The newly discovered gadget specifies the SnakeYAML Yaml class as the one that should be instantiated by Tomcat BeanFactory, and specifies the “load” method of the Yaml class in order to make the application parse a Yaml that loads JAR file from a remote server by using the Java URLClassLoader. Figure 3: New gadget creating instance of the SnakeYaml Yaml class and invoking the load method. Figure 4: Exploiting the vulnerable application using the new gadget. Mitigating JNDI Injections with Advanced WAF Advanced WAF customers under any supported version are now protected against this vulnerability. The exploitation attempt will be detected by newly released Java code injection attack signatures which can be found in signature sets that include the “Server Side Code Injection” attack type. Figure 5: Exploit attempt blocked by signature id 200104722. Additional resources https://twitter.com/steventseeley/status/1380065046458433536 https://gist.github.com/TheGrandPew/748ac740698511975eaeed5d77ecb2d9 https://www.blackhat.com/docs/us-16/materials/us-16-Munoz-A-Journey-From-JNDI-LDAP-Manipulation-To-RCE-wp.pdf https://www.veracode.com/blog/research/exploiting-jndi-injections-java
Gal_Goldshtein
Apr 14, 2021 Place Technical Articles
1.5KViews
0likes
0Comments

"}},"componentScriptGroups({\"componentId\":\"custom.widget.Beta_MetaNav\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"component({\"componentId\":\"custom.widget.Beta_Footer\"})":{"__typename":"Component","render({\"context\":{\"component\":{\"entities\":[],\"props\":{}},\"page\":{\"entities\":[],\"name\":\"TagPage\",\"props\":{},\"url\":\"https://community.f5.com/tag/NGINX%20WAF\"}}})":{"__typename":"ComponentRenderResult","html":"

Trademarks Policies Privacy California Privacy Do Not Sell My Personal Information

"}},"componentScriptGroups({\"componentId\":\"custom.widget.Beta_Footer\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"component({\"componentId\":\"custom.widget.Tag_Manager_Helper\"})":{"__typename":"Component","render({\"context\":{\"component\":{\"entities\":[],\"props\":{}},\"page\":{\"entities\":[],\"name\":\"TagPage\",\"props\":{},\"url\":\"https://community.f5.com/tag/NGINX%20WAF\"}}})":{"__typename":"ComponentRenderResult","html":" "}},"componentScriptGroups({\"componentId\":\"custom.widget.Tag_Manager_Helper\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"component({\"componentId\":\"custom.widget.Consent_Blackbar\"})":{"__typename":"Component","render({\"context\":{\"component\":{\"entities\":[],\"props\":{}},\"page\":{\"entities\":[],\"name\":\"TagPage\",\"props\":{},\"url\":\"https://community.f5.com/tag/NGINX%20WAF\"}}})":{"__typename":"ComponentRenderResult","html":"

"}},"componentScriptGroups({\"componentId\":\"custom.widget.Consent_Blackbar\"})":{"__typename":"ComponentScriptGroups","scriptGroups":{"__typename":"ComponentScriptGroupsDefinition","afterInteractive":{"__typename":"PageScriptGroupDefinition","group":"AFTER_INTERACTIVE","scriptIds":[]},"lazyOnLoad":{"__typename":"PageScriptGroupDefinition","group":"LAZY_ON_LOAD","scriptIds":[]}},"componentScripts":[]},"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/community/NavbarDropdownToggle\"]})":[{"__ref":"CachedAsset:text:en_US-components/community/NavbarDropdownToggle-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/common/OverflowNav\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/common/OverflowNav-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageView/MessageViewInline\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageView/MessageViewInline-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/common/Pager/PagerLoadMore\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/common/Pager/PagerLoadMore-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/customComponent/CustomComponent\"]})":[{"__ref":"CachedAsset:text:en_US-components/customComponent/CustomComponent-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/users/UserLink\"]})":[{"__ref":"CachedAsset:text:en_US-components/users/UserLink-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageSubject\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageSubject-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageTime\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageTime-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/nodes/NodeIcon\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/nodes/NodeIcon-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageUnreadCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageUnreadCount-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageViewCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageViewCount-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/kudos/KudosCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/kudos/KudosCount-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageRepliesCount\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageRepliesCount-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"components/messages/MessageBody\"]})":[{"__ref":"CachedAsset:text:en_US-components/messages/MessageBody-1743097583694"}],"cachedText({\"lastModified\":\"1743097583694\",\"locale\":\"en-US\",\"namespaces\":[\"shared/client/components/users/UserAvatar\"]})":[{"__ref":"CachedAsset:text:en_US-shared/client/components/users/UserAvatar-1743097583694"}]},"CachedAsset:pages-1742462926377":{"__typename":"CachedAsset","id":"pages-1742462926377","value":[{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetInvolved.MvpProgram","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved/mvp-program","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"BlogViewAllPostsPage","type":"BLOG","urlPath":"/category/:categoryId/blog/:boardId/all-posts/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"CasePortalPage","type":"CASE_PORTAL","urlPath":"/caseportal","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"CreateGroupHubPage","type":"GROUP_HUB","urlPath":"/groups/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"CaseViewPage","type":"CASE_DETAILS","urlPath":"/case/:caseId/:caseNumber","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"InboxPage","type":"COMMUNITY","urlPath":"/inbox","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetInvolved.AdvocacyProgram","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved/advocacy-program","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetHelp.NonCustomer","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/non-customer","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HelpFAQPage","type":"COMMUNITY","urlPath":"/help","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetHelp.F5Customer","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/f5-customer","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"IdeaMessagePage","type":"IDEA_POST","urlPath":"/idea/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"IdeaViewAllIdeasPage","type":"IDEA","urlPath":"/category/:categoryId/ideas/:boardId/all-ideas/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"LoginPage","type":"USER","urlPath":"/signin","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"BlogPostPage","type":"BLOG","urlPath":"/category/:categoryId/blogs/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetInvolved","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.Learn","type":"COMMUNITY","urlPath":"/c/how-do-i/learn","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1739501996000,"localOverride":null,"page":{"id":"Test","type":"CUSTOM","urlPath":"/custom-test-2","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ThemeEditorPage","type":"COMMUNITY","urlPath":"/designer/themes","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"TkbViewAllArticlesPage","type":"TKB","urlPath":"/category/:categoryId/kb/:boardId/all-articles/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"OccasionEditPage","type":"EVENT","urlPath":"/event/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"OAuthAuthorizationAllowPage","type":"USER","urlPath":"/auth/authorize/allow","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"PageEditorPage","type":"COMMUNITY","urlPath":"/designer/pages","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"PostPage","type":"COMMUNITY","urlPath":"/category/:categoryId/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ForumBoardPage","type":"FORUM","urlPath":"/category/:categoryId/discussions/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"TkbBoardPage","type":"TKB","urlPath":"/category/:categoryId/kb/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"EventPostPage","type":"EVENT","urlPath":"/category/:categoryId/events/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"UserBadgesPage","type":"COMMUNITY","urlPath":"/users/:login/:userId/badges","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"GroupHubMembershipAction","type":"GROUP_HUB","urlPath":"/membership/join/:nodeId/:membershipType","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"MaintenancePage","type":"COMMUNITY","urlPath":"/maintenance","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"IdeaReplyPage","type":"IDEA_REPLY","urlPath":"/idea/:boardId/:messageSubject/:messageId/comments/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"UserSettingsPage","type":"USER","urlPath":"/mysettings/:userSettingsTab","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"GroupHubsPage","type":"GROUP_HUB","urlPath":"/groups","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ForumPostPage","type":"FORUM","urlPath":"/category/:categoryId/discussions/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"OccasionRsvpActionPage","type":"OCCASION","urlPath":"/event/:boardId/:messageSubject/:messageId/rsvp/:responseType","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"VerifyUserEmailPage","type":"USER","urlPath":"/verifyemail/:userId/:verifyEmailToken","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"AllOccasionsPage","type":"OCCASION","urlPath":"/category/:categoryId/events/:boardId/all-events/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"EventBoardPage","type":"EVENT","urlPath":"/category/:categoryId/events/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"TkbReplyPage","type":"TKB_REPLY","urlPath":"/kb/:boardId/:messageSubject/:messageId/comments/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"IdeaBoardPage","type":"IDEA","urlPath":"/category/:categoryId/ideas/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"CommunityGuideLinesPage","type":"COMMUNITY","urlPath":"/communityguidelines","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"CaseCreatePage","type":"SALESFORCE_CASE_CREATION","urlPath":"/caseportal/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"TkbEditPage","type":"TKB","urlPath":"/kb/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ForgotPasswordPage","type":"USER","urlPath":"/forgotpassword","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"IdeaEditPage","type":"IDEA","urlPath":"/idea/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"TagPage","type":"COMMUNITY","urlPath":"/tag/:tagName","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"BlogBoardPage","type":"BLOG","urlPath":"/category/:categoryId/blog/:boardId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"OccasionMessagePage","type":"OCCASION_TOPIC","urlPath":"/event/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ManageContentPage","type":"COMMUNITY","urlPath":"/managecontent","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ClosedMembershipNodeNonMembersPage","type":"GROUP_HUB","urlPath":"/closedgroup/:groupHubId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetHelp.Community","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/community","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"CommunityPage","type":"COMMUNITY","urlPath":"/","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetInvolved.ContributeCode","type":"COMMUNITY","urlPath":"/c/how-do-i/get-involved/contribute-code","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ForumMessagePage","type":"FORUM_TOPIC","urlPath":"/discussions/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"IdeaPostPage","type":"IDEA","urlPath":"/category/:categoryId/ideas/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"BlogMessagePage","type":"BLOG_ARTICLE","urlPath":"/blog/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"RegistrationPage","type":"USER","urlPath":"/register","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"EditGroupHubPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ForumEditPage","type":"FORUM","urlPath":"/discussions/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ResetPasswordPage","type":"USER","urlPath":"/resetpassword/:userId/:resetPasswordToken","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"TkbMessagePage","type":"TKB_ARTICLE","urlPath":"/kb/:boardId/:messageSubject/:messageId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.Learn.AboutIrules","type":"COMMUNITY","urlPath":"/c/how-do-i/learn/about-irules","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"BlogEditPage","type":"BLOG","urlPath":"/blog/:boardId/:messageSubject/:messageId/edit","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetHelp.F5Support","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/f5-support","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ManageUsersPage","type":"USER","urlPath":"/users/manage/:tab?/:manageUsersTab?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ForumReplyPage","type":"FORUM_REPLY","urlPath":"/discussions/:boardId/:messageSubject/:messageId/replies/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"PrivacyPolicyPage","type":"COMMUNITY","urlPath":"/privacypolicy","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"NotificationPage","type":"COMMUNITY","urlPath":"/notifications","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"UserPage","type":"USER","urlPath":"/users/:login/:userId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HealthCheckPage","type":"COMMUNITY","urlPath":"/health","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"OccasionReplyPage","type":"OCCASION_REPLY","urlPath":"/event/:boardId/:messageSubject/:messageId/comments/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ManageMembersPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId/manage/:tab?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"SearchResultsPage","type":"COMMUNITY","urlPath":"/search","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"BlogReplyPage","type":"BLOG_REPLY","urlPath":"/blog/:boardId/:messageSubject/:messageId/replies/:replyId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"GroupHubPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"TermsOfServicePage","type":"COMMUNITY","urlPath":"/termsofservice","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetHelp","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI.GetHelp.SecurityIncident","type":"COMMUNITY","urlPath":"/c/how-do-i/get-help/security-incident","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"CategoryPage","type":"CATEGORY","urlPath":"/category/:categoryId","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"ForumViewAllTopicsPage","type":"FORUM","urlPath":"/category/:categoryId/discussions/:boardId/all-topics/(/:after|/:before)?","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"TkbPostPage","type":"TKB","urlPath":"/category/:categoryId/kbs/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"GroupHubPostPage","type":"GROUP_HUB","urlPath":"/group/:groupHubId/:boardId/create","__typename":"PageDescriptor"},"__typename":"PageResource"},{"lastUpdatedTime":1742462926377,"localOverride":null,"page":{"id":"HowDoI","type":"COMMUNITY","urlPath":"/c/how-do-i","__typename":"PageDescriptor"},"__typename":"PageResource"}],"localOverride":false},"CachedAsset:text:en_US-components/context/AppContext/AppContextProvider-0":{"__typename":"CachedAsset","id":"text:en_US-components/context/AppContext/AppContextProvider-0","value":{"noCommunity":"Cannot find community","noUser":"Cannot find current user","noNode":"Cannot find node with id {nodeId}","noMessage":"Cannot find message with id {messageId}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/Loading/LoadingDot-0":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/Loading/LoadingDot-0","value":{"title":"Loading..."},"localOverride":false},"User:user:-1":{"__typename":"User","id":"user:-1","uid":-1,"login":"Former Member","email":"","avatar":null,"rank":null,"kudosWeight":1,"registrationData":{"__typename":"RegistrationData","status":"ANONYMOUS","registrationTime":null,"confirmEmailStatus":false,"registrationAccessLevel":"VIEW","ssoRegistrationFields":[]},"ssoId":null,"profileSettings":{"__typename":"ProfileSettings","dateDisplayStyle":{"__typename":"InheritableStringSettingWithPossibleValues","key":"layout.friendly_dates_enabled","value":"false","localValue":"true","possibleValues":["true","false"]},"dateDisplayFormat":{"__typename":"InheritableStringSetting","key":"layout.format_pattern_date","value":"dd-MMM-yyyy","localValue":"MM-dd-yyyy"},"language":{"__typename":"InheritableStringSettingWithPossibleValues","key":"profile.language","value":"en-US","localValue":null,"possibleValues":["en-US"]}},"deleted":false},"Theme:customTheme1":{"__typename":"Theme","id":"customTheme1"},"CachedAsset:theme:customTheme1-1742462925945":{"__typename":"CachedAsset","id":"theme:customTheme1-1742462925945","value":{"id":"customTheme1","animation":{"fast":"150ms","normal":"250ms","slow":"500ms","slowest":"750ms","function":"cubic-bezier(0.07, 0.91, 0.51, 1)","__typename":"AnimationThemeSettings"},"avatar":{"borderRadius":"50%","collections":["custom"],"__typename":"AvatarThemeSettings"},"basics":{"browserIcon":{"imageAssetName":"JimmyPackets-512-1702592938213.png","imageLastModified":"1702592945815","__typename":"ThemeAsset"},"customerLogo":{"imageAssetName":"f5_logo_fix-1704824537976.svg","imageLastModified":"1704824540697","__typename":"ThemeAsset"},"maximumWidthOfPageContent":"1600px","oneColumnNarrowWidth":"800px","gridGutterWidthMd":"30px","gridGutterWidthXs":"10px","pageWidthStyle":"WIDTH_OF_PAGE_CONTENT","__typename":"BasicsThemeSettings"},"buttons":{"borderRadiusSm":"5px","borderRadius":"5px","borderRadiusLg":"5px","paddingY":"5px","paddingYLg":"7px","paddingYHero":"var(--lia-bs-btn-padding-y-lg)","paddingX":"12px","paddingXLg":"14px","paddingXHero":"42px","fontStyle":"NORMAL","fontWeight":"400","textTransform":"NONE","disabledOpacity":0.5,"primaryTextColor":"var(--lia-bs-white)","primaryTextHoverColor":"var(--lia-bs-white)","primaryTextActiveColor":"var(--lia-bs-white)","primaryBgColor":"var(--lia-bs-primary)","primaryBgHoverColor":"hsl(var(--lia-bs-primary-h), var(--lia-bs-primary-s), calc(var(--lia-bs-primary-l) * 0.85))","primaryBgActiveColor":"hsl(var(--lia-bs-primary-h), var(--lia-bs-primary-s), calc(var(--lia-bs-primary-l) * 0.7))","primaryBorder":"1px solid transparent","primaryBorderHover":"1px solid transparent","primaryBorderActive":"1px solid transparent","primaryBorderFocus":"1px solid var(--lia-bs-white)","primaryBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","secondaryTextColor":"var(--lia-bs-gray-900)","secondaryTextHoverColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.95))","secondaryTextActiveColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.9))","secondaryBgColor":"var(--lia-bs-gray-400)","secondaryBgHoverColor":"hsl(var(--lia-bs-gray-400-h), var(--lia-bs-gray-400-s), calc(var(--lia-bs-gray-400-l) * 0.96))","secondaryBgActiveColor":"hsl(var(--lia-bs-gray-400-h), var(--lia-bs-gray-400-s), calc(var(--lia-bs-gray-400-l) * 0.92))","secondaryBorder":"1px solid transparent","secondaryBorderHover":"1px solid transparent","secondaryBorderActive":"1px solid transparent","secondaryBorderFocus":"1px solid transparent","secondaryBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","tertiaryTextColor":"var(--lia-bs-gray-900)","tertiaryTextHoverColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.95))","tertiaryTextActiveColor":"hsl(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), calc(var(--lia-bs-gray-900-l) * 0.9))","tertiaryBgColor":"transparent","tertiaryBgHoverColor":"transparent","tertiaryBgActiveColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.04)","tertiaryBorder":"1px solid transparent","tertiaryBorderHover":"1px solid hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","tertiaryBorderActive":"1px solid transparent","tertiaryBorderFocus":"1px solid transparent","tertiaryBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","destructiveTextColor":"var(--lia-bs-danger)","destructiveTextHoverColor":"hsl(var(--lia-bs-danger-h), var(--lia-bs-danger-s), calc(var(--lia-bs-danger-l) * 0.95))","destructiveTextActiveColor":"hsl(var(--lia-bs-danger-h), var(--lia-bs-danger-s), calc(var(--lia-bs-danger-l) * 0.9))","destructiveBgColor":"var(--lia-bs-gray-300)","destructiveBgHoverColor":"hsl(var(--lia-bs-gray-300-h), var(--lia-bs-gray-300-s), calc(var(--lia-bs-gray-300-l) * 0.96))","destructiveBgActiveColor":"hsl(var(--lia-bs-gray-300-h), var(--lia-bs-gray-300-s), calc(var(--lia-bs-gray-300-l) * 0.92))","destructiveBorder":"1px solid transparent","destructiveBorderHover":"1px solid transparent","destructiveBorderActive":"1px solid transparent","destructiveBorderFocus":"1px solid transparent","destructiveBoxShadowFocus":"0 0 0 1px var(--lia-bs-primary), 0 0 0 4px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","__typename":"ButtonsThemeSettings"},"border":{"color":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","mainContent":"NONE","sideContent":"NONE","radiusSm":"3px","radius":"5px","radiusLg":"9px","radius50":"100vw","__typename":"BorderThemeSettings"},"boxShadow":{"xs":"0 0 0 1px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.08), 0 3px 0 -1px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.08)","sm":"0 2px 4px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.06)","md":"0 5px 15px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.15)","lg":"0 10px 30px hsla(var(--lia-bs-gray-900-h), var(--lia-bs-gray-900-s), var(--lia-bs-gray-900-l), 0.15)","__typename":"BoxShadowThemeSettings"},"cards":{"bgColor":"var(--lia-panel-bg-color)","borderRadius":"var(--lia-panel-border-radius)","boxShadow":"var(--lia-box-shadow-xs)","__typename":"CardsThemeSettings"},"chip":{"maxWidth":"300px","height":"30px","__typename":"ChipThemeSettings"},"coreTypes":{"defaultMessageLinkColor":"var(--lia-bs-primary)","defaultMessageLinkDecoration":"none","defaultMessageLinkFontStyle":"NORMAL","defaultMessageLinkFontWeight":"400","defaultMessageFontStyle":"NORMAL","defaultMessageFontWeight":"400","forumColor":"#0C5C8D","forumFontFamily":"var(--lia-bs-font-family-base)","forumFontWeight":"var(--lia-default-message-font-weight)","forumLineHeight":"var(--lia-bs-line-height-base)","forumFontStyle":"var(--lia-default-message-font-style)","forumMessageLinkColor":"var(--lia-default-message-link-color)","forumMessageLinkDecoration":"var(--lia-default-message-link-decoration)","forumMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","forumMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","forumSolvedColor":"#62C026","blogColor":"#730015","blogFontFamily":"var(--lia-bs-font-family-base)","blogFontWeight":"var(--lia-default-message-font-weight)","blogLineHeight":"1.75","blogFontStyle":"var(--lia-default-message-font-style)","blogMessageLinkColor":"var(--lia-default-message-link-color)","blogMessageLinkDecoration":"var(--lia-default-message-link-decoration)","blogMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","blogMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","tkbColor":"#C20025","tkbFontFamily":"var(--lia-bs-font-family-base)","tkbFontWeight":"var(--lia-default-message-font-weight)","tkbLineHeight":"1.75","tkbFontStyle":"var(--lia-default-message-font-style)","tkbMessageLinkColor":"var(--lia-default-message-link-color)","tkbMessageLinkDecoration":"var(--lia-default-message-link-decoration)","tkbMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","tkbMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","qandaColor":"#4099E2","qandaFontFamily":"var(--lia-bs-font-family-base)","qandaFontWeight":"var(--lia-default-message-font-weight)","qandaLineHeight":"var(--lia-bs-line-height-base)","qandaFontStyle":"var(--lia-default-message-link-font-style)","qandaMessageLinkColor":"var(--lia-default-message-link-color)","qandaMessageLinkDecoration":"var(--lia-default-message-link-decoration)","qandaMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","qandaMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","qandaSolvedColor":"#3FA023","ideaColor":"#F3704B","ideaFontFamily":"var(--lia-bs-font-family-base)","ideaFontWeight":"var(--lia-default-message-font-weight)","ideaLineHeight":"var(--lia-bs-line-height-base)","ideaFontStyle":"var(--lia-default-message-font-style)","ideaMessageLinkColor":"var(--lia-default-message-link-color)","ideaMessageLinkDecoration":"var(--lia-default-message-link-decoration)","ideaMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","ideaMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","contestColor":"#FCC845","contestFontFamily":"var(--lia-bs-font-family-base)","contestFontWeight":"var(--lia-default-message-font-weight)","contestLineHeight":"var(--lia-bs-line-height-base)","contestFontStyle":"var(--lia-default-message-link-font-style)","contestMessageLinkColor":"var(--lia-default-message-link-color)","contestMessageLinkDecoration":"var(--lia-default-message-link-decoration)","contestMessageLinkFontStyle":"ITALIC","contestMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","occasionColor":"#EE4B5B","occasionFontFamily":"var(--lia-bs-font-family-base)","occasionFontWeight":"var(--lia-default-message-font-weight)","occasionLineHeight":"var(--lia-bs-line-height-base)","occasionFontStyle":"var(--lia-default-message-font-style)","occasionMessageLinkColor":"var(--lia-default-message-link-color)","occasionMessageLinkDecoration":"var(--lia-default-message-link-decoration)","occasionMessageLinkFontStyle":"var(--lia-default-message-link-font-style)","occasionMessageLinkFontWeight":"var(--lia-default-message-link-font-weight)","grouphubColor":"#491B62","categoryColor":"#949494","communityColor":"#FFFFFF","productColor":"#949494","__typename":"CoreTypesThemeSettings"},"colors":{"black":"#000000","white":"#FFFFFF","gray100":"#F7F7F7","gray200":"#F7F7F7","gray300":"#E8E8E8","gray400":"#D9D9D9","gray500":"#CCCCCC","gray600":"#949494","gray700":"#707070","gray800":"#545454","gray900":"#333333","dark":"#545454","light":"#F7F7F7","primary":"#0C5C8D","secondary":"#333333","bodyText":"#222222","bodyBg":"#F5F5F5","info":"#1D9CD3","success":"#62C026","warning":"#FFD651","danger":"#C20025","alertSystem":"#FF6600","textMuted":"#707070","highlight":"#FFFCAD","outline":"var(--lia-bs-primary)","custom":["#C20025","#081B85","#009639","#B3C6D7","#7CC0EB","#F29A36"],"__typename":"ColorsThemeSettings"},"divider":{"size":"3px","marginLeft":"4px","marginRight":"4px","borderRadius":"50%","bgColor":"var(--lia-bs-gray-600)","bgColorActive":"var(--lia-bs-gray-600)","__typename":"DividerThemeSettings"},"dropdown":{"fontSize":"var(--lia-bs-font-size-sm)","borderColor":"var(--lia-bs-border-color)","borderRadius":"var(--lia-bs-border-radius-sm)","dividerBg":"var(--lia-bs-gray-300)","itemPaddingY":"5px","itemPaddingX":"20px","headerColor":"var(--lia-bs-gray-700)","__typename":"DropdownThemeSettings"},"email":{"link":{"color":"#0069D4","hoverColor":"#0061c2","decoration":"none","hoverDecoration":"underline","__typename":"EmailLinkSettings"},"border":{"color":"#e4e4e4","__typename":"EmailBorderSettings"},"buttons":{"borderRadiusLg":"5px","paddingXLg":"16px","paddingYLg":"7px","fontWeight":"700","primaryTextColor":"#ffffff","primaryTextHoverColor":"#ffffff","primaryBgColor":"#0069D4","primaryBgHoverColor":"#005cb8","primaryBorder":"1px solid transparent","primaryBorderHover":"1px solid transparent","__typename":"EmailButtonsSettings"},"panel":{"borderRadius":"5px","borderColor":"#e4e4e4","__typename":"EmailPanelSettings"},"__typename":"EmailThemeSettings"},"emoji":{"skinToneDefault":"#ffcd43","skinToneLight":"#fae3c5","skinToneMediumLight":"#e2cfa5","skinToneMedium":"#daa478","skinToneMediumDark":"#a78058","skinToneDark":"#5e4d43","__typename":"EmojiThemeSettings"},"heading":{"color":"var(--lia-bs-body-color)","fontFamily":"Inter","fontStyle":"NORMAL","fontWeight":"600","h1FontSize":"30px","h2FontSize":"25px","h3FontSize":"20px","h4FontSize":"18px","h5FontSize":"16px","h6FontSize":"16px","lineHeight":"1.2","subHeaderFontSize":"11px","subHeaderFontWeight":"500","h1LetterSpacing":"normal","h2LetterSpacing":"normal","h3LetterSpacing":"normal","h4LetterSpacing":"normal","h5LetterSpacing":"normal","h6LetterSpacing":"normal","subHeaderLetterSpacing":"2px","h1FontWeight":"var(--lia-bs-headings-font-weight)","h2FontWeight":"var(--lia-bs-headings-font-weight)","h3FontWeight":"var(--lia-bs-headings-font-weight)","h4FontWeight":"var(--lia-bs-headings-font-weight)","h5FontWeight":"var(--lia-bs-headings-font-weight)","h6FontWeight":"var(--lia-bs-headings-font-weight)","__typename":"HeadingThemeSettings"},"icons":{"size10":"10px","size12":"12px","size14":"14px","size16":"16px","size20":"20px","size24":"24px","size30":"30px","size40":"40px","size50":"50px","size60":"60px","size80":"80px","size120":"120px","size160":"160px","__typename":"IconsThemeSettings"},"imagePreview":{"bgColor":"var(--lia-bs-gray-900)","titleColor":"var(--lia-bs-white)","controlColor":"var(--lia-bs-white)","controlBgColor":"var(--lia-bs-gray-800)","__typename":"ImagePreviewThemeSettings"},"input":{"borderColor":"var(--lia-bs-gray-600)","disabledColor":"var(--lia-bs-gray-600)","focusBorderColor":"var(--lia-bs-primary)","labelMarginBottom":"10px","btnFontSize":"var(--lia-bs-font-size-sm)","focusBoxShadow":"0 0 0 3px hsla(var(--lia-bs-primary-h), var(--lia-bs-primary-s), var(--lia-bs-primary-l), 0.2)","checkLabelMarginBottom":"2px","checkboxBorderRadius":"3px","borderRadiusSm":"var(--lia-bs-border-radius-sm)","borderRadius":"var(--lia-bs-border-radius)","borderRadiusLg":"var(--lia-bs-border-radius-lg)","formTextMarginTop":"4px","textAreaBorderRadius":"var(--lia-bs-border-radius)","activeFillColor":"var(--lia-bs-primary)","__typename":"InputThemeSettings"},"loading":{"dotDarkColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.2)","dotLightColor":"hsla(var(--lia-bs-white-h), var(--lia-bs-white-s), var(--lia-bs-white-l), 0.5)","barDarkColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.06)","barLightColor":"hsla(var(--lia-bs-white-h), var(--lia-bs-white-s), var(--lia-bs-white-l), 0.4)","__typename":"LoadingThemeSettings"},"link":{"color":"var(--lia-bs-primary)","hoverColor":"hsl(var(--lia-bs-primary-h), var(--lia-bs-primary-s), calc(var(--lia-bs-primary-l) - 10%))","decoration":"none","hoverDecoration":"underline","__typename":"LinkThemeSettings"},"listGroup":{"itemPaddingY":"15px","itemPaddingX":"15px","borderColor":"var(--lia-bs-gray-300)","__typename":"ListGroupThemeSettings"},"modal":{"contentTextColor":"var(--lia-bs-body-color)","contentBg":"var(--lia-bs-white)","backgroundBg":"var(--lia-bs-black)","smSize":"440px","mdSize":"760px","lgSize":"1080px","backdropOpacity":0.3,"contentBoxShadowXs":"var(--lia-bs-box-shadow-sm)","contentBoxShadow":"var(--lia-bs-box-shadow)","headerFontWeight":"700","__typename":"ModalThemeSettings"},"navbar":{"position":"FIXED","background":{"attachment":null,"clip":null,"color":"var(--lia-bs-white)","imageAssetName":null,"imageLastModified":"0","origin":null,"position":"CENTER_CENTER","repeat":"NO_REPEAT","size":"COVER","__typename":"BackgroundProps"},"backgroundOpacity":0.8,"paddingTop":"15px","paddingBottom":"15px","borderBottom":"1px solid var(--lia-bs-border-color)","boxShadow":"var(--lia-bs-box-shadow-sm)","brandMarginRight":"30px","brandMarginRightSm":"10px","brandLogoHeight":"30px","linkGap":"10px","linkJustifyContent":"flex-start","linkPaddingY":"5px","linkPaddingX":"10px","linkDropdownPaddingY":"9px","linkDropdownPaddingX":"var(--lia-nav-link-px)","linkColor":"var(--lia-bs-body-color)","linkHoverColor":"var(--lia-bs-primary)","linkFontSize":"var(--lia-bs-font-size-sm)","linkFontStyle":"NORMAL","linkFontWeight":"400","linkTextTransform":"NONE","linkLetterSpacing":"normal","linkBorderRadius":"var(--lia-bs-border-radius-sm)","linkBgColor":"transparent","linkBgHoverColor":"transparent","linkBorder":"none","linkBorderHover":"none","linkBoxShadow":"none","linkBoxShadowHover":"none","linkTextBorderBottom":"none","linkTextBorderBottomHover":"none","dropdownPaddingTop":"10px","dropdownPaddingBottom":"15px","dropdownPaddingX":"10px","dropdownMenuOffset":"2px","dropdownDividerMarginTop":"10px","dropdownDividerMarginBottom":"10px","dropdownBorderColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","controllerBgHoverColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.1)","controllerIconColor":"var(--lia-bs-body-color)","controllerIconHoverColor":"var(--lia-bs-body-color)","controllerTextColor":"var(--lia-nav-controller-icon-color)","controllerTextHoverColor":"var(--lia-nav-controller-icon-hover-color)","controllerHighlightColor":"hsla(30, 100%, 50%)","controllerHighlightTextColor":"var(--lia-yiq-light)","controllerBorderRadius":"var(--lia-border-radius-50)","hamburgerColor":"var(--lia-nav-controller-icon-color)","hamburgerHoverColor":"var(--lia-nav-controller-icon-color)","hamburgerBgColor":"transparent","hamburgerBgHoverColor":"transparent","hamburgerBorder":"none","hamburgerBorderHover":"none","collapseMenuMarginLeft":"20px","collapseMenuDividerBg":"var(--lia-nav-link-color)","collapseMenuDividerOpacity":0.16,"__typename":"NavbarThemeSettings"},"pager":{"textColor":"var(--lia-bs-link-color)","textFontWeight":"var(--lia-font-weight-md)","textFontSize":"var(--lia-bs-font-size-sm)","__typename":"PagerThemeSettings"},"panel":{"bgColor":"var(--lia-bs-white)","borderRadius":"var(--lia-bs-border-radius)","borderColor":"var(--lia-bs-border-color)","boxShadow":"none","__typename":"PanelThemeSettings"},"popover":{"arrowHeight":"8px","arrowWidth":"16px","maxWidth":"300px","minWidth":"100px","headerBg":"var(--lia-bs-white)","borderColor":"var(--lia-bs-border-color)","borderRadius":"var(--lia-bs-border-radius)","boxShadow":"0 0.5rem 1rem hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.15)","__typename":"PopoverThemeSettings"},"prism":{"color":"#000000","bgColor":"#f5f2f0","fontFamily":"var(--font-family-monospace)","fontSize":"var(--lia-bs-font-size-base)","fontWeightBold":"var(--lia-bs-font-weight-bold)","fontStyleItalic":"italic","tabSize":2,"highlightColor":"#b3d4fc","commentColor":"#62707e","punctuationColor":"#6f6f6f","namespaceOpacity":"0.7","propColor":"#990055","selectorColor":"#517a00","operatorColor":"#906736","operatorBgColor":"hsla(0, 0%, 100%, 0.5)","keywordColor":"#0076a9","functionColor":"#d3284b","variableColor":"#c14700","__typename":"PrismThemeSettings"},"rte":{"bgColor":"var(--lia-bs-white)","borderRadius":"var(--lia-panel-border-radius)","boxShadow":" var(--lia-panel-box-shadow)","customColor1":"#bfedd2","customColor2":"#fbeeb8","customColor3":"#f8cac6","customColor4":"#eccafa","customColor5":"#c2e0f4","customColor6":"#2dc26b","customColor7":"#f1c40f","customColor8":"#e03e2d","customColor9":"#b96ad9","customColor10":"#3598db","customColor11":"#169179","customColor12":"#e67e23","customColor13":"#ba372a","customColor14":"#843fa1","customColor15":"#236fa1","customColor16":"#ecf0f1","customColor17":"#ced4d9","customColor18":"#95a5a6","customColor19":"#7e8c8d","customColor20":"#34495e","customColor21":"#000000","customColor22":"#ffffff","defaultMessageHeaderMarginTop":"14px","defaultMessageHeaderMarginBottom":"10px","defaultMessageItemMarginTop":"0","defaultMessageItemMarginBottom":"10px","diffAddedColor":"hsla(170, 53%, 51%, 0.4)","diffChangedColor":"hsla(43, 97%, 63%, 0.4)","diffNoneColor":"hsla(0, 0%, 80%, 0.4)","diffRemovedColor":"hsla(9, 74%, 47%, 0.4)","specialMessageHeaderMarginTop":"14px","specialMessageHeaderMarginBottom":"10px","specialMessageItemMarginTop":"0","specialMessageItemMarginBottom":"10px","__typename":"RteThemeSettings"},"tags":{"bgColor":"var(--lia-bs-gray-200)","bgHoverColor":"var(--lia-bs-gray-400)","borderRadius":"var(--lia-bs-border-radius-sm)","color":"var(--lia-bs-body-color)","hoverColor":"var(--lia-bs-body-color)","fontWeight":"var(--lia-font-weight-md)","fontSize":"var(--lia-font-size-xxs)","textTransform":"UPPERCASE","letterSpacing":"0.5px","__typename":"TagsThemeSettings"},"toasts":{"borderRadius":"var(--lia-bs-border-radius)","paddingX":"12px","__typename":"ToastsThemeSettings"},"typography":{"fontFamilyBase":"Atkinson Hyperlegible","fontStyleBase":"NORMAL","fontWeightBase":"400","fontWeightLight":"300","fontWeightNormal":"400","fontWeightMd":"500","fontWeightBold":"700","letterSpacingSm":"normal","letterSpacingXs":"normal","lineHeightBase":"1.3","fontSizeBase":"15px","fontSizeXxs":"11px","fontSizeXs":"12px","fontSizeSm":"13px","fontSizeLg":"20px","fontSizeXl":"24px","smallFontSize":"14px","customFonts":[],"__typename":"TypographyThemeSettings"},"unstyledListItem":{"marginBottomSm":"5px","marginBottomMd":"10px","marginBottomLg":"15px","marginBottomXl":"20px","marginBottomXxl":"25px","__typename":"UnstyledListItemThemeSettings"},"yiq":{"light":"#ffffff","dark":"#000000","__typename":"YiqThemeSettings"},"colorLightness":{"primaryDark":0.36,"primaryLight":0.74,"primaryLighter":0.89,"primaryLightest":0.95,"infoDark":0.39,"infoLight":0.72,"infoLighter":0.85,"infoLightest":0.93,"successDark":0.24,"successLight":0.62,"successLighter":0.8,"successLightest":0.91,"warningDark":0.39,"warningLight":0.68,"warningLighter":0.84,"warningLightest":0.93,"dangerDark":0.41,"dangerLight":0.72,"dangerLighter":0.89,"dangerLightest":0.95,"__typename":"ColorLightnessThemeSettings"},"localOverride":false,"__typename":"Theme"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/Loading/LoadingDot-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/Loading/LoadingDot-1743097583694","value":{"title":"Loading..."},"localOverride":false},"CachedAsset:text:en_US-components/common/EmailVerification-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/common/EmailVerification-1743097583694","value":{"email.verification.title":"Email Verification Required","email.verification.message.update.email":"To participate in the community, you must first verify your email address. The verification email was sent to {email}. To change your email, visit My Settings.","email.verification.message.resend.email":"To participate in the community, you must first verify your email address. The verification email was sent to {email}. Resend email."},"localOverride":false},"CachedAsset:text:en_US-pages/tags/TagPage-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-pages/tags/TagPage-1743097583694","value":{"tagPageTitle":"Tag:\"{tagName}\" | {communityTitle}","tagPageForNodeTitle":"Tag:\"{tagName}\" in \"{title}\" | {communityTitle}","name":"Tags Page","tag":"Tag: {tagName}"},"localOverride":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bi0zNC0xM2k0MzE3N0Q2NjFBRDg5NDAy\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bi0zNC0xM2k0MzE3N0Q2NjFBRDg5NDAy","mimeType":"image/png"},"Category:category:Articles":{"__typename":"Category","id":"category:Articles","entityType":"CATEGORY","displayId":"Articles","nodeType":"category","depth":1,"title":"Articles","shortTitle":"Articles","parent":{"__ref":"Category:category:top"},"categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:top":{"__typename":"Category","id":"category:top","displayId":"top","nodeType":"category","depth":0,"title":"Top"},"Tkb:board:TechnicalArticles":{"__typename":"Tkb","id":"board:TechnicalArticles","entityType":"TKB","displayId":"TechnicalArticles","nodeType":"board","depth":2,"conversationStyle":"TKB","title":"Technical Articles","description":"F5 SMEs share good practice.","avatar":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bi0zNC0xM2k0MzE3N0Q2NjFBRDg5NDAy\"}"},"profileSettings":{"__typename":"ProfileSettings","language":null},"parent":{"__ref":"Category:category:Articles"},"ancestors":{"__typename":"CoreNodeConnection","edges":[{"__typename":"CoreNodeEdge","node":{"__ref":"Community:community:zihoc95639"}},{"__typename":"CoreNodeEdge","node":{"__ref":"Category:category:Articles"}}]},"userContext":{"__typename":"NodeUserContext","canAddAttachments":false,"canUpdateNode":false,"canPostMessages":false,"isSubscribed":false},"boardPolicies":{"__typename":"BoardPolicies","canPublishArticleOnCreate":{"__typename":"PolicyResult","failureReason":{"__typename":"FailureReason","message":"error.lithium.policies.forums.policy_can_publish_on_create_workflow_action.accessDenied","key":"error.lithium.policies.forums.policy_can_publish_on_create_workflow_action.accessDenied","args":[]}},"canReadNode":{"__typename":"PolicyResult","failureReason":null}},"tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"shortTitle":"Technical Articles","tagPolicies":{"__typename":"TagPolicies","canSubscribeTagOnNode":{"__typename":"PolicyResult","failureReason":{"__typename":"FailureReason","message":"error.lithium.policies.labels.action.corenode.subscribe_labels.allow.accessDenied","key":"error.lithium.policies.labels.action.corenode.subscribe_labels.allow.accessDenied","args":[]}},"canManageTagDashboard":{"__typename":"PolicyResult","failureReason":{"__typename":"FailureReason","message":"error.lithium.policies.labels.action.corenode.admin_labels.allow.accessDenied","key":"error.lithium.policies.labels.action.corenode.admin_labels.allow.accessDenied","args":[]}}}},"CachedAsset:quilt:f5.prod:pages/tags/TagPage:board:TechnicalArticles-1743097588405":{"__typename":"CachedAsset","id":"quilt:f5.prod:pages/tags/TagPage:board:TechnicalArticles-1743097588405","value":{"id":"TagPage","container":{"id":"Common","headerProps":{"removeComponents":["community.widget.bannerWidget"],"__typename":"QuiltContainerSectionProps"},"items":[{"id":"tag-header-widget","layout":"ONE_COLUMN","bgColor":"var(--lia-bs-white)","showBorder":"BOTTOM","sectionEditLevel":"LOCKED","columnMap":{"main":[{"id":"tags.widget.TagsHeaderWidget","__typename":"QuiltComponent"}],"__typename":"OneSectionColumns"},"__typename":"OneColumnQuiltSection"},{"id":"messages-list-for-tag-widget","layout":"ONE_COLUMN","columnMap":{"main":[{"id":"messages.widget.messageListForNodeByRecentActivityWidget","props":{"viewVariant":{"type":"inline","props":{"useUnreadCount":true,"useViewCount":true,"useAuthorLogin":true,"clampBodyLines":3,"useAvatar":true,"useBoardIcon":false,"useKudosCount":true,"usePreviewMedia":true,"useTags":false,"useNode":true,"useNodeLink":true,"useTextBody":true,"truncateBodyLength":-1,"useBody":true,"useRepliesCount":true,"useSolvedBadge":true,"timeStampType":"conversation.lastPostingActivityTime","useMessageTimeLink":true,"clampSubjectLines":2}},"panelType":"divider","useTitle":false,"hideIfEmpty":false,"pagerVariant":{"type":"loadMore"},"style":"list","showTabs":true,"tabItemMap":{"default":{"mostRecent":true,"mostRecentUserContent":false,"newest":false},"additional":{"mostKudoed":true,"mostViewed":true,"mostReplies":false,"noReplies":false,"noSolutions":false,"solutions":false}}},"__typename":"QuiltComponent"}],"__typename":"OneSectionColumns"},"__typename":"OneColumnQuiltSection"}],"__typename":"QuiltContainer"},"__typename":"Quilt"},"localOverride":false},"CachedAsset:quiltWrapper:f5.prod:Common:1742462839017":{"__typename":"CachedAsset","id":"quiltWrapper:f5.prod:Common:1742462839017","value":{"id":"Common","header":{"backgroundImageProps":{"assetName":"header.jpg","backgroundSize":"COVER","backgroundRepeat":"NO_REPEAT","backgroundPosition":"LEFT_CENTER","lastModified":"1702932449000","__typename":"BackgroundImageProps"},"backgroundColor":"transparent","items":[{"id":"custom.widget.Beta_MetaNav","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"community.widget.navbarWidget","props":{"showUserName":false,"showRegisterLink":true,"style":{"boxShadow":"var(--lia-bs-box-shadow-sm)","linkFontWeight":"700","controllerHighlightColor":"hsla(30, 100%, 50%)","dropdownDividerMarginBottom":"10px","hamburgerBorderHover":"none","linkFontSize":"15px","linkBoxShadowHover":"none","backgroundOpacity":0.4,"controllerBorderRadius":"var(--lia-border-radius-50)","hamburgerBgColor":"transparent","linkTextBorderBottom":"none","hamburgerColor":"var(--lia-nav-controller-icon-color)","brandLogoHeight":"48px","linkLetterSpacing":"normal","linkBgHoverColor":"transparent","collapseMenuDividerOpacity":0.16,"paddingBottom":"10px","dropdownPaddingBottom":"15px","dropdownMenuOffset":"2px","hamburgerBgHoverColor":"transparent","borderBottom":"0","hamburgerBorder":"none","dropdownPaddingX":"10px","brandMarginRightSm":"10px","linkBoxShadow":"none","linkJustifyContent":"center","linkColor":"var(--lia-bs-primary)","collapseMenuDividerBg":"var(--lia-nav-link-color)","dropdownPaddingTop":"10px","controllerHighlightTextColor":"var(--lia-yiq-dark)","background":{"imageAssetName":"","color":"var(--lia-bs-white)","size":"COVER","repeat":"NO_REPEAT","position":"CENTER_CENTER","imageLastModified":""},"linkBorderRadius":"var(--lia-bs-border-radius-sm)","linkHoverColor":"var(--lia-bs-primary)","position":"FIXED","linkBorder":"none","linkTextBorderBottomHover":"2px solid #0C5C8D","brandMarginRight":"30px","hamburgerHoverColor":"var(--lia-nav-controller-icon-color)","linkBorderHover":"none","collapseMenuMarginLeft":"20px","linkFontStyle":"NORMAL","linkPaddingX":"10px","paddingTop":"10px","linkPaddingY":"5px","linkTextTransform":"NONE","dropdownBorderColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.08)","controllerBgHoverColor":"hsla(var(--lia-bs-black-h), var(--lia-bs-black-s), var(--lia-bs-black-l), 0.1)","linkDropdownPaddingX":"var(--lia-nav-link-px)","linkBgColor":"transparent","linkDropdownPaddingY":"9px","controllerIconColor":"#0C5C8D","dropdownDividerMarginTop":"10px","linkGap":"10px","controllerIconHoverColor":"#0C5C8D"},"links":{"sideLinks":[],"mainLinks":[{"children":[{"linkType":"INTERNAL","id":"migrated-link-1","params":{"boardId":"TechnicalForum","categoryId":"Forums"},"routeName":"ForumBoardPage"},{"linkType":"INTERNAL","id":"migrated-link-2","params":{"boardId":"WaterCooler","categoryId":"Forums"},"routeName":"ForumBoardPage"}],"linkType":"INTERNAL","id":"migrated-link-0","params":{"categoryId":"Forums"},"routeName":"CategoryPage"},{"children":[{"linkType":"INTERNAL","id":"migrated-link-4","params":{"boardId":"codeshare","categoryId":"CrowdSRC"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"migrated-link-5","params":{"boardId":"communityarticles","categoryId":"CrowdSRC"},"routeName":"TkbBoardPage"}],"linkType":"INTERNAL","id":"migrated-link-3","params":{"categoryId":"CrowdSRC"},"routeName":"CategoryPage"},{"children":[{"linkType":"INTERNAL","id":"migrated-link-7","params":{"boardId":"TechnicalArticles","categoryId":"Articles"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"article-series","params":{"boardId":"article-series","categoryId":"Articles"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"security-insights","params":{"boardId":"security-insights","categoryId":"Articles"},"routeName":"TkbBoardPage"},{"linkType":"INTERNAL","id":"migrated-link-8","params":{"boardId":"DevCentralNews","categoryId":"Articles"},"routeName":"TkbBoardPage"}],"linkType":"INTERNAL","id":"migrated-link-6","params":{"categoryId":"Articles"},"routeName":"CategoryPage"},{"children":[{"linkType":"INTERNAL","id":"migrated-link-10","params":{"categoryId":"CommunityGroups"},"routeName":"CategoryPage"},{"linkType":"INTERNAL","id":"migrated-link-11","params":{"categoryId":"F5-Groups"},"routeName":"CategoryPage"}],"linkType":"INTERNAL","id":"migrated-link-9","params":{"categoryId":"GroupsCategory"},"routeName":"CategoryPage"},{"children":[],"linkType":"INTERNAL","id":"migrated-link-12","params":{"boardId":"Events","categoryId":"top"},"routeName":"EventBoardPage"},{"children":[],"linkType":"INTERNAL","id":"migrated-link-13","params":{"boardId":"Suggestions","categoryId":"top"},"routeName":"IdeaBoardPage"},{"children":[],"linkType":"EXTERNAL","id":"Common-external-link","url":"https://community.f5.com/c/how-do-i","target":"SELF"}]},"className":"QuiltComponent_lia-component-edit-mode__lQ9Z6","showSearchIcon":false},"__typename":"QuiltComponent"},{"id":"community.widget.bannerWidget","props":{"backgroundColor":"transparent","visualEffects":{"showBottomBorder":false},"backgroundImageProps":{"backgroundSize":"COVER","backgroundPosition":"CENTER_CENTER","backgroundRepeat":"NO_REPEAT"},"fontColor":"#222222"},"__typename":"QuiltComponent"},{"id":"community.widget.breadcrumbWidget","props":{"backgroundColor":"var(--lia-bs-primary)","linkHighlightColor":"#FFFFFF","visualEffects":{"showBottomBorder":false},"backgroundOpacity":60,"linkTextColor":"#FFFFFF"},"__typename":"QuiltComponent"}],"__typename":"QuiltWrapperSection"},"footer":{"backgroundImageProps":{"assetName":null,"backgroundSize":"COVER","backgroundRepeat":"NO_REPEAT","backgroundPosition":"CENTER_CENTER","lastModified":null,"__typename":"BackgroundImageProps"},"backgroundColor":"var(--lia-bs-body-color)","items":[{"id":"custom.widget.Beta_Footer","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"custom.widget.Tag_Manager_Helper","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"},{"id":"custom.widget.Consent_Blackbar","props":{"widgetVisibility":"signedInOrAnonymous","useTitle":true,"useBackground":false,"title":"","lazyLoad":false},"__typename":"QuiltComponent"}],"__typename":"QuiltWrapperSection"},"__typename":"QuiltWrapper","localOverride":false},"localOverride":false},"CachedAsset:text:en_US-components/common/ActionFeedback-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/common/ActionFeedback-1743097583694","value":{"joinedGroupHub.title":"Welcome","joinedGroupHub.message":"You are now a member of this group and are subscribed to updates.","groupHubInviteNotFound.title":"Invitation Not Found","groupHubInviteNotFound.message":"Sorry, we could not find your invitation to the group. The owner may have canceled the invite.","groupHubNotFound.title":"Group Not Found","groupHubNotFound.message":"The grouphub you tried to join does not exist. It may have been deleted.","existingGroupHubMember.title":"Already Joined","existingGroupHubMember.message":"You are already a member of this group.","accountLocked.title":"Account Locked","accountLocked.message":"Your account has been locked due to multiple failed attempts. Try again in {lockoutTime} minutes.","editedGroupHub.title":"Changes Saved","editedGroupHub.message":"Your group has been updated.","leftGroupHub.title":"Goodbye","leftGroupHub.message":"You are no longer a member of this group and will not receive future updates.","deletedGroupHub.title":"Deleted","deletedGroupHub.message":"The group has been deleted.","groupHubCreated.title":"Group Created","groupHubCreated.message":"{groupHubName} is ready to use","accountClosed.title":"Account Closed","accountClosed.message":"The account has been closed and you will now be redirected to the homepage","resetTokenExpired.title":"Reset Password Link has Expired","resetTokenExpired.message":"Try resetting your password again","invalidUrl.title":"Invalid URL","invalidUrl.message":"The URL you're using is not recognized. Verify your URL and try again.","accountClosedForUser.title":"Account Closed","accountClosedForUser.message":"{userName}'s account is closed","inviteTokenInvalid.title":"Invitation Invalid","inviteTokenInvalid.message":"Your invitation to the community has been canceled or expired.","inviteTokenError.title":"Invitation Verification Failed","inviteTokenError.message":"The url you are utilizing is not recognized. Verify your URL and try again","pageNotFound.title":"Access Denied","pageNotFound.message":"You do not have access to this area of the community or it doesn't exist","eventAttending.title":"Responded as Attending","eventAttending.message":"You'll be notified when there's new activity and reminded as the event approaches","eventInterested.title":"Responded as Interested","eventInterested.message":"You'll be notified when there's new activity and reminded as the event approaches","eventNotFound.title":"Event Not Found","eventNotFound.message":"The event you tried to respond to does not exist.","redirectToRelatedPage.title":"Showing Related Content","redirectToRelatedPageForBaseUsers.title":"Showing Related Content","redirectToRelatedPageForBaseUsers.message":"The content you are trying to access is archived","redirectToRelatedPage.message":"The content you are trying to access is archived","relatedUrl.archivalLink.flyoutMessage":"The content you are trying to access is archived View Archived Content"},"localOverride":false},"CachedAsset:component:custom.widget.Beta_MetaNav-en-1742462941765":{"__typename":"CachedAsset","id":"component:custom.widget.Beta_MetaNav-en-1742462941765","value":{"component":{"id":"custom.widget.Beta_MetaNav","template":{"id":"Beta_MetaNav","markupLanguage":"HANDLEBARS","style":null,"texts":null,"defaults":{"config":{"applicablePages":[],"description":"MetaNav menu at the top of every page.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Beta_MetaNav","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"CUSTOM","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"MetaNav menu at the top of every page.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Beta_Footer-en-1742462941765":{"__typename":"CachedAsset","id":"component:custom.widget.Beta_Footer-en-1742462941765","value":{"component":{"id":"custom.widget.Beta_Footer","template":{"id":"Beta_Footer","markupLanguage":"HANDLEBARS","style":null,"texts":null,"defaults":{"config":{"applicablePages":[],"description":"DevCentral´s custom footer.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Beta_Footer","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"CUSTOM","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"DevCentral´s custom footer.","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Tag_Manager_Helper-en-1742462941765":{"__typename":"CachedAsset","id":"component:custom.widget.Tag_Manager_Helper-en-1742462941765","value":{"component":{"id":"custom.widget.Tag_Manager_Helper","template":{"id":"Tag_Manager_Helper","markupLanguage":"HANDLEBARS","style":null,"texts":null,"defaults":{"config":{"applicablePages":[],"description":"Helper widget to inject Tag Manager scripts into head element","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Tag_Manager_Helper","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"CUSTOM","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"Helper widget to inject Tag Manager scripts into head element","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:component:custom.widget.Consent_Blackbar-en-1742462941765":{"__typename":"CachedAsset","id":"component:custom.widget.Consent_Blackbar-en-1742462941765","value":{"component":{"id":"custom.widget.Consent_Blackbar","template":{"id":"Consent_Blackbar","markupLanguage":"HTML","style":null,"texts":null,"defaults":{"config":{"applicablePages":[],"description":"","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"components":[{"id":"custom.widget.Consent_Blackbar","form":null,"config":null,"props":[],"__typename":"Component"}],"grouping":"TEXTHTML","__typename":"ComponentTemplate"},"properties":{"config":{"applicablePages":[],"description":"","fetchedContent":null,"__typename":"ComponentConfiguration"},"props":[],"__typename":"ComponentProperties"},"form":null,"__typename":"Component","localOverride":false},"globalCss":null,"form":null},"localOverride":false},"CachedAsset:text:en_US-components/community/Breadcrumb-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/community/Breadcrumb-1743097583694","value":{"navLabel":"Breadcrumbs","dropdown":"Additional parent page navigation"},"localOverride":false},"CachedAsset:text:en_US-components/tags/TagsHeaderWidget-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/tags/TagsHeaderWidget-1743097583694","value":{"tag":"{tagName}","topicsCount":"{count} {count, plural, one {Topic} other {Topics}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageListForNodeByRecentActivityWidget-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageListForNodeByRecentActivityWidget-1743097583694","value":{"title@userScope:other":"Recent Content","title@userScope:self":"Contributions","title@board:FORUM@userScope:other":"Recent Discussions","title@board:BLOG@userScope:other":"Recent Blogs","emptyDescription":"No content to show","MessageListForNodeByRecentActivityWidgetEditor.nodeScope.label":"Scope","title@instance:1706288370055":"Content Feed","title@instance:1743095186784":"Most Recent Updates","title@instance:1704317906837":"Content Feed","title@instance:1743095018194":"Most Recent Updates","title@instance:1702668293472":"Community Feed","title@instance:1743095117047":"Most Recent Updates","title@instance:1704319314827":"Blog Feed","title@instance:1743095235555":"Most Recent Updates","title@instance:1704320290851":"My Contributions","title@instance:1703720491809":"Forum Feed","title@instance:1743095311723":"Most Recent Updates","title@instance:1703028709746":"Group Content Feed","title@instance:VTsglH":"Content Feed"},"localOverride":false},"Category:category:Forums":{"__typename":"Category","id":"category:Forums","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Forum:board:TechnicalForum":{"__typename":"Forum","id":"board:TechnicalForum","forumPolicies":{"__typename":"ForumPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Forum:board:WaterCooler":{"__typename":"Forum","id":"board:WaterCooler","forumPolicies":{"__typename":"ForumPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:DevCentralNews":{"__typename":"Tkb","id":"board:DevCentralNews","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:GroupsCategory":{"__typename":"Category","id":"category:GroupsCategory","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:F5-Groups":{"__typename":"Category","id":"category:F5-Groups","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:CommunityGroups":{"__typename":"Category","id":"category:CommunityGroups","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Occasion:board:Events":{"__typename":"Occasion","id":"board:Events","boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"occasionPolicies":{"__typename":"OccasionPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Idea:board:Suggestions":{"__typename":"Idea","id":"board:Suggestions","boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"ideaPolicies":{"__typename":"IdeaPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Category:category:CrowdSRC":{"__typename":"Category","id":"category:CrowdSRC","categoryPolicies":{"__typename":"CategoryPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:codeshare":{"__typename":"Tkb","id":"board:codeshare","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:communityarticles":{"__typename":"Tkb","id":"board:communityarticles","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:security-insights":{"__typename":"Tkb","id":"board:security-insights","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Tkb:board:article-series":{"__typename":"Tkb","id":"board:article-series","tkbPolicies":{"__typename":"TkbPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}},"boardPolicies":{"__typename":"BoardPolicies","canReadNode":{"__typename":"PolicyResult","failureReason":null}}},"Conversation:conversation:332019":{"__typename":"Conversation","id":"conversation:332019","topic":{"__typename":"TkbTopicMessage","uid":332019},"lastPostingActivityTime":"2025-01-30T08:33:04.797-08:00","solved":false},"User:user:154941":{"__typename":"User","uid":154941,"login":"Fouad_Chmainy","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0xNTQ5NDEtMTkzOTJpNzVDODM1NDJBRUE0RTk1Mg"},"id":"user:154941"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktT3VGNjB1?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktT3VGNjB1?revision=4","title":"clipboard_image-1-1721822252047.png","associationType":"BODY","width":891,"height":290,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktR3NxVEhC?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktR3NxVEhC?revision=4","title":"clipboard_image-2-1721822341543.png","associationType":"BODY","width":595,"height":458,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktMElCb2FU?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktMElCb2FU?revision=4","title":"clipboard_image-3-1721822463177.png","associationType":"BODY","width":751,"height":135,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktdDR2WEdO?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktdDR2WEdO?revision=4","title":"clipboard_image-4-1721822463179.png","associationType":"BODY","width":584,"height":460,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktaWRnazBx?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktaWRnazBx?revision=4","title":"clipboard_image-5-1721822463181.png","associationType":"BODY","width":593,"height":310,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktQUJLbmtU?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktQUJLbmtU?revision=4","title":"clipboard_image-6-1721822463183.png","associationType":"BODY","width":602,"height":512,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktV25sYUhq?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktV25sYUhq?revision=4","title":"clipboard_image-7-1721822463187.png","associationType":"BODY","width":670,"height":553,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktM0NDUXdB?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktM0NDUXdB?revision=4","title":"image.png","associationType":"BODY","width":452,"height":302,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTkteVNkSkU0?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTkteVNkSkU0?revision=4","title":"image.png","associationType":"BODY","width":295,"height":241,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktWVVDRFIw?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktWVVDRFIw?revision=4","title":"image.png","associationType":"BODY","width":201,"height":219,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktbXp3OWlm?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktbXp3OWlm?revision=4","title":"clipboard_image-8-1721823243969.png","associationType":"BODY","width":1895,"height":835,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktQlVhQmRt?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktQlVhQmRt?revision=4","title":"clipboard_image-9-1721823243972.png","associationType":"BODY","width":560,"height":276,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktZTdPWHdC?revision=4\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktZTdPWHdC?revision=4","title":"image.png","associationType":"BODY","width":352,"height":185,"altText":""},"TkbTopicMessage:message:332019":{"__typename":"TkbTopicMessage","subject":"What is Message Queue Telemetry Transport (MQTT)? How to secure MQTT?","conversation":{"__ref":"Conversation:conversation:332019"},"id":"message:332019","revisionNum":4,"uid":332019,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:154941"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":" MQTT is a messaging protocol broadly used in IoT and connected services, very lightweight and reliable even over poor quality networks. It is designed lightweight so it can work on constrained devices but, even in its latest version MQTTv5, the attack surface is very large. ","introduction":"","metrics":{"__typename":"MessageMetrics","views":306},"postTime":"2024-08-12T05:00:00.031-07:00","lastPublishTime":"2024-08-12T05:00:00.031-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Table of Content \n 1. What is MQTT? \n 1.1. Presentation \n 1.2. Constrained devices \n 1.3. Use Cases \n 1.4. Alternatives to MQTT \n 2. Architecture \n 2.1. Overall Architecture \n 2.2. Terminology \n 2.2.1. Topic \n 2.2.2. Message \n 2.2.3. Subscriber \n 2.2.4. Publisher \n 2.2.5. Broker \n 2.2.6. Who is the Client, who is the Server? \n 2.3. Traffic overview \n 2.4. MQTT Quality of Service (QoS) \n 2.5. Load Balancing of brokers \n 2.5.1. Extend Natively with NGINX \n 2.5.2. Extend with NJS \n 3. MQTT Brokers \n 3.1. Most popular MQTT brokers \n 3.1. Why proxy MQTT traffic? \n 4. MQTT Security \n 4.1. Security Context and Attack surface \n 4.2. IoT devices generally don’t encrypt communication \n 4.3 Authentication and AccessControl \n 4.4. Injections \n 5. How can NGINX enrich your MQTT use cases? \n 5.1. What is NGINX? \n 5.2. Traffic optimisation \n 5.3. MQTT Security \n 5.4. Analytics and Telemetry \n \n 1. What is MQTT? \n 1.1. Presentation \n Message Queue Telemetry Transport (MQTT) is a standard messaging protocol for the Internet of Things (IoT). \n MQTT is designed with a small footprint and a lightweight publish/subscribe messaging transport, which makes it ideal for connecting IoT devices for various use cases, from connecting smart home devices to industrial robots or connected cars. \n Benefits of MQTT \n \n Lightweight : The hardware requirements to run MQTT clients are very small so it can be used on any small microchips. \n Network Efficiency: MQTT messages are small, so it does not require high bandwidth to operate. \n Reliable: Reliability of message delivery is important for many IoT use cases. This is why MQTT has different quality of service levels. \n Support for Unreliable Networks: Many IoT devices connect over unreliable cellular networks. MQTT’s support for persistent sessions reduces the time to reconnect the client with the broker. \n Security Enabled: MQTT has support for message encryption and authentication. We will see in this document that we often make trade-offs on security to keep the lightness of devices. \n \n MQTT Publish / Subscribe Architecture \n \n The current implementation of MQTT is MQTTv5 which brings more scalability, improved features and better error handling than its older brother MQTT v3. \n References: \n https://mqtt.org/ \n \n 1.2. Constrained devices \n When you talk about IoT you directly think about your connected coffee machine, connected toothbrush or sensors hidden everywhere in your apartment. You certainly don’t want to have a computer sized add-on to your devices that consumes more power than the device itself. \n That’s why there is a strong need to have constrained devices that have low power consumption requirements and have a small footprint. However, this also means limited compute performance including security processing. \n There are different classed for constrained devices depending on their RAM and flash requirements which directly impact their power consumption. \n \n \n \n \n \n Classes of Constrained Devices \n \n \n \n \n CLASS \n \n \n RAM Size \n \n \n Flash Size \n \n \n \n \n Class 0, C0 \n \n \n << 10 KiB \n \n \n << 100KB \n \n \n \n \n Class 1, C1 \n \n \n ~ 10 KiB \n \n \n ~ 100 KiB \n \n \n \n \n Class 2, C2 \n \n \n ~50KB \n \n \n ~250KB \n \n \n \n \n \n https://datatracker.ietf.org/doc/html/rfc7228 \n \n 1.3. Use Cases \n When I started looking into messaging protocols and MQTT, I only had 2 use cases in mind: smart homes and connected cars. An example of a smart home use case would be a connected temperature sensor. These tend to be more expensive than traditional thermometers, but it can provide nice graphs on my mobile device and does the conversion between °F and °C automatically. In terms of connected cars, more and more vehicle manufacturers offer remote telemetry, mobile device connectivity and data-enriched emergency notification services. \n In my research, I was amazed to discover how prevalent IoT and Messaging Protocols are across various industries. Here are some examples: \n \n \n \n \n \n Automotive: MQTT can be used by automotive manufacturers to collect car’s sensors data to process them centrally and give more assistance to the driver and enrich the driving experience and security. It is also used to deliver software updates over-the-air or to augment navigation systems capabilities. \n Logistics: MQTT can be used for real-time tracking of assets and transportation vehicles. It is also used to have a constantly up to date inventory of stocks. \n Retail: Smart stores, inventory, customer tracking and analytics, Point-of-Sale integration. \n Manufacturing: Robots and equipment monitoring to identify failures and improve energy consumption, optimize productivity efficiency, coordination between supply chains. \n Medical & Healthcare: With the progress in medical technologies, the cost of medical consultations combined with the lack of medical care in rural areas, there is greater need than ever to provide remote patient monitoring for vital metrics such as blood pressure or blood sugar. Data can be collected by connected medical equipment at home and sent to a central location for analysis and alerting. \n Smart Home: anything that can make you change your Internet router configuration from a /24 to a /16 network like multimedia, light controls, Temperature, home security… \n Smart Cities: Monitoring any environmental factor such as air quality, weather, temperature or react rapidly to emergency situations. \n Oil & Gas: Petrochemical industries are very sensitive and require constant monitoring and control, safety monitoring and alerting and environment observability. \n … \n \n \n 1.4. Alternatives to MQTT \n When it comes to considering a communication protocol for IoT or Machine-2-Machine data exchange, you need to consider the capabilities and constraints of the devices themselves: \n \n Energy efficiency so it does not drain too much power when devices are powered by USB or batteries. \n Performance, depending on the use case the device could receive multiple measurements per minute and needs to respond to consumers sub-second. \n Resource requirements – devices that are affordable and can fit in a constrained space. \n Network quality: In many cases, IoT devices need to communicate over-the-air via mobile networks where network quality is not always guaranteed. \n Message reliability: Unfortunately, packet loss can be a reality. Depending on the use case and the importance of every single message, it should have an inner mechanism to reduce packet loss and retransmit missing packets. \n Queueing: In large-scale deployments, thousands or up to millions of devices will be exchanging messages rapidly. The infrastructure should scale and have a modern message bus that clients can connect to in order to publish or subscribe to messages. \n \n - … \n \n Although there are multiple choices for communication protocols such as: Constrained Application Protocol (CoAP), eXtensible Messaging and Presence Protocol (XMPP), Advanced Message Queuing Protocol (AMQP) and of course Hypertext Transfer Protocol (HTTP), MQTT is, by far, the one that has the best balance between network/hardware/energy requirements vs reliability/queueing/network capabilities. \n Of course, when ample physical space and power supply afford you the ability introduce edge computing capabilities, other protocols like Websockets and gRPC can also be considered. However, you may need to explicitly manage resiliency and message bus distribution capabilities when using these protocols. \n References: \n https://core.ac.uk/download/pdf/160743474.pdf \n \n 2. Architecture \n 2.1. Overall Architecture \n \n \n \n \n 2.2. Terminology \n 2.2.1. Topic \n An MQTT Topic is an identifier or a filter for the message. \n For example, a temperature sensor can send a message with the following topic “temperature/house/floor1/room3”. \n \n 2.2.2. Message \n The message is the payload of the MQTT publication for a specific topic. \n Following the same temperature example, the message can be “20 degrees Celsius” \n \n 2.2.3. Subscriber \n An MQTT subscriber “subscribes” to particular topics available on a MQTT Broker and can read the related messages. \n A subscriber can ask for specific topics “temperature/house/floor1/room3” or wildcard the topics to get all messages under a specific topic level. For example: I can read all MQTT messages related to temperature by subscribing to “temperature/#” \n \n 2.2.4. Publisher \n An MQTT publisher sends messages with a topic to a MQTT Broker. \n \n 2.2.5. Broker \n A broker is a server that receives MQTT messages as input from publishers and in turn publish them to subscribers. \n \n 2.2.6. Who is the Client, who is the Server? \n A client is a device or an application, such as an IoT device or a mobile app, making an MQTT connection to the broker. \n \n Clients and Servers can be either MQTT publishers, subscribers, or both. \n \n 2.3. Traffic overview \n \n \n \n \n \n \n 2.4. MQTT Quality of Service (QoS) \n Unfortunately, the reality is we are delivering messages to and from constrained devices over unreliable networks. When consider a temperature sensor, it may be not so catastrophic to miss a temperature measurement within a 5-minute time slot, but for other use cases like healthcare or automotive, omissions can have tragic consequences. Therefore, we need to consider Quality of Service (QoS) levels that can be associated with MQTT implementations. \n MQTT has 3 levels of QoS: \n - QoS 0 – “at most once” aka fire and forget: the message is sent without confirmation or follow-up \n \n \n \n \n - QoS 1 – “at least once”: the publisher keeps a copy of the published message until it receives a PUBACK confirmation reception message from the broker. If it does not receive a successful receipt message from the broker before expiration of a timeout period, it resends the message. \n \n \n \n \n \n - QoS 2 – “exactly once”: when the publisher sends a message to the broker, it expects a PUBREC acknowledgement. On receiving the PUBREC, the sender removes the packet and sends a PUBREL to tell the broker to release the message If it does not receive any during a timeout period, it sends the PUBLISH packet with a duplicate flag (DUP) \n \n \n \n So, how to select an appropriate QoS value? First, I would say: just put “2” and add to it “which can do more can do less” but I think again at the constrained devices. The higher the QoS level, the more bandwidth and compute it requires. \n Here are few tips on how to choose the best QoS: \n - QoS0 is faster than QoS1 which is faster than QoS2. \n - Use QoS0 if you have a very reliable network. \n - QoS1 tolerates duplicate messages. \n - Use QoS0 when you don’t mind losing some message occasionally. \n \n \n 2.5. Load Balancing of brokers \n Of course, like any networking service, MQTT Brokers should be deployed in N redundant instances to provide capacity and high availability. \n As with any stateful protocol you try to load balance, you need stickiness or persistency to make sure a single MQTT publisher sticks for the same connection to the same broker. \n What happens when you do not have an appropriate session persistence mechanism and you have set up QoS? You cannot rely completely on source IP address as a persistence criterion, as connected devices can roam across networks thus present themselves with a different IP address and be connected to a different broker every time. \n \n 2.5.1. Extend Natively with NGINX \n NGINX can get the client_id from the MQTT packet and persist on it. \n NGINX can substitute the client ID during the MQTT CONNECT message, which most of the time should be a unique identifier such as the serial number of the device combined with a different identifier (such as the client SSL certificate serial number). \n stream {\n mqtt on;\n\n server {\n listen 2883 ssl;\n ssl_certificate /etc/nginx/certs/emqx.pem;\n ssl_certificate_key /etc/nginx/certs/emqx.key;\n ssl_client_certificate /etc/nginx/certs/ca.crt; \n ssl_session_cache shared:SSL:10m;\n ssl_verify_client on;\n proxy_pass 10.0.0.113:1883;\n proxy_connect_timeout 1s; \n\n mqtt_set_connect clientid $ssl_client_serial;\n }\n}\n \n \n 2.5.2. Extend with NJS \n njs is a subset of the JavaScript language that allows extending nginx functionality. The traditional use cases for njs are: \n \n Complex access control and security checks in njs before a request reaches an upstream server \n Manipulating response headers \n Writing flexible asynchronous content handlers and filters \n \n You can find a great example of MQTTv5 implementation extending the capabilities with NJS on Doug_Gallarda personal GitHub repository: https://github.com/gallarda/mqtt5 \n \n 3. MQTT Brokers \n 3.1. Most popular MQTT brokers \n The purpose of this document is not to compare MQTT brokers; simply introduce you to the concept. You can find plenty of comparisons on the Internet. For example, this one https://emqx.medium.com/a-comprehensive-comparison-of-open-source-mqtt-brokers-2023-e70257cc5b75 and https://www.emqx.com/en/blog/open-mqtt-benchmarking-comparison-mqtt-brokers-in-2023 which were both written by EMQ Technologies. I am not making any bias judgement in the ranking, but rest assured you will find the content to be flush with detail. \n Amongst the most popular MQTT brokers we have the following: \n \n Mosquitto \n HiveMQ \n EMQX \n Solace \n \n Depending on the solution you choose, licensing may be set per broker host, per connections and/or messages per second. \n Beyond sharing message processing loads evenly between brokers, an intermediate proxy can block undesirable and unauthenticated connections attempts, filter legitimate and well-structured messages so you pay only for real production traffic. \n \n 3.1. Why proxy MQTT traffic? \n In 2022, the Eclipse Foundation published an IoT & Edge Developer Survey Report which highlighted some concerns related to this. Examine the following chart presented in this report: \n \n The Eclipse foundation explains these numbers by: \n \n An increase in connectivity concerns underscores the lack of computational capacity for efficient built-in security. \n Security still resembles major concern despite the percentage drop (from 46% in 2021) \n A decrease in deployment-related concerns (from 31% to 20% in 2022) indicates that less solutions are moving past the PoC phase, and developers are focusing more on successful solutions rollout to assure overall better user experience. \n Concerns around integration complexity have also decreased (by 11% compared to 2021). As the number of deployments increase, developers see less complexity in the need for additional integrations with complementary technologies and systems. \n \n \n \n 4. MQTT Security \n \n 4.1. Security Context and Attack surface \n In early 2024, did you hear about the 3 million toothbrushes conducting a massive DDoS attack? (https://www.securityweek.com/3-million-toothbrushes-abused-for-ddos-attacks-real-or-not/). As it turns out, this was fake news. However, this scenario is certainly within the realm of possibility. \n In fact, massive DDoS attacks directed by botnets of IoTs is very common. Think about billions of devices (~25B in 2024 according to analysts) with a fair portion of them being completely vulnerable: \n \n Not encrypted \n Never updated \n Poor code \n \n I am not saying it happens because of a lack of security awareness, knowledge or skills, I honestly think it is because of a trade-off engineers made between the technical overhead of security requirements and: \n \n The lightness of the MQTT protocol in terms of hardware requirements for clients. Due to their reduced form factor, their power requirements and price, MQTT devices are generally constrained devices as per RFC 7228 which cannot handle optimal TLS processing. \n The large number of devices makes it difficult to manage the security settings, updates and credentials management over-the-air at scale. \n \n Of course, sometimes there is also a lack of security awareness and the appreciation of the impact of a cyber-attack. \n \n 4.2. IoT devices generally don’t encrypt communication \n This is a bold claim. How do we know this? Leverage one of the first steps employed by ethical hackers: reconnaissance! \n Let’s search for opened non encrypted MQT devices on Internet. How? There are various search engines that discover and index internet-connected devices. Shodan is one of such services. You can run a search on Shodan for: MQTT port: 1883 code:0 and see results like the following: \n \n Notes: \n - 1883 is the default port for unencrypted MQTT. \n - Code:0 indicates unauthenticated \n I was looking into whitepapers on MQTT security, and I found several publications with Shodan outputs April 27 th , 2017 = 24,998 brokers with default ports successfully indexed by Shodan (https://www.researchgate.net/figure/Result-of-MQTT-broker-on-port-1883-in-Shodan_fig2_322059897) and they were just looking for MQTT brokers on port 1883 regardless of their Connection code. \n Now (February 29 th , 2024), and only unauthenticated brokers, I get x19 results. \n Now, second step is looking for authentication, any brokers having a connection code of 0 will accept any clients without any authentication. Again, any client means both publishers AND subscribers. In this case, you can pretty much choose what you want to exploit: \n \n Read for confidential data (you can subscribe to all topics with “#”) \n Drain out messages from brokers (you can ACK messages, so they are deleted from brokers). \n Spam the subscribers by publishing wrong messages. \n Flood with bad messages (i.e. L7 DDoS: search for mqtt malaria on google, there are plenty of open MQTT “stress” tools L). \n \n \n \n 4.3. Authentication and access control \n MQTT brokers supports username password authentication. If no credentials are provided, you will get a connection code of 5, if your credentials are wrong the connection code will be 4. \n Again, if traffic is not encrypted, every part of your MQTT packet is in clear text including username and password. \n Like any username/password authenticated application, brokers are subject to brute force, directory or credential stuffing attacks. \n \n \n 4.4. Injections \n So far, we have discussed the implications of attacks on the publisher, broker and subscriber components. What are some secondary consequences? \n \n A subscriber will likely store the message data somewhere in a database. SQL database? J \n Unfiltered message data may be presented in a modern webpage? J \n MQTT is a TCP messaging protocol that can’t be inspected by traditional L7 security solutions? J \n \n I did a small test with a simple paho python (https://pypi.org/project/paho-mqtt/) MQTT temperature app (reach out if you want the code). \n Note: \n Paho provides a client class which enables applications to connect to an MQTT broker to publish messages, and to subscribe to topics and receive published messages. It also provides some helper functions to make publishing one off messages to an MQTT server very straightforward. \n I was too lazy to set up a SQL database and store data into it. So, I am just sharing the JSON received data from the python code to Flask. So no SQL Injection for today,… XSS will be enough. \n \n \n \n \n And, here what I got on the web app rendering the temperature: \n \n The XSS attack has been successfully made its way until the web application without being identified and intercepted because it have been encapsulated in a MQTT message which cannot be inspected natively by a WAF. \n \n 5. How can NGINX enrich your MQTT use cases? \n 5.1. What is NGINX? \n NGINX is a lightweight and highly performant software-based load balancer and reverse proxy. It natively supports multiple protocols like HTTP for general-purpose web and API traffic but also more specific protocols like MQTT. \n NGINX possesses many built-in features for traffic processing and handling. However, it can also be extended by the usage of its Javascript scripting engine, called NJS (https://nginx.org/en/docs/njs/). NJS can be used to take actions like steering traffic or rewriting packet contents based on multiple conditions. \n \n There are multiple use cases where NGINX, either natively or by extending the capabilities using NJS for parsing and taking decision: \n \n 5.2.Traffic optimisation \n \n Reduce latency \n Uniform load balancing of brokers \n Steering messages to brokers based on topic \n Steering messages to brokers based on QoS \n Steering messages to brokers based on QoS \n Steering messages to brokers based on message content \n ... \n \n \n 5.3.MQTT Security \n \n TLS Offload \n Client Authentication offload \n Client Authorization (filter only to publish not to subscribe for example). \n Filtering broker fingerprinting attempts \n Filtering unwanted information ($SYS) \n Inspecting messages for injection attempts. \n \n \n 5.4. Analytics and Telemetry \n NGINX provides an OpenTelemetry module to help you analyze your software performance by instrumenting, generating, collecting, and exporting telemetry data. \n https://docs.nginx.com/nginx/admin-guide/dynamic-modules/opentelemetry/ \n ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"23051","kudosSumWeight":0,"repliesCount":1,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktT3VGNjB1?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktR3NxVEhC?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktMElCb2FU?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktdDR2WEdO?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktaWRnazBx?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktQUJLbmtU?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktV25sYUhq?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDg","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktM0NDUXdB?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDk","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTkteVNkSkU0?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDEw","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktWVVDRFIw?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDEx","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktbXp3OWlm?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDEy","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktQlVhQmRt?revision=4\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDEz","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzIwMTktZTdPWHdC?revision=4\"}"}}],"totalCount":13,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:334519":{"__typename":"Conversation","id":"conversation:334519","topic":{"__typename":"TkbTopicMessage","uid":334519},"lastPostingActivityTime":"2024-10-09T10:28:39.575-07:00","solved":false},"User:user:194786":{"__typename":"User","uid":194786,"login":"Janibasha","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0xOTQ3ODYtMjA5NDJpMEI1Q0JDRDNGRkQ2MUM0Mw"},"id":"user:194786"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktOUVjdU9S?revision=7\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktOUVjdU9S?revision=7","title":"no-dataguard-result.png","associationType":"BODY","width":3122,"height":1054,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktZ0hwcUJs?revision=7\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktZ0hwcUJs?revision=7","title":"image.png","associationType":"BODY","width":1795,"height":1537,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktbU9SbVI1?revision=7\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktbU9SbVI1?revision=7","title":"owasp-llm-nap.jpeg","associationType":"BODY","width":1670,"height":660,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktRXcwUTBy?revision=7\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktRXcwUTBy?revision=7","title":"svc.JPG","associationType":"BODY","width":3213,"height":614,"altText":""},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktRXRHaUxX?revision=7\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktRXRHaUxX?revision=7","title":"llm-ssn-hidden.jpg","associationType":"BODY","width":3663,"height":1709,"altText":""},"TkbTopicMessage:message:334519":{"__typename":"TkbTopicMessage","subject":"Mitigate OWASP LLM Security Risk: Sensitive Information Disclosure Using F5 NGINX App Protect","conversation":{"__ref":"Conversation:conversation:334519"},"id":"message:334519","revisionNum":7,"uid":334519,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:194786"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":" This short WAF security article covered the critical security gaps present in current generative AI applications, emphasizing the urgent need for robust protection measures in LLM design deployments. Finally we also demonstrated how F5 Nginx App Protect v5 offers an effective solution to mitigate the OWASP LLM Top 10 risks. ","introduction":"","metrics":{"__typename":"MessageMetrics","views":328},"postTime":"2024-10-07T05:00:00.027-07:00","lastPublishTime":"2024-10-09T10:28:39.575-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Introduction: \n This article covers the basics of the rise of AI (Gen AI) and how F5 products can protect these AI backends. The rise of Large Language Models (LLMs) has marked a transformative era in AI, enabling machines to produce and comprehend text with human-like proficiency. These sophisticated models are now integral to applications in customer support, content creation, and even scientific research. However, their advanced capabilities also cause big security worries, especially around accidentally sharing sensitive information. These models can sometimes share private data from their training data. This means we need strong protection systems to reduce these risks. Addressing these challenges, the OWASP LLM Top 10 project has been created to identify and prioritize the most critical security threats associated with LLMs. Out of this top 10, LLM-06 risk specifically focuses on sensitive information disclosure, emphasizing the importance of stringent data handling protocols and privacy safeguards to prevent unintended data leaks to ensure the secure and ethical use of LLM technology. In this article, we are going to see how F5 Nginx App Protect v5 can protect the LLM backends from LLM06: Sensitive Information Disclosure risk. \n \n Use case: \n We are going to deploy a Gen AI application which takes URL hosting data and is passed on to a backend LLM application. Once data is analyzed by LLM, users can ask questions about this data and LLM will come back with the right answers. We have deployed this application inside AWS EKS cluster and 2 application services are running inside this cluster. Front-end services serve the UI, and the backend hosts the LLM model. How and what of this application with its internal tools and LLM model is not of importance and can find many free tools online. Since this article focusses on LLM06: Sensitive Information Disclosure risk, we will pass a website URL containing some dummy SSN of random users. Once this website data is loaded to LLM, we can ask for the SSN of a user and LLM will return the SSN from this data. SSNs are sensitive information and should always be protected as it will lead to personal data exploitation. In this case, LLM model does not have security rules to find and protect this data. So, it will be directly shown in the response as shown below. \n \n To protect this LLM backend service, we are going to deploy and configure NGINX App Protect Version 5 as a k8s workload in the data path. The latest release of NGINX App Protect v5 has made the WAF process more ridiculously easy, thereby making it more efficient and optimized. All data traffic will be validated by the NGINX App Protect before being exposed in the response. In this use case, since we want to mask the SSN, we are going to configure the data-guard feature with its appropriate configuration files onboarded to this container. \n \n Above configuration file can be downloaded from NGINX App Protect WAF configuration guide. \n Data guard is a WAF feature which detects and masks Credit Card Number (CCN) and/or U.S. Social Security Number (SSN) and/or custom patterns in HTTP responses. Since data-guard feature is enabled, SSNs of users in the LLM backend response are detected and masked by the NGINX App Protect thereby protecting the personal data. For more info on NGINX App Protect data guard feature, check this link. NOTE: Since this is just for demo and focuses only on LLM workload protection, we are using NGINX App Protect v5. But as per customer practices, users can configure NGINX Ingress Controller, Secure Mesh, etc. \n \n \n Deployment Steps: \n \n Check the service cluster IP of backend LLM service and update it in below yaml file upstream server to create nginx config file configmap apiVersion: v1\nkind: ConfigMap\nmetadata:\n name: nginx-conf-map-api\n namespace: default\ndata:\n nginx.conf: |\n user nginx;\n\n worker_processes auto;\n load_module modules/ngx_http_app_protect_module.so;\n\n error_log /var/log/nginx/error.log debug;\n\n events {\n worker_connections 10240;\n }\n\n http {\n include /etc/nginx/mime.types;\n default_type application/octet-stream;\n sendfile on;\n keepalive_timeout 65;\n \n # NGINX App Protect WAF\n app_protect_enforcer_address 127.0.0.1:50000;\n\n upstream main_DNS_name {\n server 172.20.41.242:8000;\n }\n server {\n listen 80;\n proxy_http_version 1.1;\n proxy_read_timeout 600;\n proxy_connect_timeout 600;\n proxy_send_timeout 600;\n\n app_protect_enable on;\n app_protect_policy_file \"/etc/app_protect/bundles/NAP_API_Policy.tgz\";\n app_protect_security_log_enable on;\n app_protect_security_log log_all /etc/app_protect/bundles/security.log;\n\n location / {\n client_max_body_size 0;\n default_type text/html;\n # set your backend here\n proxy_pass http://main_DNS_name;\n proxy_set_header Host $host;\n }\n }\n }\n \n Build and push a NGINX Plus docker image to your private registry by following this link \n Copy your JWT token and run below command to create a k8s secret # kubectl create secret docker-registry regcred --docker-server=private-registry.nginx.com --docker-username=<JWT Token> --docker-password=none \n Check the below file, update API policy bundle URL in init container and docker image info in nginx container. Apply this file to install nginx deployment and pods. apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: nap5-deployment\nspec:\n selector:\n matchLabels:\n app: nap5\n replicas: 1\n template:\n metadata:\n labels:\n app: nap5\n spec:\n imagePullSecrets:\n - name: regcred\n initContainers:\n - name: init-fetchbundle\n image: curlimages/curl:8.9.1\n command: \n - sh\n - -c\n - |\n echo \"Downloading file...\"\n curl -vvv -L https://github.com/f5devcentral/f5-xc-terraform-examples/raw/main/workflow-guides/NAP_API_Policy.tgz -o /etc/app_protect/bundles/NAP_API_Policy.tgz \n volumeMounts:\n - name: app-protect-bundles\n mountPath: /etc/app_protect/bundles \n containers:\n - name: nginx\n image: <registry-url>:tag-name\n imagePullPolicy: IfNotPresent\n volumeMounts:\n - name: app-protect-bd-config\n mountPath: /opt/app_protect/bd_config\n - name: app-protect-config\n mountPath: /opt/app_protect/config\n - name: nginx-conf-map-api-volume\n mountPath: /etc/nginx/nginx.conf\n subPath: nginx.conf\n - name: nap-api-policy-volume\n mountPath: /etc/nginx/NAP_API_Policy.json\n subPath: NAP_API_Policy.json \n \n - name: waf-enforcer\n image: private-registry.nginx.com/nap/waf-enforcer:5.2.0\n imagePullPolicy: IfNotPresent\n env:\n - name: ENFORCER_PORT\n value: \"50000\"\n volumeMounts:\n - name: app-protect-bd-config\n mountPath: /opt/app_protect/bd_config \n \n - name: waf-config-mgr\n image: private-registry.nginx.com/nap/waf-config-mgr:5.2.0\n imagePullPolicy: IfNotPresent\n securityContext:\n allowPrivilegeEscalation: false\n capabilities:\n drop:\n - all\n volumeMounts:\n - name: app-protect-bd-config\n mountPath: /opt/app_protect/bd_config\n - name: app-protect-config\n mountPath: /opt/app_protect/config\n - name: app-protect-bundles\n mountPath: /etc/app_protect/bundles\n \n volumes:\n - name: app-protect-bd-config\n emptyDir: {}\n - name: app-protect-config\n emptyDir: {}\n - name: app-protect-bundles\n emptyDir: {}\n - name: nginx-conf-map-api-volume\n configMap:\n name: nginx-conf-map-api\n - name: nap-api-policy-volume\n configMap:\n name: nap-api-policy \n Next deploy NGINX App Protect service using below file apiVersion: v1\nkind: Service\nmetadata:\n name: nap5\n labels:\n app: nap5\n service: nap5\nspec:\n ports:\n - protocol: TCP\n port: 80\n targetPort: 80\n selector:\n app: nap5\n type: ClusterIP \n \n \n Check the cluster services and copy the NGINX App Protect service cluster ip \n Update App Protect cluster ip address in your Gen AI application frontend yaml file openAI address to create deployment and load balancer service \n \n Testing: \n \n Once setup is complete, check the cluster services command and open the Gen AI front end load balancer service URL in a browser \n Enter the web page input as https://dlptest.com/sample-data/namessndob/ and in query provide \"What is Robert Aragon's SSN?\" \n After some time, validate that SSN number is masked in the response \n \n Conclusion: \n This article highlights the critical security gaps present in current Gen AI applications, emphasizing the urgent need for robust protection measures in LLM design deployments. In the latter half, we demonstrated how F5 NGINX App Protect v5, with its advanced security features, offers an effective solution to mitigate the OWASP LLM Top 10 risks. By leveraging these capabilities, organizations can significantly enhance the security and resilience of their AI applications. \n \n References: \n \n https://genai.owasp.org/llm-top-10/ \n https://genai.owasp.org/llmrisk/llm06-sensitive-information-disclosure/ \n https://docs.nginx.com/nginx-app-protect-waf/v5/admin-guide/deploy-on-kubernetes/ \n https://docs.nginx.com/nginx-app-protect-waf/v5/admin-guide/compiler/ \n \n \n NOTE: This article covered only one risk and stay tuned for more exciting articles on remaining OWASP LLM Top 10 risks prevention using F5 products. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"10168","kudosSumWeight":2,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktOUVjdU9S?revision=7\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktZ0hwcUJs?revision=7\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktbU9SbVI1?revision=7\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktRXcwUTBy?revision=7\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMzQ1MTktRXRHaUxX?revision=7\"}"}}],"totalCount":5,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:329488":{"__typename":"Conversation","id":"conversation:329488","topic":{"__typename":"TkbTopicMessage","uid":329488},"lastPostingActivityTime":"2024-07-01T08:58:54.246-07:00","solved":false},"User:user:419867":{"__typename":"User","uid":419867,"login":"Akash_Ananthanarayan","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS00MTk4NjctQ1IwTHhL?image-coordinates=0%2C420%2C1080%2C1500"},"id":"user:419867"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk0ODgtWXlPMzhq?revision=13\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk0ODgtWXlPMzhq?revision=13","title":"NGINXDemo.jpg","associationType":"TEASER","width":840,"height":723,"altText":""},"TkbTopicMessage:message:329488":{"__typename":"TkbTopicMessage","subject":"Safeguard your Apps and APIs using NGINX App Protect WAF 5.0","conversation":{"__ref":"Conversation:conversation:329488"},"id":"message:329488","revisionNum":13,"uid":329488,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:419867"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":" \n ","introduction":"","metrics":{"__typename":"MessageMetrics","views":282},"postTime":"2024-06-21T05:00:00.036-07:00","lastPublishTime":"2024-07-01T08:58:54.246-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Discover the latest features of NGINX App Protect 5.X and learn how it can secure your applications and APIs by checking out this article and demo video. \n Today’s dynamic business landscape requires organizations to run applications across various platforms, including cloud, on-premises, and hybrid environments. This has increased the demand for robust security solutions to effectively shield these applications from potential threats. The recent introduction of F5 NGINX App Protect version 5.0 in March 2024 addresses this need by presenting a unified, containerized security solution. This update aims to reduce the attack surface by minimizing dependencies and effectively decreasing vulnerability exposure. \n NGINX App Protect version 5.0 will not replace version 4.x. Instead, both versions will be maintained to meet various deployment needs. Version 4.x will continue as a packaged and updated product, while version 5.0 and later will be offered as containers compatible with Docker and Kubernetes. \n What is new in version 5.X? \n NGINX App Protect version 5.0 now includes support for NGINX OSS, which means that users can now access advanced security features without needing to transition to NGINX Plus. This update allows users to enhance the security of their applications while continuing to utilize NGINX OSS, offering flexibility and improved protection against potential threats. This advancement streamlines security configurations, resulting in time savings and reduced total cost of ownership, providing a more efficient and cost-effective solution. \n Our advanced container solution revolutionizes the process by eliminating the need to create a new containerized image for each signature update. This encompasses many threats, including attacks, malicious bots, APIs, and threat campaigns. Consequently, security operations are streamlined, compatibility with underlying operating systems is improved, and development cycles are enhanced. This gives DevOps teams the valuable opportunity to focus on other critical project elements, refining their workflow and overall efficiency. \n An exciting enhancement in the NGINX App Protect version 5.0 is the introduction of a distinct control plane. This upgrade means that policies are consolidated for easier management, supported by a dedicated compiler available for the SecOps team. This configuration empowers the team to swiftly and effectively modify and update policies, ensuring a more seamless and efficient data plane well-equipped to tackle any incoming challenges. \n NGINX App Protect version 5.0 is engineered to deliver exceptional performance in cloud and hybrid deployments. Its scalable architecture accommodates both small and large-scale deployments, and its seamless integration with existing DevOps and SecOps workflows streamlines implementation. \n \n This video will demonstrate how to install NGINX App Protect WAF 5.x and compile a new policy using the NGINX WAF compiler to secure your applications. \n \n \n Conclusion \n NGINX App Protect version 5.0 is a modern WAF designed with a modern architecture that is well-suited for contemporary application environments. This WAF enables DevOps teams to optimize their operations by effortlessly adopting container-ready formats, eliminating the requirement to manually create containerized images for each release and signature update. This streamlines the deployment process and enhances overall operational efficiency. \n To learn more about version 5.x release visit the latest blog, A Containerized WAF to Secure Apps and APIs. \n To get started, please visit our documentation page. \n Ready to experience NGINX App Protect v5.x? You can get started with a 30-day free trial or reach out to F5 sales for more information today. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"3832","kudosSumWeight":0,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0zMjk0ODgtWXlPMzhq?revision=13\"}"}}],"totalCount":1,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:290946":{"__typename":"Conversation","id":"conversation:290946","topic":{"__typename":"TkbTopicMessage","uid":290946},"lastPostingActivityTime":"2023-09-26T14:00:47.849-07:00","solved":false},"User:user:194331":{"__typename":"User","uid":194331,"login":"Mikhail_Fedoro1","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/m_assets/avatars/default/avatar-2.svg?time=0"},"id":"user:194331"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtMTA5NjZpN0RGQ0Y4MDA4OEU4MkFCNQ?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtMTA5NjZpN0RGQ0Y4MDA4OEU4MkFCNQ?revision=1","title":"0151T000002dlf8QAA.png","associationType":"BODY","width":2786,"height":1036,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtMTI2MDBpNUMwN0Q5RjhBNEUxNDI0NA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtMTI2MDBpNUMwN0Q5RjhBNEUxNDI0NA?revision=1","title":"0151T000002dlfDQAQ.png","associationType":"BODY","width":2784,"height":1180,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtNjQxNmk4MTBERjI4ODE5RkY0NEJC?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtNjQxNmk4MTBERjI4ODE5RkY0NEJC?revision=1","title":"0151T000002dlf9QAA.png","associationType":"BODY","width":2786,"height":1434,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtMjc0M2lGOTkxRTA4ODA1NjU4N0I2?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtMjc0M2lGOTkxRTA4ODA1NjU4N0I2?revision=1","title":"0151T000002dlfNQAQ.png","associationType":"BODY","width":2784,"height":1554,"altText":null},"TkbTopicMessage:message:290946":{"__typename":"TkbTopicMessage","subject":"Dashboards for NGINX App Protect","conversation":{"__ref":"Conversation:conversation:290946"},"id":"message:290946","revisionNum":1,"uid":290946,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:194331"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":2092},"postTime":"2020-06-22T12:40:36.000-07:00","lastPublishTime":"2020-06-22T12:40:36.000-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Introduction NGINX App Protect is a new generation WAF from F5 which is built in accordance with UNIX philosophy such that it does one thing well everything else comes from integrations. NGINX App Protect is extremely good at HTTP traffic security. It inherited a powerful WAF engine from BIG-IP and light footprint and high performance from NGINX. Therefore NGINX App Protect brings fine-grained security to all kinds of insertion points where NGINX use to be either on-premises or cloud-based. Therefore NGINX App Protect is a powerful and flexible security tool but without any visualization capabilities which are essential for a good security product. As mentioned above everything besides primary functionality comes from integrations. In order to introduce visualization capabilities, I've developed an integration between NGINX App Protect and ELK stack (Elasticsearch, Logstash, Kibana) as one of the most adopted stacks for log collection and visualization. Based on logs from NGINX App Protect ELK generates dashboards to clearly visualize what WAF is doing. Overview Dashboard Currently, there are two dashboards available. \"Overview\" dashboard provides a high-level view of the current situation and also allows to discover historical trends. You are free to select any time period of interest and filter data simply by clicking on values right on the dashboard. Table at the bottom of the dashboard lists all requests within a time frame and allows to see how exactly a request looked like. False Positives Dashboard Another useful dashboard called \"False Positives\" helps to identify false positives and adjust WAF policy based on findings. For example, the chart below shows the number of unique IPs that hit a signature. Under normal conditions when traffic is mostly legitimate \"per signature\" graphs should fluctuate around zero because legitimate users are not supposed to hit any of signatures. Therefore if there is a spike and amount of unique IPs which hit a signature is close to the total amount of sources then likely there is a false positive and policy needs to be adjusted. Conclusion This is an open-source and community-driven project. The more people contribute the better it becomes. Feel free to use it for your projects and contribute code or ideas directly to the repo. The plan is to make these dashboards suitable for all kinds of F5 WAF flavors including AWAF and EAP. This should be simple because it only requires logstash pipeline adjustment to unify logs format stored in elasticsearch index. If you have a project for AWAF or EAP going on and would like to use dashboards please feel free to develop and create a pull request with an adjusted logstash pipeline to normalize logs from other WAFs. Github repo: https://github.com/464d41/f5-waf-elk-dashboards Feel free to reach me with questions. Good luck! ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"2896","kudosSumWeight":4,"repliesCount":1,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtMTA5NjZpN0RGQ0Y4MDA4OEU4MkFCNQ?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtMTI2MDBpNUMwN0Q5RjhBNEUxNDI0NA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtNjQxNmk4MTBERjI4ODE5RkY0NEJC?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5NDYtMjc0M2lGOTkxRTA4ODA1NjU4N0I2?revision=1\"}"}}],"totalCount":4,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:285542":{"__typename":"Conversation","id":"conversation:285542","topic":{"__typename":"TkbTopicMessage","uid":285542},"lastPostingActivityTime":"2023-05-09T09:22:32.046-07:00","solved":false},"User:user:354380":{"__typename":"User","uid":354380,"login":"Romain","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/m_assets/avatars/default/avatar-4.svg?time=0"},"id":"user:354380"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItMzM3M2kwQjhEQURBODM5QjZGOUIw?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItMzM3M2kwQjhEQURBODM5QjZGOUIw?revision=1","title":"0151T000003pkE6QAI.jpg","associationType":"BODY","width":1364,"height":769,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItODA3Nmk1QkEzN0ZENTgzM0FFRkNG?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItODA3Nmk1QkEzN0ZENTgzM0FFRkNG?revision=1","title":"0151T000003pkE1QAI.jpg","associationType":"BODY","width":1215,"height":683,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNjQwN2k2RUY3ODQzMzE2RTkxNTJE?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNjQwN2k2RUY3ODQzMzE2RTkxNTJE?revision=1","title":"0151T000003pkDsQAI.jpg","associationType":"BODY","width":1215,"height":684,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNDIzNmkzOTYzNzM4MThENUJERThD?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNDIzNmkzOTYzNzM4MThENUJERThD?revision=1","title":"0151T000003pjaQQAQ.png","associationType":"BODY","width":1216,"height":515,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNjg0NWlDMjE0NEEzODkxNjdBNEY4?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNjg0NWlDMjE0NEEzODkxNjdBNEY4?revision=1","title":"0151T000003pkDrQAI.jpg","associationType":"BODY","width":1215,"height":542,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItMTE4MjFpQzVFNUIyMTIyQ0Y0MEQ1OQ?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItMTE4MjFpQzVFNUIyMTIyQ0Y0MEQ1OQ?revision=1","title":"0151T000003pkDwQAI.jpg","associationType":"BODY","width":1213,"height":715,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItOTYxM2lDN0Q1OTc3RTNBMTBCMjIw?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItOTYxM2lDN0Q1OTc3RTNBMTBCMjIw?revision=1","title":"0151T000003pkEBQAY.jpg","associationType":"BODY","width":1223,"height":549,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNDY4NGkyMDk3Mzc0NzQxQ0VBMTYz?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNDY4NGkyMDk3Mzc0NzQxQ0VBMTYz?revision=1","title":"0151T000003pkEGQAY.jpg","associationType":"BODY","width":1221,"height":721,"altText":null},"TkbTopicMessage:message:285542":{"__typename":"TkbTopicMessage","subject":"Implementing BIG-IP WAF logging and visibility with ELK","conversation":{"__ref":"Conversation:conversation:285542"},"id":"message:285542","revisionNum":1,"uid":285542,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:354380"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":13302},"postTime":"2020-09-21T10:36:50.000-07:00","lastPublishTime":"2020-09-21T10:36:50.000-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Scope This technical article is useful for BIG-IP users familiar with web application security and the implementation and use of the Elastic Stack. This includes, application security professionals, infrastructure management operators and SecDevOps/DevSecOps practitioners. The focus is for WAF logs exclusively. Firewall, Bot, or DoS mitigation logging into the Elastic Stack is the subject of a future article. Introduction This article focusses on the required configuration for sending Web Application Firewall (WAF) logs from the BIG-IP Advanced WAF (or BIG-IP ASM) module to an Elastic Stack (a.k.a. Elasticsearch-Logstash-Kibana or ELK). First, this article goes over the configuration of BIG-IP. It is configured with a security policy and a logging profile attached to the virtual server that is being protected. This can be configured via the BIG-IP user interface (TMUI) or through the BIG-IP declarative interface (AS3). The configuration of the Elastic Strack is discussed next. The configuration of filters adapted to processing BIP-IP WAF logs. Finally, the article provides some initial guidance to the metrics that can be taken into consideration for visibility. It discusses the use of dashboards and provides some recommendations with regards to the potentially useful visualizations. Pre-requisites and Initial Premise For the purposes of this article and to follow the steps outlined below, the user will need to have at least one BIG-IP Adv. WAF running TMOS version 15.1 or above (note that this may work with previous version but has not been tested). The target BIG-IP is already configured with: A virtual Server A WAF policy An operational Elastic Stack is also required. The administrator will need to have configuration and administrative privileges on both the BIG-IP and Elastic Stack infrastructure. They will also need to be familiar with the network topology linking the BIG-IP with the Elastic Search cluster/infrastructure. It is assumed that you want to use your Elastic Search (ELK) logging infrastructure to gain visibility into BIG-IP WAF events. Logging Profile Configuration An essential part of getting WAF logs to the proper destination(s) is the Logging Profile. The following will go over the configuration of the Logging Profile that sends data to the Elastic Stack. Overview of the steps: Create Logging Profile Associate Logging Profile with the Virtual Server After following the procedure below On the wire, logs lines sent from the BIG-IP are comma separated value pairs that look something like the sample below: Aug 25 03:07:19 localhost.localdomainASM:unit_hostname=\"bigip1\",management_ip_address=\"192.168.41.200\",management_ip_address_2=\"N/A\",http_class_name=\"/Common/log_to_elk_policy\",web_application_name=\"/Common/log_to_elk_policy\",policy_name=\"/Common/log_to_elk_policy\",policy_apply_date=\"2020-08-10 06:50:39\",violations=\"HTTP protocol compliance failed\",support_id=\"5666478231990524056\",request_status=\"blocked\",response_code=\"0\",ip_client=\"10.43.0.86\",route_domain=\"0\",method=\"GET\",protocol=\"HTTP\",query_string=\"name='\",x_forwarded_for_header_value=\"N/A\",sig_ids=\"N/A\",sig_names=\"N/A\",date_time=\"2020-08-25 03:07:19\",severity=\"Error\",attack_type=\"Non-browser Client,HTTP Parser Attack\",geo_location=\"N/A\",ip_address_intelligence=\"N/A\",username=\"N/A\",session_id=\"0\",src_port=\"39348\",dest_port=\"80\",dest_ip=\"10.43.0.201\",sub_violations=\"HTTP protocol compliance failed:Bad HTTP version\",virus_name=\"N/A\",violation_rating=\"5\",websocket_direction=\"N/A\",websocket_message_type=\"N/A\",device_id=\"N/A\",staged_sig_ids=\"\",staged_sig_names=\"\",threat_campaign_names=\"N/A\",staged_threat_campaign_names=\"N/A\",blocking_exception_reason=\"N/A\",captcha_result=\"not_received\",microservice=\"N/A\",tap_event_id=\"N/A\",tap_vid=\"N/A\",vs_name=\"/Common/adv_waf_vs\",sig_cves=\"N/A\",staged_sig_cves=\"N/A\",uri=\"/random\",fragment=\"\",request=\"GET /random?name=' or 1 = 1' HTTP/1.1\\r\\n\",response=\"Response logging disabled\"\n Please choose one of the methods below. The configuration can be done through the web-based user interface (TMUI), the command line interface (TMSH), directly with a declarative AS3 REST API call, or with the BIG-IP native REST API. This last option is not discussed herein. TMUI Steps: Create Profile Connect to the BIG-IP web UI and login with administrative rights Navigate to Security >> Event Logs >> Logging Profiles Select “Create” Fill out the configuration fields as follows: Profile Name (mandatory) Enable Application Security Set Storage Destination to Remote Storage Set Logging Format to Key-Value Pairs (Splunk) In the Server Addresses field, enter an IP Address and Port then click on Add as shown below: Click on Create Add Logging Profile to virtual server with the policy Select target virtual server and click on the Security tab (Local Traffic >> Virtual Servers : Virtual Server List >> [target virtualserver] ) Highlight the Log Profile from the Available column and put it in the Selected column as shown in the example below (log profile is “log_all_to_elk”): Click on Update At this time the BIG-IP will forward logs Elastic Stack. TMSH Steps: Create profile ssh into the BIG-IP command line interface (CLI) from the tmsh prompt enter the following: create security log profile [name_of_profile] application add { [name_of_profile] { logger-type remote remote-storage splunk servers add { [IP_address_for_ELK]:[TCP_Port_for_ELK] { } } } }\n For example: create security log profile dc_show_creation_elk application add { dc_show_creation_elk { logger-type remote remote-storage splunk servers add { 10.45.0.79:5244 { } } } }\n 3. ensure that the changes are saved: save sys config partitions all\n Add Logging Profile to virtual server with the policy 1. From the tmsh prompt (assuming you are still logged in) enter the following: modify ltm virtual [VS_name] security-log-profiles add { [name_of_profile] }\n For example: modify ltm virtual adv_waf_vs security-log-profiles add { dc_show_creation_elk }\n 2. ensure that the changes are saved: save sys config partitions all\n At this time the BIG-IP sends logs to the Elastic Stack. AS3 Application Services 3 (AS3) is a BIG-IP configuration API endpoint that allows the user to create an application from the ground up. For more information on F5’s AS3, refer to link. In order to attach a security policy to a virtual server, the AS3 declaration can either refer to a policy present on the BIG-IP or refer to a policy stored in XML format and available via HTTP to the BIG-IP (ref. link). The logging profile can be created and associated to the virtual server directly as part of the AS3 declaration. For more information on the creation of a WAF logging profile, refer to the documentation found here. The following is an example of a pa rt of an AS3 declaration that will create security log profile that can be used to log to Elastic Stack: \"secLogRemote\": {\n \"class\": \"Security_Log_Profile\",\n \"application\": {\n \"localStorage\": false,\n \"maxEntryLength\": \"10k\",\n \"protocol\": \"tcp\",\n \"remoteStorage\": \"splunk\",\n \"reportAnomaliesEnabled\": true,\n \"servers\": [\n {\n \"address\": \"10.45.0.79\",\n \"port\": \"5244\"\n }\n ]\n }\n In the sample above, the ELK stack IP address is 10.45.0.79 and listens on port 5244 for BIG-IP WAF logs. Note that the log format used in this instance is “Splunk”. There are no declared filters and thus, only the illegal requests will get logged to the Elastic Stack. A sample AS3 declaration can be found here. ELK Configuration The Elastic Stack configuration consists of creating a new input on Logstash. This is achieved by adding an input/filter/ output configuration to the Logstash configuration file. Optionally, the Logstash administrator might want to create a separate pipeline – for more information, refer to this link. The following is a Logstash configuration known to work with WAF logs coming from BIG-IP: input {\n syslog {\n port => 5244\n }\n}\nfilter {\n grok {\n match => {\n \"message\" => [\n \"attack_type=\\\"%{DATA:attack_type}\\\"\",\n \",blocking_exception_reason=\\\"%{DATA:blocking_exception_reason}\\\"\",\n \",date_time=\\\"%{DATA:date_time}\\\"\",\n \",dest_port=\\\"%{DATA:dest_port}\\\"\",\n \",ip_client=\\\"%{DATA:ip_client}\\\"\",\n \",is_truncated=\\\"%{DATA:is_truncated}\\\"\",\n \",method=\\\"%{DATA:method}\\\"\",\n \",policy_name=\\\"%{DATA:policy_name}\\\"\",\n \",protocol=\\\"%{DATA:protocol}\\\"\",\n \",request_status=\\\"%{DATA:request_status}\\\"\",\n \",response_code=\\\"%{DATA:response_code}\\\"\",\n \",severity=\\\"%{DATA:severity}\\\"\",\n \",sig_cves=\\\"%{DATA:sig_cves}\\\"\",\n \",sig_ids=\\\"%{DATA:sig_ids}\\\"\",\n \",sig_names=\\\"%{DATA:sig_names}\\\"\",\n \",sig_set_names=\\\"%{DATA:sig_set_names}\\\"\",\n \",src_port=\\\"%{DATA:src_port}\\\"\",\n \",sub_violations=\\\"%{DATA:sub_violations}\\\"\",\n \",support_id=\\\"%{DATA:support_id}\\\"\",\n \"unit_hostname=\\\"%{DATA:unit_hostname}\\\"\",\n \",uri=\\\"%{DATA:uri}\\\"\",\n \",violation_rating=\\\"%{DATA:violation_rating}\\\"\",\n \",vs_name=\\\"%{DATA:vs_name}\\\"\",\n \",x_forwarded_for_header_value=\\\"%{DATA:x_forwarded_for_header_value}\\\"\",\n \",outcome=\\\"%{DATA:outcome}\\\"\",\n \",outcome_reason=\\\"%{DATA:outcome_reason}\\\"\",\n \",violations=\\\"%{DATA:violations}\\\"\",\n \",violation_details=\\\"%{DATA:violation_details}\\\"\",\n \",request=\\\"%{DATA:request}\\\"\"\n ]\n }\n break_on_match => false\n }\n mutate {\n split => { \"attack_type\" => \",\" }\n split => { \"sig_ids\" => \",\" }\n split => { \"sig_names\" => \",\" }\n split => { \"sig_cves\" => \",\" }\n split => { \"staged_sig_ids\" => \",\" }\n split => { \"staged_sig_names\" => \",\" }\n split => { \"staged_sig_cves\" => \",\" }\n split => { \"sig_set_names\" => \",\" }\n split => { \"threat_campaign_names\" => \",\" }\n split => { \"staged_threat_campaign_names\" => \",\" }\n split => { \"violations\" => \",\" }\n split => { \"sub_violations\" => \",\" }\n }\n if [x_forwarded_for_header_value] != \"N/A\" {\n mutate { add_field => { \"source_host\" => \"%{x_forwarded_for_header_value}\"}}\n } else {\n mutate { add_field => { \"source_host\" => \"%{ip_client}\"}}\n }\n geoip {\n source => \"source_host\"\n }\n}\noutput {\n elasticsearch {\n hosts => ['localhost:9200']\n index => \"big_ip-waf-logs-%{+YYY.MM.dd}\"\n }\n}\n After adding the configuration above to the Logstash parameters, you will need to restart the Logstash instance to take the new logs into configuration. The sample above is also available here. The Elastic Stack is now ready to process the incoming logs. You can start sending traffic to your policy and start seeing logs populating the Elastic Stack. If you are looking for a test tool to generate traffic to your Virtual Server, F5 provides a simple WAF tester tool that can be found here. At this point, you can start creating dashboards on the Elastic Stack that will satisfy your operational needs with the following overall steps: · Ensure that the log index is being created (Stack Management >> Index Management) · Create a Kibana Index Pattern (Stack Management>>Index patterns) · You can now peruse the logs from the Kibana discover menu (Discover) · And start creating visualizations that will be included in your Dashboards (Dashboards >> Editing Simple WAF Dashboard) A complete Elastic Stack configuration can be found here – note that this can be used with both BIG-IP WAF and NGINX App Protect. Conclusion You can now leverage the widely available Elastic Stack to log and visualize BIG-IP WAF logs. From dashboard perspective it may be useful to track the following metrics: -Request Rate -Response codes -The distribution of requests in term of clean, blocked or alerted status -Identify the top talkers making requests -Track the top URL’s being accessed -Top violator source IP An example or the dashboard might look like the following: ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"14690","kudosSumWeight":5,"repliesCount":6,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItMzM3M2kwQjhEQURBODM5QjZGOUIw?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItODA3Nmk1QkEzN0ZENTgzM0FFRkNG?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNjQwN2k2RUY3ODQzMzE2RTkxNTJE?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNDIzNmkzOTYzNzM4MThENUJERThD?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNjg0NWlDMjE0NEEzODkxNjdBNEY4?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItMTE4MjFpQzVFNUIyMTIyQ0Y0MEQ1OQ?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItOTYxM2lDN0Q1OTc3RTNBMTBCMjIw?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDg","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODU1NDItNDY4NGkyMDk3Mzc0NzQxQ0VBMTYz?revision=1\"}"}}],"totalCount":8,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:290908":{"__typename":"Conversation","id":"conversation:290908","topic":{"__typename":"TkbTopicMessage","uid":290908},"lastPostingActivityTime":"2022-12-15T16:48:46.454-08:00","solved":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtMjEyMzhpRERCN0RDMzkzOUNBN0IwMg?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtMjEyMzhpRERCN0RDMzkzOUNBN0IwMg?revision=2","title":"nginx-appProtectDoS.png","associationType":"BODY","width":916,"height":509,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtMzcyN2lDMkQ4NEMzMDI0QjdCOTcy?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtMzcyN2lDMkQ4NEMzMDI0QjdCOTcy?revision=2","title":"0151T0000040hXxQAI.png","associationType":"BODY","width":1550,"height":512,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtOTczMWk4QjJCNDgzQkYwOUM0M0I2?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtOTczMWk4QjJCNDgzQkYwOUM0M0I2?revision=2","title":"0151T0000040hXnQAI.png","associationType":"BODY","width":1550,"height":637,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtMTIzNzJpOThCQTQ1NUNFNTgzRjRERg?revision=2\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtMTIzNzJpOThCQTQ1NUNFNTgzRjRERg?revision=2","title":"0151T0000040hY2QAI.png","associationType":"BODY","width":1550,"height":207,"altText":null},"TkbTopicMessage:message:290908":{"__typename":"TkbTopicMessage","subject":"L7 DoS Protection with NGINX App Protect DoS","conversation":{"__ref":"Conversation:conversation:290908"},"id":"message:290908","revisionNum":2,"uid":290908,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:194331"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":4547},"postTime":"2021-07-26T08:28:57.000-07:00","lastPublishTime":"2022-12-15T16:48:46.454-08:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Intro \n NGINX security modules ecosystem becomes more and more solid. Current App Protect WAF offering is now extended by App Protect DoS protection module. App Protect DoS inherits and extends the state-of-the-art behavioral L7 DoS protection that was initially implemented on BIG-IP and now protects thousands of workloads around the world. \n In this article, I’ll give a brief explanation of underlying ML-based DDoS prevention technology and demonstrate few examples of how precisely it stops various L7 DoS attacks. \n Technology \n It is important to emphasize the difference between the general volumetric-based protection approach that most of the market uses and ML-based technology that powers App Protect DoS. \n Volumetric-based DDoS protection is an old and well-known mechanism to prevent DDoS attacks. As the name says, such a mechanism counts the number of requests sharing the same source or destination, then simply drops or applies rate-limiting after some threshold crossed. For instance, requests sourcing the same IP are dropped after 100 RPS, requests going to the same URL after 200 RPS, and rate-limiting kicks in after 500 RPS for the entire site. \n Obviously, the major drawback of such an approach is that the selection criterion is too rough. It can cause erroneous drops of valid user requests and overall service degradation. The phenomenon when a security measure blocks good requests is called a “false positive”. \n App Protect DoS implements much more intelligent techniques to detect and fight off DDoS attacks. At a high level, it monitors all ongoing traffic and builds a statistical model in other words a baseline in a process called “learning”. The learning process almost never stops, therefore a baseline automatically adjusts to the current web application layout, a pattern of use, and traffic intensity. This is important because it drastically reduces maintenance cost and reaction speed for the solution. There is no more need to manually customize protection configuration for every application or traffic change. \n Infinite learning produces a legitimate question. Why can’t the system learn attack traffic as a baseline and how does it detect an attack then? To answer this question let us define what a DDoS attack is. A DDoS attack is a traffic stream that intends to deny or degrade access to a service. Note, the definition above doesn’t focus on the amount of traffic. ‘Low and slow’ DDoS attacks can hurt a service as severely as volumetric do. Traffic is only considered malicious when a service level degrades. So, this means that attack traffic can become a baseline, but it is not a big deal since protected service doesn’t suffer. \n Now only the “service degradation” term separates us from the answer. How does the App Protect DoS measure a service degradation? As humans, we usually measure the quality of a web service in delays. The longer it takes to get a response the more we swear. App Protect DoS mimics human behavior by measuring latency for every single transaction and calculates the level of stress for a service. If overall stress crosses a threshold App Protect DoS declares an attack. Think of it; a service degradation triggers an attack signal, not a traffic volume. Volume is harmless if an application server manages to respond quickly. \n Nice! The attack is detected for a solid reason. What happens next? First of all, the learning process stops and rolls back to a moment when the stress level was low. The statistical model of the traffic that was collected during peacetime becomes a baseline for anomaly detection. App Protect DoS keeps monitoring the traffic during an attack and uses machine learning to identify the exact request pattern that causes a service degradation. Opposed to old-school volumetric techniques it doesn’t use just a single parameter like source IP or URL, but actually builds as accurate as possible signature of entire request that causes harm. The overall number of parameters that App Protect DoS extracts from every request is in the dozens. A signature usually contains about a dozen including source IP, method, path, headers, payload content structure, and others. Now you can see that App Protect DoS accuracy level is insane comparing to volumetric vectors. \n The last part is mitigation. App Protect DoS has a whole inventory of mitigation tools including accurate signatures, bad actor detection, rate-limiting or even slowing down traffic across the board, which it uses to return service. The strategy of using those is convoluted but the main objective is to be as accurate as possible and make no harm to valid users. In most cases, App Protect DoS only mitigates requests that match specific signatures and only when the stress threshold for a service is crossed. Therefore, the probability of false positives is vanishingly low. \n The additional beauty of this technology is that it almost doesn’t require any configuration. Once enabled on a virtual server it does all the job \"automagically\" and reports back to your security operation center. The following lines present a couple of usage examples. \n Demo \n Demo topology is straightforward. On one end I have a couple of VMs. One of them continuously generates steady traffic flow simulating legitimate users. The second one is supposed to generate various L7 DoS attacks pretending to be an attacker. On the other end, one VM hosts a demo application and another one hosts NGINX with App Protect DoS as a protection tool. VM on a side runs ELK cluster to visualize App Protect DoS activity. \n \n Workflow of a demo aims to showcase a basic deployment example and overall App Protect DoS protection technology. First, I’ll configure NGINX to forward traffic to a demo application and App Protect DoS to apply for DDoS protection. Then a VM that simulates good users will send continuous traffic flow to App Protect DoS to let it learn a baseline. Once a baseline is established attacker VM will hit a demo app with various DoS attacks. While all this battle is going on our objective is to learn how App Protect DoS behaves, and that good user's experience remains unaffected. \n Similar to App Protect WAF App Protect DoS is implemented as a separate module for NGINX. It installs to a system as an apt/yum package. Then hooks into NGINX configuration via standard “load_module” directive. \n load_module modules/ngx_http_app_protect_dos_module.so;\n \n Once loaded protection enables under either HTTP, server, or location sections. Depending on what would you like to protect. \n app_protect_dos_enable [on|off]\n \n By default, App Protect DoS takes a protection configuration from a local policy file “/etc/nginx/BADOSDefaultPolicy.json” \n { \"mitigation_mode\" : \"standard\", \"use_automation_tools_detection\": \"on\", \"signatures\" : \"on\", \"bad_actors\" : \"on\" }\n \n As I mentioned before App Protect DoS doesn’t require complex config and only takes four parameters. Moreover, default policy covers most of the use cases therefore, a user only needs to enable App Protect DoS on a protected object. \n The next step is to simulate good users’ traffic to let App Protect DoS learn a good traffic pattern. I use a custom bash script that generates about 6-8 requests per second like an average surfing activity. \n While inspecting traffic and building a statistical model of good traffic App Protect DoS sends logs and metrics to Elasticsearch so we can monitor all its activity. \n \n The dashboard above represents traffic before/after App Protect DoS, degree of application stress, and mitigations in place. Note that the rate of client-side transactions matches the rate of server-side transactions. Meaning that all requests are passing through App Protect DoS and there are not any mitigations applied. Stress value remains steady since the backend easily handles the current rate and latency does not increase. \n \n Now I am launching an HTTP flood attack. It generates several thousands of requests per second that can easily overwhelm an unprotected web server. My server has App Protect DoS in front applying all its’ intelligence to fight off the DoS attack. After a few minutes of running the attack traffic, the dashboard shows the following situation. \n \n The attack tool generated roughly 1000 RPS. Two charts on the left-hand side show that all transactions went through App Protect DoS and were reaching a demo app for a couple of minutes causing service degradation. Right after service stress has reached a threshold an attack was declared (vertical red line on all charts). \n As soon as the attack has been declared App Protect DoS starts to apply mitigations to resume the service back to life. As I mentioned before App Protect DoS tries its best not to harm legitimate traffic. Therefore, it iterates from less invasive mitigations to more invasive. During the first several seconds when App Protect DoS just detected an attack and specific anomaly signature is not calculated yet. App Protect DoS applies an HTTP redirect to all requests across the board. Such measure only adds a tiny bit of latency for a web browser but allows it to quickly filter out all not-so-intelligent attack tools that can’t follow redirects. In less than a minute specific anomaly signature gets generated. Note how detailed it is. \n \n The signature contains 11 attributes that cover all aspects: method, path, headers, and a payload. Such a level of granularity and reaction time is not feasible neither for volumetric vectors nor a SOC operator armed with a regex engine. \n Once a signature is generated App Protect DoS reduces the scope of mitigation to only requests that match the signature. It eliminates a chance to affect good traffic at all. Matching traffic receives a redirect and then a challenge in case if an attacker is smart enough to follow redirects. \n After few minutes of observation App Protect DoS identifies bad actors since most of the requests come from the same IP addresses (right-bottom chart). Then switches mitigation to bad actor challenge. Despite this measure hits all the same traffic it allows App Protect DoS to protect itself. It takes much fewer CPU cycles to identify a target by IP address than match requests against the signature with 11 attributes. From now on App Protect DoS continues with the most efficient protection until attack traffic stops and server stress goes away. \n \n The technology overview and the demo above expose only a tiny bit of App Protect DoS protection logic. A whole lot of it engages for more complicated attacks. However, the results look impressive. None of the volumetric protection mechanisms or even a human SOC operator can provide such accurate mitigation within such a short reaction time. It is only possible when a machine fights a machine. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"10865","kudosSumWeight":1,"repliesCount":3,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtMjEyMzhpRERCN0RDMzkzOUNBN0IwMg?revision=2\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtMzcyN2lDMkQ4NEMzMDI0QjdCOTcy?revision=2\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtOTczMWk4QjJCNDgzQkYwOUM0M0I2?revision=2\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA5MDgtMTIzNzJpOThCQTQ1NUNFNTgzRjRERg?revision=2\"}"}}],"totalCount":4,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:290790":{"__typename":"Conversation","id":"conversation:290790","topic":{"__typename":"TkbTopicMessage","uid":290790},"lastPostingActivityTime":"2021-09-09T14:31:59.000-07:00","solved":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA3OTAtNDc4aUYyQzZDMkJDMDdCRkVFNzU?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA3OTAtNDc4aUYyQzZDMkJDMDdCRkVFNzU?revision=1","title":"0151T000003uv4pQAA.png","associationType":"BODY","width":775,"height":316,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA3OTAtMTMxMTZpNUY2MUYzRDgxOTk2RENBRg?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA3OTAtMTMxMTZpNUY2MUYzRDgxOTk2RENBRg?revision=1","title":"0151T000003uv4kQAA.png","associationType":"BODY","width":775,"height":312,"altText":null},"TkbTopicMessage:message:290790":{"__typename":"TkbTopicMessage","subject":"WAF Policy Editor - a Web-Based Tool to Configure a Declarative WAF Policy","conversation":{"__ref":"Conversation:conversation:290790"},"id":"message:290790","revisionNum":1,"uid":290790,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:194331"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":1591},"postTime":"2021-09-09T14:31:59.000-07:00","lastPublishTime":"2021-09-09T14:31:59.000-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Introduction \n\n The declarative policy is a great step forward to unify WAF configuration across F5 WAFs. However, the policy format is JSON; easy for machines to deal with but not so easy for humans. Currently, the process of putting together a declarative WAF policy requires a human to carefully read through tons of online documentation, figure out what features need to be enabled, and how each feature works. Once understood, the typical human will read one more time to examine configuration examples and then type out the JSON structure, without typos. Any typo may cause the WAF engine to reject a policy. Not a terribly user-friendly procedure. The following project exists to address this kind of complexity. \n\n The Project \n\n WAF Policy Editor is a web-based tool that implements a UI to put together a declarative WAF policy. The basic concept is simple. Everything you configure in the UI will translate into a JSON file automatically and vice versa. The following screenshot gives an overview of the UI. \n\n \nThe menu ribbon at the very top lists all of the supported features. Input fields in the middle configure a policy. The text area at the bottom represents the current policy state. The user is free to modify a policy via either input fields or directly in the JSON. Both representations are synchronized.\n Another important aspect is that the policy editor continuously verifies policy validity and notifies a user if the configuration doesn't comply with a policy schema. For instance, the screenshot below informs a user that an application language field can only contain specific values. \n\n \n\n The Workflows \n\n There are few workflows for which this tool is designed. \n\n The basic workflow is when a user configures a policy from scratch using the UI to set up desired features. Once done a policy can be simply either copied or downloaded as a file. \n\n The second approach allows modifying an existing policy. It is similar to previous, however, once a user pastes an existing policy JSON to the text area it gets automatically translated to UI so a user can make desired modifications. \n\n The last workflow allows to modify the existing policy as well but instead of pasting a policy a user can simply reference it as a query string parameter \"ref\". This is handy when a policy is located in a repository. Example link \n\n Even though it is just a small community team working on this project it actively evolving. Every week new features arrive. If you find the project useful please give it a try and leave your feedback right to GitHub issues. Your feedback is critical in order to steer as many efforts as possible to the most voted features. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"2751","kudosSumWeight":1,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA3OTAtNDc4aUYyQzZDMkJDMDdCRkVFNzU?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA3OTAtMTMxMTZpNUY2MUYzRDgxOTk2RENBRg?revision=1\"}"}}],"totalCount":2,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:290849":{"__typename":"Conversation","id":"conversation:290849","topic":{"__typename":"TkbTopicMessage","uid":290849},"lastPostingActivityTime":"2021-09-07T08:25:30.000-07:00","solved":false},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTIyOTZpRjUzMzdGN0Q3RDc1MUU0OA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTIyOTZpRjUzMzdGN0Q3RDc1MUU0OA?revision=1","title":"0151T000003uuN8QAI.png","associationType":"BODY","width":775,"height":543,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMjk1M2kyMUZBMTkwNEE4NTQxQkQ4?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMjk1M2kyMUZBMTkwNEE4NTQxQkQ4?revision=1","title":"0151T000003uuN5QAI.png","associationType":"BODY","width":775,"height":408,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktODI4NWk0RUJCMzFBMjc3RTBEMTUy?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktODI4NWk0RUJCMzFBMjc3RTBEMTUy?revision=1","title":"0151T000003uuN2QAI.png","associationType":"BODY","width":775,"height":394,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTIzMWlBQzAzNzlCNjA4RDM5NTVB?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTIzMWlBQzAzNzlCNjA4RDM5NTVB?revision=1","title":"0151T000003uuN3QAI.png","associationType":"BODY","width":775,"height":666,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTU5MjhpODJFN0UyNzM2RTM2QUQ4Nw?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTU5MjhpODJFN0UyNzM2RTM2QUQ4Nw?revision=1","title":"0151T000003uuN7QAI.png","associationType":"BODY","width":775,"height":142,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTUwODhpOUZBMTFGOTlCMDg3Mjg4RQ?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTUwODhpOUZBMTFGOTlCMDg3Mjg4RQ?revision=1","title":"0151T000003uuNDQAY.png","associationType":"BODY","width":775,"height":246,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTQ0MDFpMzYyOUNCQjhGRjc5QkEyNg?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTQ0MDFpMzYyOUNCQjhGRjc5QkEyNg?revision=1","title":"0151T000003uuNCQAY.png","associationType":"BODY","width":775,"height":401,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktNzU5Mmk3OEEzQzhENzYxNkI2Q0Iz?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktNzU5Mmk3OEEzQzhENzYxNkI2Q0Iz?revision=1","title":"0151T000003uuN4QAI.png","associationType":"BODY","width":775,"height":280,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktNDU5NWk4NzlFOURDMTRCMDI5MjM3?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktNDU5NWk4NzlFOURDMTRCMDI5MjM3?revision=1","title":"0151T000003uuN9QAI.png","associationType":"BODY","width":775,"height":459,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTE0MzBpMThFNjY4RDJEODNERTQwRA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTE0MzBpMThFNjY4RDJEODNERTQwRA?revision=1","title":"0151T000003uuNHQAY.png","associationType":"BODY","width":775,"height":343,"altText":null},"TkbTopicMessage:message:290849":{"__typename":"TkbTopicMessage","subject":"Cloud Template for App Protect WAF","conversation":{"__ref":"Conversation:conversation:290849"},"id":"message:290849","revisionNum":1,"uid":290849,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:194331"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":717},"postTime":"2021-09-07T08:25:30.000-07:00","lastPublishTime":"2021-09-07T08:25:30.000-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Introduction\n Everybody needs a WAF. However, when it gets to a deployment stage a team usually realizes that production-grade deployment going to be far more complex than a demo environment. In the case of a cloud deployment VPC networking, infrastructure security, VM images, auto-scaling, logging, visibility, automation, and many more topics require detailed analysis. Usually, it takes at least a few weeks for an average team to design and implement a production-grade WAF in a cloud. That is the one side of the problem. Additionally, cloud deployment best practices are the same for everyone, therefore most of well-made WAF deployments follow a similar path and become similar at the end. The statements above bring us to an obvious conclusion that proper WAF deployment can be templatized. So a team doesn’t spend time on deployment and maintenance but starts to use a WAF from day zero. The following paragraphs introduce a project that implements a Cloud Formation template to deploy production-grade WAF in AWS cloud just in a few clicks.\n Project (GitHub)\n On a high level, the project implements a Cloud Formation template that automatically deploys a production-grade WAF to AWS cloud. The template aims to follow cloud deployment best practices to set up a complete solution that is fully automated, requires minimum to no infrastructure management, therefore, allows a team to focus on application security. The following picture represents the overall solution structure.\n \n The solution includes a definition of three main components.\n Auto-scaling data plane based on official NGINX App Protect AWS AMI images. Git repository as the source of data plane and security configuration. Visibility dashboards displaying the WAF health and security data.\n Therefore it becomes a complete and easy-to-use solution to protect applications whether they run in AWS or in any other location.\n Data Plane: Data plane auto-scales based on the amount of incoming traffic and uses official NGINX App Protect AWS AMIs to spin up new VM instances. That removes the operational headache and optimizes costs since WAF dynamically adjusts the amount of computing resources and charges a user on an as-you-go basis.\n Configuration Plane: Solution configuration follows GitOps principles. The template creates the AWS CodeCommit git repository as a source of forwarding and security configuration. AWS CodeDeploy pipeline automatically delivers a configuration across all data plane VMs.\n Visibility: Alongside the data plane and configuration repository the template sets up a set of visibility dashboards in AWS CloudWatch. Data plane VMs send logs and metrics to CloudWatch service that visualizes incoming data as a set of charts and tables showing WAF health and security violations.\n \n\n Therefore these three components form a complete WAF solution that is easy to deploy, doesn't impose any operational headache and provides handy interfaces for WAF configuration and visibility right out of the box.\n Demo\n As mentioned above, one of the main advantages of this project is the ease of WAF deployment. It only requires downloading the AWS CloudFormation template file from the project repository and deploy it whether via AWS Console or AWS CLI.\n \n Template requests a number of parameters, however, all they are optional.\n \n As soon as stack creation is complete WAF is ready to use. Template outputs contain WAF URL and pointer to configuration repository.\n \n By default the WAF responds with static page.\n \n\n As a next step, I'll put this cloud WAF instance in front of a web application. Similar to any other NGINX instance, I'll configure it to forward traffic to the app and inspect all requests with App Protect WAF.\n As mentioned before, all config lives in a git repo that resides in the AWS CodeCommit service. I'm adjusting the NGINX configuration to forward traffic to the protected application.\n \n Once committed to the repo, a pipeline delivers the change to all data plane VMs.\n \n Therefore all traffic redirects to a protected application\n (screenshot below is not of a real company, and used for demo purposes only).\n \n Similar to NGINX configuration App Protect policy resides in the same repository. Similarly, all changes reflect running VMs.\n \n Once the configuration is complete, a user can observe system health and security-related data via pre-configured AWS CloudWatch dashboards.\n \n Outline\n As you can see, the use of a template to deploy a cloud WAF allows to significantly reduce time spent on WAF deployment and maintenance. Handy interfaces for configuration and visibility turn this project into a boxed solution allowing a user to easily operate a WAF and focus on application security.\n Please comment if you find useful to have this kind of solution in major public clouds marketplaces. It is a community project so far, and we need as much feedback as possible to steer one properly. Feel free to give it a try and leave feedback here or at the project's git repository.\n P.S.: Take a look to another community project that contributes to F5 WAF ecosystem: WAF Policy Editor ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"5229","kudosSumWeight":3,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTIyOTZpRjUzMzdGN0Q3RDc1MUU0OA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMjk1M2kyMUZBMTkwNEE4NTQxQkQ4?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktODI4NWk0RUJCMzFBMjc3RTBEMTUy?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTIzMWlBQzAzNzlCNjA4RDM5NTVB?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTU5MjhpODJFN0UyNzM2RTM2QUQ4Nw?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDY","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTUwODhpOUZBMTFGOTlCMDg3Mjg4RQ?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDc","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTQ0MDFpMzYyOUNCQjhGRjc5QkEyNg?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDg","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktNzU5Mmk3OEEzQzhENzYxNkI2Q0Iz?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDk","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktNDU5NWk4NzlFOURDMTRCMDI5MjM3?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDEw","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yOTA4NDktMTE0MzBpMThFNjY4RDJEODNERTQwRA?revision=1\"}"}}],"totalCount":10,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:288791":{"__typename":"Conversation","id":"conversation:288791","topic":{"__typename":"TkbTopicMessage","uid":288791},"lastPostingActivityTime":"2021-06-02T11:09:23.000-07:00","solved":false},"User:user:303102":{"__typename":"User","uid":303102,"login":"Ulises_Alonso","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/images/dS0zMDMxMDItNnNGUVZV?image-coordinates=90%2C0%2C517%2C427"},"id":"user:303102"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODg3OTEtMjAyM2lEQjkwQjg3NEM2OEZCODRC?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODg3OTEtMjAyM2lEQjkwQjg3NEM2OEZCODRC?revision=1","title":"0151T0000040YRCQA2.png","associationType":"BODY","width":1400,"height":788,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODg3OTEtNTAxMWlENTIwQzBFM0Q3Mjg4OEI5?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODg3OTEtNTAxMWlENTIwQzBFM0Q3Mjg4OEI5?revision=1","title":"0151T0000040YU6QAM.png","associationType":"BODY","width":1400,"height":827,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODg3OTEtNTcxaUM1NjZFNkM4MjA2NkJGMTE?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODg3OTEtNTcxaUM1NjZFNkM4MjA2NkJGMTE?revision=1","title":"0151T0000040YU2QAM.png","associationType":"BODY","width":1400,"height":827,"altText":null},"TkbTopicMessage:message:288791":{"__typename":"TkbTopicMessage","subject":"Deploying NGINXplus with AppProtect in Tanzu Kubernetes Grid","conversation":{"__ref":"Conversation:conversation:288791"},"id":"message:288791","revisionNum":1,"uid":288791,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:303102"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":1327},"postTime":"2021-06-02T11:09:23.000-07:00","lastPublishTime":"2021-06-02T11:09:23.000-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Introduction Tanzu Kubernetes Grid (aka TKG) is VMware's main Kubernetes offering. Although Tanzu Kubernetes Grid is a certified conformant Kubernetes offering the different Kubernetes offerings can be customized in different ways. In the case of TKG a remarkable feature is the use of Pod Security Policies by default. TKG clusters are very easily spin-up in either public or private clouds by means of creating a single declaration YAML file such as the following: apiVersion: run.tanzu.vmware.com/v1alpha1 \nkind: TanzuKubernetesCluster \nmetadata:\n name: tkg1 \n namespace: tkg1 \nspec:\n distribution:\n version: v1.18.15+vmware.1-tkg.1.600e412\n topology:\n controlPlane:\n count: 1 \n class: best-effort-medium \n storageClass: vsan-default-storage-policy\n workers:\n count: 1 \n class: best-effort-medium \n storageClass: vsan-default-storage-policy\n As you can see from the schema a TKG cluster is deployed just as another Kubernetes resource. How does it work? In the case of public clouds, these TanzuKubernetesCluster resources are instantiated from a bootstrap cluster named \"management Kubernetes cluster\" whilst when using vSphere with Tanzu, the TanzuKubernetesCluster resources are instantiated from vSphere with Tanzu's supervisor cluster. In this blog post it will be shown an example from start to end: Creating wildcard certificate from custom CA with easy-rsa. Enabling Harbor registry. Installing NGINXplus with AppProtect. Using NGINXplus Ingress Controller without AppProtect. Adding AppProtect to an Ingress resource. AppProtect is an NGINXplus module for WAF and Bot protection based on market leading F5 BIG-IP's ASM. AppProtect provides enhanced capabilities and performance for those who require more than what mod_auth provides. We will finish with two relevant considerations: Updating NGINXplus Ingress controller using Helm. This is used for example for scaling-out NGINXplus hence improving the overall performance. Using NGINXplus alongside with other Ingress Controllers (such as Contour). Prerequisites You need an NGINXplus license which can be retrieved from https://www.nginx.com/free-trial-request-nginx-ingress-controller/. This license is in practice a cert/key pair with the file names nginx-repo.{crt,key} referenced later on. The following software needs to be present in your in your machine: Docker v18.09+ GNU Make git Helm3 OpenSSL https://github.com/OpenVPN/easy-rsa.git Create a wildcard certificate with easy-rsa In the next steps it will be created a Certificate Authority (CA) and from it a wildcard certificate/key pair which will be loaded into Kubernetes as a TLS secret. This wildcard certificate will be used by all the services which will expose through Ingress. Retrieve easy-rsa and initialize a CA (output summarized): $ git clone https://github.com/OpenVPN/easy-rsa.git\n$ cd easyrsa3/\n$ ./easyrsa init-pki\n$ ./easyrsa build-ca\n Generate the wildcard key/cert pair (output summarized): $ ./easyrsa gen-req wildcard\nCommon Name (eg: your user, host, or server name) [wildcard]:*.tkg.bd.f5.com\n\nKeypair and certificate request completed. Your files are:\nreq: /Users/alonsocamaro/Documents/VMware-Tanzu/tanzu/easy-rsa/easyrsa3/pki/reqs/wildcard.req\nkey: /Users/alonsocamaro/Documents/VMware-Tanzu/tanzu/easy-rsa/easyrsa3/pki/private/wildcard.key\n$ ./easyrsa sign-req server wildcard\nRequest subject, to be signed as a server certificate for 825 days:\n\nsubject=\n commonName = *.tkg.bd.f5.com\n\nThe Subject's Distinguished Name is as follows\ncommonName :ASN.1 12:'*.tkg.bd.f5.com'\nCertificate is to be certified until Aug 21 15:58:24 2023 GMT (825 days)\n\nWrite out database with 1 new entries\nData Base Updated\n\nCertificate created at: /Users/alonsocamaro/Documents/VMware-Tanzu/tanzu/easy-rsa/easyrsa3/pki/issued/wildcard.crt\n The certificate is stored in ./pki/issued/wildcard.crt and the key is stored encrypted in pki/private/wildcard.key. Import these into a Kubernetes secret using the next steps: $ openssl rsa -in ./pki/private/wildcard.key -out ./pki/private/wildcard-unencrypted.key\nEnter pass phrase for ./pki/private/wildcard.key:\nwriting RSA key\n$ kubectl create ns ingress-nginx\n$ kubectl create -n ingress-nginx secret tls wildcard-tls --key ./pki/private/wildcard-unencrypted.key --cert ./pki/issued/wildcard.crt\nsecret/wildcard-tls created\n$ rm ./pki/private/wildcard-unencrypted.key\n As you might have noticed the secret is loaded in the namespace ingress-nginx where NGINXplus Ingress Controller will be installed. Enable your image registry in Tanzu You need an image registry. When using vSphere with Tanzu this comes with Harbor. In this case you have to follow the next steps: Enable Harbor by following https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-AE24CF79-3C74-4CCD-B7C7-757AD082D86A.html. Trust Harbor's self-signed certificate: In the case of using vSphere with Tanzu Update 2 (U2) follow https://tanzu.vmware.com/content/blog/how-to-set-up-harbor-registry-self-signed-certificates-tanzu-kubernetes-clusters. If using a previous version follow https://cormachogan.com/2020/06/23/integrating-embedded-vsphere-with-kubernetes-harbor-registry-with-tkg-guest-clusters/ with the caveat that you will need to re-apply the changes if you upgrade the TKG cluster. Alternatively, you could also use your own certificates and follow https://docs.vmware.com/en/VMware-Tanzu-Kubernetes-Grid/1.3/vmware-tanzu-kubernetes-grid-13/GUID-cluster-lifecycle-secrets.html#trust-custom-ca-certificates-on-cluster-nodes-3. Installing NGINXplus Ingress Controller This blog shows step by step everything that needs to be done to create an AppProtect-secured Ingress Controller with a wildcard certificate that will created as well. This blog only assumes that the TKG cluster is up and running. If you want to perform further customizations you can check https://docs.nginx.com/nginx-ingress-controller and https://docs.nginx.com/nginx-app-protect/configuration. In this blog it has been used TKG 1.3 in vSphere with Tanzu with NSX-T. The steps are similar when using any other supported TKG environment. Build the NGINXplus DOCKER image Define the registry endpoint and namespace where the TKG cluster will be deployed: $ REGISTRY=<registry IP or FQDN>\n$ NS=<your namespace>\n Log in the registry $ docker login $REGISTRY\nUsername: <your user>\nPassword: <your password>\nLogin Succeeded\n Retrieve NGINXplus $ git clone https://github.com/nginxinc/kubernetes-ingress/\n$ cd kubernetes-ingress\n$ git checkout v1.11.1\n Copy the license files into base folder of NGINXplus $ cp $LICDIR/nginx-repo.{crt,key} .\n Build the image $ make debian-image-nap-plus PREFIX=$REGISTRY/$NS/nginx-plus-ingress TARGET=container\nDocker version 19.03.8, build afacb8b\ndocker build --build-arg IC_VERSION=1.11.1-32745366 --build-arg GIT_COMMIT=32745366 --build-arg VERSION=1.11.1 --target container -f build/Dockerfile -t 10.105.210.67/tkg1/nginx-plus-ingress:1.11.1 . --build-arg BUILD_OS=debian-plus-ap --build-arg PLUS=-plus --secret id=nginx-repo.crt,src=nginx-repo.crt --secret id=nginx-repo.key,src=nginx-repo.key\n[+] Building 4.9s (24/24) FINISHED \n After this we can verify the image is ready in our local docker: $ docker images\nREPOSITORY TAG IMAGE ID CREATED SIZE\n$REGISTRY/$NS/nginx-plus-ingress 1.11.1 70113ec38914 35 minutes ago 626MB\n If we wanted to push it into another namespace we would perform an image tag operation as follows: docker image tag 70113ec38914 $REGISTRY/$ANOTHERNS/nginx-plus-ingress:1.11.1\n Upload the image into the repository: make push PREFIX=$REGISTRY/$NS/nginx-plus-ingress\n Configure NGINXplus installation Switch to the helm chart folder cd deployments/helm-chart\n Make a backup of the default nginx-plus config file cp values-plus.yaml values-plus.yaml.orig\n We will edit the file values-plus.yaml as follows in order to: Enable AppProtect Allow to specify a wildcard TLS certificate that we will use for all the services. Expose NGINXplus using a LoadBalancer with a Cluster externalTrafficPolicy. Exposing NGINXplus (or any other Ingress Controller such as Contour) using Cluster externalTrafficPolicy is required given that the NSX-T native load balancer doesn't perform any health checking when creating a Service of Type LoadBalancer. We will see how to improve this in future blogs with the use of BIG-IP. controller:\n nginxplus: true\n image:\n repository: nginx-plus-ingress\n tag: \"1.11.1\"\n service:\n externalTrafficPolicy: Cluster\n appprotect:\n ## Enable the App Protect module in the Ingress Controller.\n enable: true\n wildcardTLS:\n ## The base64-encoded TLS certificate for every Ingress host that has TLS enabled but no secret specified.\n ## If the parameter is not set, for such Ingress hosts NGINX will break any attempt to establish a TLS connection.\n cert: \"\"\n\n ## The base64-encoded TLS key for every Ingress host that has TLS enabled but no secret specified.\n ## If the parameter is not set, for such Ingress hosts NGINX will break any attempt to establish a TLS connection.\n key: \"\"\n\n ## The secret with a TLS certificate and key for every Ingress host that has TLS enabled but no secret specified.\n ## The value must follow the following format: `<namespace>/<name>`.\n ## Used as an alternative to specifying a certificate and key using `controller.wildcardTLS.cert` and `controller.wildcardTLS.key` parameters.\n ## Format: <namespace>/<secret_name>\n secret: ingress-nginx/wildcard-tls\n This file can also be found in https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/values-plus.yaml Apply the required PodSecurityPolicy before NGINXplus installation The next step creates a PodSecurityPolicy which is required by Tanzu Kubernetes Grid and it is bound to the Service Account ingress-nginx used in the regular NGINXplus install. $ kubectl apply -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/nginx-psp.yaml\npodsecuritypolicy.policy/ingress-nginx created\nclusterrole.rbac.authorization.k8s.io/ingress-nginx-psp created\nclusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-psp created\n Install NGINXplus Ingress controller using Helm From the deployments/helm-chart directory of the downloaded NGINXplus, it is just needed to run the next command: $ helm -n ingress-nginx install ingress-nginx -f values-plus.yaml .\nNAME: ingress-nginx\nLAST DEPLOYED: Mon May 17 15:16:20 2021\nNAMESPACE: ingress-nginx\nSTATUS: deployed\nREVISION: 1\nTEST SUITE: None\nNOTES:\nThe NGINX Ingress Controller has been installed.\n Checking the resulting installation When checking the resulting resources we can see that by default a single POD is created. We can scale up/down this as required by using Helm. This will be shown later on. Note also that by default the NGINXplus Ingress Controller is automatically exposed using the Service Type LoadBalancer resource which configures an external load balancer. In this case the external load balancer is NSX-T's native LB as shown in the screenshot. below When using vSphere networking this would have been HAproxy by default. In next blogs we will show how to use F5 BIG-IP in TKG clusters instead. $ kubectl -n ingress-nginx get all\nNAME READY STATUS RESTARTS AGE\npod/ingress-nginx-nginx-ingress-7d4587b44c-n9b8l 1/1 Running 0 16h\nNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE\nservice/ingress-nginx-nginx-ingress LoadBalancer 100.69.127.66 10.105.210.68 80:30527/TCP,443:31185/TCP 16h\nNAME READY UP-TO-DATE AVAILABLE AGE\ndeployment.apps/ingress-nginx-nginx-ingress 1/1 1 1 16h\nNAME DESIRED CURRENT READY AGE\nreplicaset.apps/ingress-nginx-nginx-ingress-7d4587b44c 1 1 1 16h\n In the next screenshots we can see the resulting configuration in NSX-T: Both pools for port 80 and port 443 point to the worker's node addresses. This means that the traffic flow will be NSX-T LB -> ClusterIP -> Ingress Controller's POD address (in the same or in another node). This is the case of any regular Ingress Controller including TKG's provided Contour. In next blogs it will be shown how these many layers of indirection can be bypassed using F5 BIG-IP. Using NGINXplus Ingress Controller without AppProtect Creating a regular Ingress resource In this initial example we will create two services (coffee and tea) which will be exposed with an Ingress resource called cafe-ingress. This will expose the services in the URL https://cafe.tkg.bd.f5.com/coffee and https://cafe.tkg.bd.f5.com/tea using the previously created wildcard certificate for *.tkg.bd.f5.com as depicted in the next diagram. To create this setup run the following commands: $ kubectl create ns test\n$ kubectl -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/cafe-rbac.yaml\n$ kubectl -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/cafe.yaml\n$ kubectl -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/cafe-ingress.yaml\n This is the same example provided in the official NGINXplus documentation but we add the cafe-rbac.yaml declaration which creates the necessary PodSecurity policies and bindings for TKG. To verify the result first we will check the Ingress resource itself: $ kubectl -n test get ingress\nNAME CLASS HOSTS ADDRESS PORTS AGE\ncafe-ingress nginx cafe.tkg.bd.f5.com 10.105.210.68 80, 443 3m44s\n where we can observe that the IP address is the one of the external loadbalancer seen before. To verify it is all working as expected we will use curl as follows: $ curl --cacert ca-tkg.bd.f5.com.crt --resolve cafe.tkg.bd.f5.com:443:10.105.210.68 https://cafe.tkg.bd.f5.com/coffee \nServer address: 100.96.1.16:8080\nServer name: coffee-86954d94fd-pnvpq\nDate: 18/May/2021:16:18:59 +0000\nURI: /coffee\nRequest ID: 63964930a2d1038af5f204ef8fbe91fc\n which has the following key parameters: Use --cacert to specify our CA crt file previously created Use --resolve to allow curl resolve the FQDN of the request Adding AppProtect to an Ingress resource Additional configuration Our deployed NGINXplus has AppProtect built-in. It is up to the user of the Ingress resource if it wants to enable it, on a per Ingress basis. In our example we will apply the AppProtect security policies in the user namespace \"test\". We will also create a syslog store in the ingress-nginx namespace. All these can be customized. Ultimately, the user just needs to add the following annotations in order to secure the cafe site: annotations:\n appprotect.f5.com/app-protect-policy: \"test/dataguard-alarm\"\n appprotect.f5.com/app-protect-enable: \"True\"\n appprotect.f5.com/app-protect-security-log-enable: \"True\"\n appprotect.f5.com/app-protect-security-log: \"test/logconf\"\n appprotect.f5.com/app-protect-security-log-destination: \"syslog:server=100.70.175.24:514\"\n The custom AppProtect policy used in this example contains DataGuard protection for Credit Card Number, US Social Security number leaks and a custom signature. It is also defined where to send the AppProtect logs. These are sent in SYSLOG/TCP mode independently of the regular logs generated by NGINXplus. To make all these happen first we will create the syslog server: kubectl apply -n ingress-nginx -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/syslog-rbac.yaml\nkubectl apply -n ingress-nginx -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/syslog.yaml\n Next, we will create the AppProtect policies: kubectl apply -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/ap-apple-uds.yaml\nkubectl apply -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/ap-dataguard-alarm-policy.yaml\nkubectl apply -n test -f https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/ap-logconf.yaml\n Finally we will add the above annotations to the Ingress resource. For that, we will need to get SYSLOG's POD address and replace it in the cafe-ingress-ap.yaml definition. curl -O https://raw.githubusercontent.com/f5devcentral/f5-bd-tanzu-tkg-nginxplus/main/cafe-ingress-ap.yaml\nSYSLOG_IP=<IP address of syslog's POD>\nsed -e \"s/SYSLOG/$SYSLOG_IP/\" cafe-ingress-ap.yaml > cafe-ingress-ap-syslog.yaml\nkubectl apply -n test -f cafe-ingress-ap-syslog.yaml\n Note: it might take few seconds to make the AppProtect configuration effective. Verifying AppProtect Run the following command to watch the requests live as handled by AppProtect: kubectl -n ingress-nginx exec -it <SYSLOG POD NAME> -- tail -f /var/log/messages\n Send a request that triggers the custom signature: curl --cacert ca-tkg.bd.f5.com.crt --resolve cafe.tkg.bd.f5.com:443:10.105.210.69 \"https://cafe.tkg.bd.f5.com/coffee/\" -X POST -d \"apple\"\n You should see a log similar to the following one in the syslog logs: May 24 13:43:23 ingress-nginx-nginx-ingress-7d4587b44c-wvrxs ASM:attack_type=\"Non-browser Client,Brute Force Attack\",blocking_exception_reason=\"N/A\",date_time=\"2021-05-24 13:43:23\",dest_port=\"443\",ip_client=\"10.105.210.69\",is_truncated=\"false\",method=\"POST\",policy_name=\"dataguard-alarm\",protocol=\"HTTPS\",request_status=\"blocked\",response_code=\"0\",severity=\"Critical\",sig_cves=\"N/A\",sig_ids=\"300000000\",sig_names=\"Apple_medium_acc [Fruits]\",sig_set_names=\"{apple_sigs}\",src_port=\"4096\",sub_violations=\"N/A\",support_id=\"15704273797572010868\",threat_campaign_names=\"N/A\",unit_hostname=\"ingress-nginx-nginx-ingress-7d4587b44c-wvrxs\",uri=\"/coffee/\",violation_rating=\"3\",vs_name=\"24-cafe.tkg.bd.f5.com:9-/coffee\",x_forwarded_for_header_value=\"N/A\",outcome=\"REJECTED\",outcome_reason=\"SECURITY_WAF_VIOLATION\",violations=\"Attack signature detected,Bot Client Detected\",violation_details=\"<?xml version='1.0' encoding='UTF-8'?><BAD_MSG><violation_masks><block>10000000200c00-3030430000070</block><alarm>2477f0ffcbbd0fea-8003f35cb000007c</alarm><learn>200000-20</learn><staging>0-0</staging></violation_masks><request-violations><violation><viol_index>42</viol_index><viol_name>VIOL_ATTACK_SIGNATURE</viol_name><context>request</context><sig_data><sig_id>300000000</sig_id><blocking_mask>7</blocking_mask><kw_data><buffer>YXBwbGU=</buffer><offset>0</offset><length>5</length></kw_data></sig_data></violation></request-violations></BAD_MSG>\",bot_signature_name=\"curl\",bot_category=\"HTTP Library\",bot_anomalies=\"N/A\",enforced_bot_anomalies=\"N/A\",client_class=\"Untrusted Bot\",request=\"POST /coffee/ HTTP/1.1\\r\\nHost: cafe.tkg.bd.f5.com\\r\\nUser-Agent: curl/7.64.1\\r\\nAccept: */*\\r\\nContent-Length: 5\\r\\nContent-Type: application/x-www-form-urlencoded\\r\\n\\r\\napple\"\n Updating NGINXplus Ingress controller using Helm By default a single NGINXplus instance is created, if you want to increase the performance of it, scaling-out is as simple as editing the values-plus.yaml file and setting a replicaCount parameter with the desired value: controller:\n replicaCount: 4\n And running helm upgrade as follows $ helm -n ingress-nginx upgrade ingress-nginx -f values-plus.yaml .\nRelease \"ingress-nginx\" has been upgraded. Happy Helming!\nNAME: ingress-nginx\nLAST DEPLOYED: Thu May 20 14:20:38 2021\nNAMESPACE: ingress-nginx\nSTATUS: deployed\nREVISION: 2\nTEST SUITE: None\nNOTES:\nThe NGINX Ingress Controller has been installed.\n Using NGINXplus alongside with other Ingress Controllers (such as Contour). NGINXplus does support Ingress/v1 resource version available in Kubernetes 1.18+ as well as previous Ingress/v1beta1 API resource version for backwards compatibility. Contour Ingress Controller is provided in TKG by VMware as an add-on which is not installed by default. If installed, you have to be aware that Contour, at time of this writting (May 2021), only supports the older Ingress/v1beta1 API resource version. This means that when defining Ingress resources you have to specify the Ingress Controller to use by means of adding the following annotation: kubernetes.io/ingress.class: <ingress conroller name>\n where <ingress controller name> could be nginx or contour. For further details on this topic you can check https://docs.nginx.com/nginx-ingress-controller/installation/running-multiple-ingress-controllers/ Conclusion In this blog post we have gone through all the steps required to install and use NGINXplus with AppProtect in Tanzu Kubernetes Grid with a real world example. Overall, the installation is the same as in any Kubernetes but the following two items need to be taken into account: Before deploying, make sure that the appropriate PodSecurityPolicies are in place for either NGINXplus or the workloads. PodSecurityPolicies are not enabled by default in many Kubernetes distributions so this represents a change from the usual practice. If deploying NGINXplus alongside another Ingress Controller make sure that the Ingress resources are defined appropriately in order to select the right Ingress Controller for the corresponding Ingress resource. In this blog we used NGINXplus in a TKG cluster deployed in an on-premises infrastructure (vSphere with Tanzu) with the Antrea CNI and NSX-T networking. The steps would have been the same if it had been used vSphere networking or Calico CNI. The only difference could come when exposing it through the external load balancer. If the external load balancer performed health checking it would be preferable to use local externalTrafficPolicy since this avoids a hop and allows keeping the client source address. In future blogs we will post how to expose NGINXplus more effectively and in a cloud-agnostic manner by using BIG-IP as external load balancer. ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"27332","kudosSumWeight":1,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODg3OTEtMjAyM2lEQjkwQjg3NEM2OEZCODRC?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODg3OTEtNTAxMWlENTIwQzBFM0Q3Mjg4OEI5?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODg3OTEtNTcxaUM1NjZFNkM4MjA2NkJGMTE?revision=1\"}"}}],"totalCount":3,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"Conversation:conversation:282849":{"__typename":"Conversation","id":"conversation:282849","topic":{"__typename":"TkbTopicMessage","uid":282849},"lastPostingActivityTime":"2021-04-14T08:20:39.000-07:00","solved":false},"User:user:189460":{"__typename":"User","uid":189460,"login":"Gal_Goldshtein","registrationData":{"__typename":"RegistrationData","status":null},"deleted":false,"avatar":{"__typename":"UserAvatar","url":"https://community.f5.com/t5/s/zihoc95639/m_assets/avatars/default/avatar-8.svg?time=0"},"id":"user:189460"},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktMzc1NGlFNDU5RkFBNzFBNzYyMUI1?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktMzc1NGlFNDU5RkFBNzFBNzYyMUI1?revision=1","title":"0EM1T000002c71n.png","associationType":"BODY","width":850,"height":164,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktNDc2OGkzNzYyMjQ4RkY0RDBFNUY5?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktNDc2OGkzNzYyMjQ4RkY0RDBFNUY5?revision=1","title":"0EM1T000002c71o.png","associationType":"BODY","width":1311,"height":141,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktOTY4MmkxNjk1N0Y2MzFBNEJCRTU4?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktOTY4MmkxNjk1N0Y2MzFBNEJCRTU4?revision=1","title":"0EM1T000002c71p.png","associationType":"BODY","width":1308,"height":121,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktMTA1ODZpNzlBM0Y0OTAxMjM0RkMwRA?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktMTA1ODZpNzlBM0Y0OTAxMjM0RkMwRA?revision=1","title":"0EM1T000002c71q.png","associationType":"BODY","width":1227,"height":629,"altText":null},"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktMTU2NjVpMjBGM0U5REZCNUNCNDU2Rg?revision=1\"}":{"__typename":"AssociatedImage","url":"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktMTU2NjVpMjBGM0U5REZCNUNCNDU2Rg?revision=1","title":"0EM1T000002c71r.png","associationType":"BODY","width":907,"height":520,"altText":null},"TkbTopicMessage:message:282849":{"__typename":"TkbTopicMessage","subject":"Mitigating New Gadget Leveraging JNDI Injection into Remote Code Execution Using Advanced WAF","conversation":{"__ref":"Conversation:conversation:282849"},"id":"message:282849","revisionNum":1,"uid":282849,"depth":0,"board":{"__ref":"Tkb:board:TechnicalArticles"},"author":{"__ref":"User:user:189460"},"teaser@stripHtml({\"removeProcessingText\":true,\"truncateLength\":-1})":"","introduction":"","metrics":{"__typename":"MessageMetrics","views":1455},"postTime":"2021-04-14T08:20:39.000-07:00","lastPublishTime":"2021-04-14T08:20:39.000-07:00","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})":" Recently a new gadget that bypass existing Java restrictions for JNDI injection was published via Tweet made by a security researcher identified by the twitter handle @PewGrand. JNDI – Java Naming and Directory Interface is a Java API that allows the application developer to retrieve objects based on a given name. JNDI supports different implementations like Remote Method Invocation (RMI), Lightweight Directory Access Protocol (LDAP) and others. The JNDI Injection vulnerability class was first presented in Black Hat 2016 by Alvaro Munoz and Oleksandr Mirosh who demonstrated the different ways where applications utilizing JNDI can be exploited into Remote Code Execution. Applications are vulnerable to JNDI Injections when user-controlled input is being passed as argument to methods like “lookup” of the “InitialContext” Java class, which is part of the Java Naming API. This method is responsible for resolving a name it received into an object identified by the supplied name. Figure 1: Example of vulnerable endpoint in a Spring application The name that the “lookup” method receives can also be in the form of LDAP or RMI URLs and therefore vulnerable applications may be forced to fetch a malicious object from attacker controlled RMI or LDAP server. In the last few years Oracle has applied several restrictions that aimed to prevent attackers from exploiting JNDI Injection vulnerabilities. One example of such restriction is the “trustURLCodebase” property which was introduced in Java Development Kit 8 – Update 121. This property prevents vulnerable applications from loading malicious objects from remote RMI repositories. Later, similar restriction was added also to cover LDAP repositories. Since those restrictions were added, exploiting JNDI Injection vulnerabilities now depends on existing gadgets, which means the classes used in the exploit must reside in the vulnerable application class path for the exploit to work. The new gadget discovered by @PewGrand uses similar approach as another previously disclosed gadget. They both start a rogue LDAP server that hosts an LDAP entry which specifies the “javaClassName” and “javaSerializedData” attributes. Those attributes can be used by Java LDAP implementations to reference Java objects in LDAP entries. Once the vulnerable application will fetch this entry from the attacker-controlled LDAP server the object referenced by the “javaSerializedData” attribute will be deserialized. Another point of similarity to the previously disclosed gadget is that both gadgets specify the BeanFactory class of Apache Tomcat as the object factory of the serialized object stored in the LDAP entry. The factory class is responsible for creating an instance of the class specified in the LDAP entry. The reason that the BeanFactory is a good pick for both gadgets is because it allows the rogue LDAP server to also specify a method that receives a string as a parameter which should be invoked after creating the instance from the specified class. This is achieved by specifying the “forceString” JNDI Reference. This is also where the difference between the two gadgets relies - the previous gadget was pointing to the “ELProcessor” class and specified the “eval” method as the “forceString” JNDI Reference, which allowed the gadget to execute arbitrary Java EL Expression template and thus achieve arbitrary code execution. Figure 2: Previous gadget creating instance of the ELProcessor class and invoking its “eval” method. The newly discovered gadget specifies the SnakeYAML Yaml class as the one that should be instantiated by Tomcat BeanFactory, and specifies the “load” method of the Yaml class in order to make the application parse a Yaml that loads JAR file from a remote server by using the Java URLClassLoader. Figure 3: New gadget creating instance of the SnakeYaml Yaml class and invoking the load method. Figure 4: Exploiting the vulnerable application using the new gadget. Mitigating JNDI Injections with Advanced WAF Advanced WAF customers under any supported version are now protected against this vulnerability. The exploitation attempt will be detected by newly released Java code injection attack signatures which can be found in signature sets that include the “Server Side Code Injection” attack type. Figure 5: Exploit attempt blocked by signature id 200104722. Additional resources https://twitter.com/steventseeley/status/1380065046458433536 https://gist.github.com/TheGrandPew/748ac740698511975eaeed5d77ecb2d9 https://www.blackhat.com/docs/us-16/materials/us-16-Munoz-A-Journey-From-JNDI-LDAP-Manipulation-To-RCE-wp.pdf https://www.veracode.com/blog/research/exploiting-jndi-injections-java ","body@stripHtml({\"removeProcessingText\":true,\"removeSpoilerMarkup\":true,\"removeTocMarkup\":true,\"truncateLength\":-1})@stringLength":"4826","kudosSumWeight":0,"repliesCount":0,"readOnly":false,"images":{"__typename":"AssociatedImageConnection","edges":[{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDE","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktMzc1NGlFNDU5RkFBNzFBNzYyMUI1?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDI","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktNDc2OGkzNzYyMjQ4RkY0RDBFNUY5?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDM","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktOTY4MmkxNjk1N0Y2MzFBNEJCRTU4?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDQ","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktMTA1ODZpNzlBM0Y0OTAxMjM0RkMwRA?revision=1\"}"}},{"__typename":"AssociatedImageEdge","cursor":"MjUuMnwyLjF8b3wyNXxfTlZffDU","node":{"__ref":"AssociatedImage:{\"url\":\"https://community.f5.com/t5/s/zihoc95639/images/bS0yODI4NDktMTU2NjVpMjBGM0U5REZCNUNCNDU2Rg?revision=1\"}"}}],"totalCount":5,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}},"videos":{"__typename":"VideoConnection","edges":[],"totalCount":0,"pageInfo":{"__typename":"PageInfo","hasNextPage":false,"endCursor":null,"hasPreviousPage":false,"startCursor":null}}},"CachedAsset:text:en_US-components/community/Navbar-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/community/Navbar-1743097583694","value":{"community":"Community Home","inbox":"Inbox","manageContent":"Manage Content","tos":"Terms of Service","forgotPassword":"Forgot Password","themeEditor":"Theme Editor","edit":"Edit Navigation Bar","skipContent":"Skip to content","migrated-link-9":"Groups","migrated-link-7":"Technical Articles","migrated-link-8":"DevCentral News","migrated-link-1":"Technical Forum","migrated-link-10":"Community Groups","migrated-link-2":"Water Cooler","migrated-link-11":"F5 Groups","Common-external-link":"How Do I...?","migrated-link-0":"Forums","article-series":"Article Series","migrated-link-5":"Community Articles","migrated-link-6":"Articles","security-insights":"Security Insights","migrated-link-3":"CrowdSRC","migrated-link-4":"CodeShare","migrated-link-12":"Events","migrated-link-13":"Suggestions"},"localOverride":false},"CachedAsset:text:en_US-components/community/NavbarHamburgerDropdown-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/community/NavbarHamburgerDropdown-1743097583694","value":{"hamburgerLabel":"Side Menu"},"localOverride":false},"CachedAsset:text:en_US-components/community/BrandLogo-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/community/BrandLogo-1743097583694","value":{"logoAlt":"Khoros","themeLogoAlt":"Brand Logo"},"localOverride":false},"CachedAsset:text:en_US-components/community/NavbarTextLinks-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/community/NavbarTextLinks-1743097583694","value":{"more":"More"},"localOverride":false},"CachedAsset:text:en_US-components/authentication/AuthenticationLink-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/authentication/AuthenticationLink-1743097583694","value":{"title.login":"Sign In","title.registration":"Register","title.forgotPassword":"Forgot Password","title.multiAuthLogin":"Sign In"},"localOverride":false},"CachedAsset:text:en_US-components/nodes/NodeLink-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/nodes/NodeLink-1743097583694","value":{"place":"Place {name}"},"localOverride":false},"CachedAsset:text:en_US-components/tags/TagSubscriptionAction-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/tags/TagSubscriptionAction-1743097583694","value":{"success.follow.title":"Following Tag","success.unfollow.title":"Unfollowed Tag","success.follow.message.followAcrossCommunity":"You will be notified when this tag is used anywhere across the community","success.unfollowtag.message":"You will no longer be notified when this tag is used anywhere in this place","success.unfollowtagAcrossCommunity.message":"You will no longer be notified when this tag is used anywhere across the community","unexpected.error.title":"Error - Action Failed","unexpected.error.message":"An unidentified problem occurred during the action you took. Please try again later.","buttonTitle":"{isSubscribed, select, true {Unfollow} false {Follow} other{}}","unfollow":"Unfollow"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageListTabs-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageListTabs-1743097583694","value":{"mostKudoed":"{value, select, IDEA {Most Votes} other {Most Likes}}","mostReplies":"Most Replies","mostViewed":"Most Viewed","newest":"{value, select, IDEA {Newest Ideas} OCCASION {Newest Events} other {Newest Topics}}","newestOccasions":"Newest Events","mostRecent":"Most Recent","noReplies":"No Replies Yet","noSolutions":"No Solutions Yet","solutions":"Solutions","mostRecentUserContent":"Most Recent","trending":"Trending","draft":"Drafts","spam":"Spam","abuse":"Abuse","moderation":"Moderation","tags":"Tags","PAST":"Past","UPCOMING":"Upcoming","sortBymostRecent":"Sort By Most Recent","sortBymostRecentUserContent":"Sort By Most Recent","sortBymostKudoed":"Sort By Most Likes","sortBymostReplies":"Sort By Most Replies","sortBymostViewed":"Sort By Most Viewed","sortBynewest":"Sort By Newest Topics","sortBynewestOccasions":"Sort By Newest Events","otherTabs":" Messages list in the {tab} for {conversationStyle}","guides":"Guides","archives":"Archives"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/QueryHandler-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/QueryHandler-1743097583694","value":{"title":"Query Handler"},"localOverride":false},"CachedAsset:text:en_US-components/community/NavbarDropdownToggle-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/community/NavbarDropdownToggle-1743097583694","value":{"ariaLabelClosed":"Press the down arrow to open the menu"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/OverflowNav-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/OverflowNav-1743097583694","value":{"toggleText":"More"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageView/MessageViewInline-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageView/MessageViewInline-1743097583694","value":{"bylineAuthor":"{bylineAuthor}","bylineBoard":"{bylineBoard}","anonymous":"Anonymous","place":"Place {bylineBoard}","gotoParent":"Go to parent {name}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/common/Pager/PagerLoadMore-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/common/Pager/PagerLoadMore-1743097583694","value":{"loadMore":"Show More"},"localOverride":false},"CachedAsset:text:en_US-components/customComponent/CustomComponent-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/customComponent/CustomComponent-1743097583694","value":{"errorMessage":"Error rendering component id: {customComponentId}","bannerTitle":"Video provider requires cookies to play the video. Accept to continue or {url} it directly on the provider's site.","buttonTitle":"Accept","urlText":"watch"},"localOverride":false},"CachedAsset:text:en_US-components/users/UserLink-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/users/UserLink-1743097583694","value":{"authorName":"View Profile: {author}","anonymous":"Anonymous"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageSubject-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageSubject-1743097583694","value":{"noSubject":"(no subject)"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageTime-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageTime-1743097583694","value":{"postTime":"Published: {time}","lastPublishTime":"Last Update: {time}","conversation.lastPostingActivityTime":"Last posting activity time: {time}","conversation.lastPostTime":"Last post time: {time}","moderationData.rejectTime":"Rejected time: {time}"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/nodes/NodeIcon-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/nodes/NodeIcon-1743097583694","value":{"contentType":"Content Type {style, select, FORUM {Forum} BLOG {Blog} TKB {Knowledge Base} IDEA {Ideas} OCCASION {Events} other {}} icon"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageUnreadCount-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageUnreadCount-1743097583694","value":{"unread":"{count} unread","comments":"{count, plural, one { unread comment} other{ unread comments}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageViewCount-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageViewCount-1743097583694","value":{"textTitle":"{count, plural,one {View} other{Views}}","views":"{count, plural, one{View} other{Views}}"},"localOverride":false},"CachedAsset:text:en_US-components/kudos/KudosCount-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/kudos/KudosCount-1743097583694","value":{"textTitle":"{count, plural,one {{messageType, select, IDEA{Vote} other{Like}}} other{{messageType, select, IDEA{Votes} other{Likes}}}}","likes":"{count, plural, one{like} other{likes}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageRepliesCount-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageRepliesCount-1743097583694","value":{"textTitle":"{count, plural,one {{conversationStyle, select, IDEA{Comment} OCCASION{Comment} other{Reply}}} other{{conversationStyle, select, IDEA{Comments} OCCASION{Comments} other{Replies}}}}","comments":"{count, plural, one{Comment} other{Comments}}"},"localOverride":false},"CachedAsset:text:en_US-components/messages/MessageBody-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-components/messages/MessageBody-1743097583694","value":{"showMessageBody":"Show More","mentionsErrorTitle":"{mentionsType, select, board {Board} user {User} message {Message} other {}} No Longer Available","mentionsErrorMessage":"The {mentionsType} you are trying to view has been removed from the community.","videoProcessing":"Video is being processed. Please try again in a few minutes.","bannerTitle":"Video provider requires cookies to play the video. Accept to continue or {url} it directly on the provider's site.","buttonTitle":"Accept","urlText":"watch"},"localOverride":false},"CachedAsset:text:en_US-shared/client/components/users/UserAvatar-1743097583694":{"__typename":"CachedAsset","id":"text:en_US-shared/client/components/users/UserAvatar-1743097583694","value":{"altText":"{login}'s avatar","altTextGeneric":"User's avatar"},"localOverride":false}}}},"page":"/tags/TagPage/TagPage","query":{"nodeId":"board:TechnicalArticles","tagName":"NGINX WAF"},"buildId":"q_bLpq2mflH0BeZigxpj6","runtimeConfig":{"buildInformationVisible":false,"logLevelApp":"info","logLevelMetrics":"info","openTelemetryClientEnabled":false,"openTelemetryConfigName":"f5","openTelemetryServiceVersion":"25.2.0","openTelemetryUniverse":"prod","openTelemetryCollector":"http://localhost:4318","openTelemetryRouteChangeAllowedTime":"5000","apolloDevToolsEnabled":false,"inboxMuteWipFeatureEnabled":false},"isFallback":false,"isExperimentalCompile":false,"dynamicIds":["./components/customComponent/CustomComponent/CustomComponent.tsx","./components/community/Navbar/NavbarWidget.tsx","./components/community/Breadcrumb/BreadcrumbWidget.tsx","./components/tags/TagsHeaderWidget/TagsHeaderWidget.tsx","./components/messages/MessageListForNodeByRecentActivityWidget/MessageListForNodeByRecentActivityWidget.tsx","./components/tags/TagSubscriptionAction/TagSubscriptionAction.tsx","./components/customComponent/CustomComponentContent/TemplateContent.tsx","../shared/client/components/common/List/ListGroup/ListGroup.tsx","./components/messages/MessageView/MessageView.tsx","./components/messages/MessageView/MessageViewInline/MessageViewInline.tsx","../shared/client/components/common/Pager/PagerLoadMore/PagerLoadMore.tsx","./components/customComponent/CustomComponentContent/HtmlContent.tsx","./components/customComponent/CustomComponentContent/CustomComponentScripts.tsx"],"appGip":true,"scriptLoader":[]}