Protecting APIs with NGINX App Protect

Recently NGINX App Protect learned how to ingest Open API file. With that feature protecting APIs with NGINX app protect become much easier. App Protect automatically configures itself to pass only allowed paths, methods, and parameters based on Open API file contents. Let's take a quick look how it works and how it helps.

Configuring WAF to protect APIs is a big mess. For example Httpbin service which I use for this article has 12 API endpoints. 12 endpoints multiplied by 4 methods to implement CRUD give 48 combinations. Plus every endpoint has some parameters... You can imagine how much work it is to manually write a WAF policy to accurately allowlist all these combinations. Now what if API layout changes? I think this is not something anyone want to spend time on.

Fortunately Open API file format was developed to become a single source of truth of an API definition. It formally describes entire API structure including endpoints, methods, parameters, etc.. Any tool can use it in order to interact with an API. WAF is not an exception. When open api file is referenced from a WAF policy NGINX App Protect automatically configures itself to allowlist all API resources. As a good side effect of it is that list of application specific resources becomes segregated from a WAF policy. This means you can update a policy or resources independently. Or even reuse same policy to protect different APIs.

As a demo I have built a simple topology. Httpbin application instance as API backend, NGINX App protect as a WAF and WAF dashboards to visualize activity. All components are deployed on top of serverless AWS platforms including WAF. Serverless NGINX App Protect receives traffic, filters it and forwards to Httpbin backend. WAF dashboards visualize WAF activity based on logs received from NGINX App Protect. If you are not aware of what serverless NGINX App Protect or WAF dashboards are I encourage you to take a look to introductory articles.

At the beginning NGINX just forwards traffic to Httpbin instance and App Protect has default policy in place. Default policy is not aware of any API specific urls or parameters and only has some generic features enabled. Therefore any endpoint is accessible and all invalid requests reach application, put unnecessary load and expose breaches.

$ curl -k https://snap-alb-1333484792.us-east-2.elb.amazonaws.com/doesntexist
< HTTP/2 404
< date: Tue, 29 Sep 2020 00:12:47 GMT
< content-type: text/html
< content-length: 233
< access-control-allow-origin: *
< access-control-allow-credentials: true
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p>

To protect application from invalid traffic WAF needs to allow only requests to expected urls, methods, parameters. Using open api file from inside of policy helps to define all allowed resources in one line. Documentation says there are two ways of referencing an open api file:

as local file
as remote url

I will use local file in this example due to serverless WAF nature which follows gitops paradigm and stores all configuration files in a gitlab repo. So policy transforms to following one:

{
  "name": "app_protect_default_policy",
  "template": { "name": "POLICY_TEMPLATE_NGINX_BASE" },
  "applicationLanguage": "utf-8",
  "enforcementMode": "blocking",
  "open-api-files": [
     {
        "filename": "file://httpbin-openapi.json"
     }
  ]
}

From this point NGINX App Protect becomes aware of entire API structure. However default policy doesn't have all violations enabled to effectively block invalid requests based on API structure. Documentation also gives a set of violations to enable:

VIOL_FILE_UPLOAD_IN_BODY
VIOL_MANDATORY_REQUEST_BODY
VIOL_PARAMETER_LOCATION
VIOL_MANDATORY_PARAMETER
VIOL_JSON_SCHEMA
VIOL_PARAMETER_ARRAY_VALUE
VIOL_PARAMETER_VALUE_BASE64
VIOL_FILE_UPLOAD
VIOL_URL_CONTENT_TYPE
VIOL_PARAMETER_STATIC_VALUE
VIOL_PARAMETER_VALUE_LENGTH
VIOL_PARAMETER_DATA_TYPE
VIOL_PARAMETER_NUMERIC_VALUE
VIOL_PARAMETER_VALUE_REGEXP
VIOL_URL
VIOL_PARAMETER
VIOL_PARAMETER_EMPTY_VALUE
VIOL_PARAMETER_REPEATED

{
  "name": "app_protect_default_policy",
  "template": { "name": "POLICY_TEMPLATE_NGINX_BASE" },
  "applicationLanguage": "utf-8",
  "enforcementMode": "blocking",
  "open-api-files": [
     {
        "filename": "file://httpbin-openapi.json"
     }
  ],
  "blocking-settings": {
            "violations": [
                {
                    "block": true,
                    "description": "Disallowed file upload content detected in body",
                    "name": "VIOL_FILE_UPLOAD_IN_BODY"
                },
        ...omitted...
            ]
        }
}

Once an open api file and a policy are committed to serverless NGINX App Protect repo and applied to the traffic all invalid requests aren't forwarded to a backend but receive blocking response page directly from WAF instead.

$ curl -kv https://snap-alb-1333484792.us-east-2.elb.amazonaws.com/doesntexist
{"supportID": "18105457996499725594"}

That is it. No more monstrous hand written WAF policies. Open api file reference makes WAF API aware in one line.

One more thing which a good WAF can't live without is visibility. NGINX App Protect is totally GUIless and the only way to get some insight of what does it do is to integrate it with external visibility tools like WAF Dashboards for Kibana. Integration process is simple. Once ELK stack is deployed just two steps left:

Configure NGINX App Protect to send logs to log processor (logstash)
Import dashboards to Kibana

Configuring WAF to send logs requires to commit two more directives to serverless NGINX App Protect repo. Once applied WAF starts to send logs to logstash and logstash in turn parses them and stores in elasticsearch.

app_protect_security_log_enable on;
app_protect_security_log "/etc/app_protect/conf/log_default.json" syslog:server=logstash.example.com:5144

Step of importing dashboards to Kibana isn't much harder. Documentation requires just couple commands:

KIBANA_URL=https://your.kibana:5601
jq -s . kibana/overview-dashboard.ndjson | jq '{"objects": . }' | \
curl -k --location --request POST "$KIBANA_URL/api/kibana/dashboards/import" \
    --header 'kbn-xsrf: true' \
    --header 'Content-Type: text/plain' -d @- \
    | jq

jq -s . kibana/false-positives-dashboards.ndjson | jq '{"objects": . }' | \
curl -k --location --request POST "$KIBANA_URL/api/kibana/dashboards/import" \
    --header 'kbn-xsrf: true' \
    --header 'Content-Type: text/plain' -d @- \
    | jq

Once done you can observe all NGINX App Protect activity on the dashboards. There are lots of useful information like top talkers, frequently accessed URLs, blocking reasons and so on. All this info helps to keep track of what WAF is up to and fine tune policy based on these insights.

NGINX App Protect gets new features quickly and becomes full featured, flexible and lighting fast WAF. Community keeps up and produces ecosystem of tools to make WAF operator experience even better. Feel free to explore tools I covered in this article and join the community to make live even better.

Tools:

Do not hesitate to contact me directly with questions of any kind about NGINX App Protect or any project used in this article.

Good luck!

Published Oct 14, 2020

Version 1.0

Ret. Employee

Joined December 09, 2019

View Profile

Mikhail_Fedoro1

Ret. Employee

Joined December 09, 2019

View Profile

$ curl -k https://snap-alb-1333484792.us-east-2.elb.amazonaws.com/doesntexist\n< HTTP/2 404\n< date: Tue, 29 Sep 2020 00:12:47 GMT\n< content-type: text/html\n< content-length: 233\n< access-control-allow-origin: *\n< access-control-allow-credentials: true\n<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>\n

{\n \"name\": \"app_protect_default_policy\",\n \"template\": { \"name\": \"POLICY_TEMPLATE_NGINX_BASE\" },\n \"applicationLanguage\": \"utf-8\",\n \"enforcementMode\": \"blocking\",\n \"open-api-files\": [\n {\n \"filename\": \"file://httpbin-openapi.json\"\n }\n ],\n \"blocking-settings\": {\n \"violations\": [\n {\n \"block\": true,\n \"description\": \"Disallowed file upload content detected in body\",\n \"name\": \"VIOL_FILE_UPLOAD_IN_BODY\"\n },\n ...omitted...\n ]\n }\n}\n

KIBANA_URL=https://your.kibana:5601\njq -s . kibana/overview-dashboard.ndjson | jq '{\"objects\": . }' | \\\ncurl -k --location --request POST \"$KIBANA_URL/api/kibana/dashboards/import\" \\\n --header 'kbn-xsrf: true' \\\n --header 'Content-Type: text/plain' -d @- \\\n | jq\n\njq -s . kibana/false-positives-dashboards.ndjson | jq '{\"objects\": . }' | \\\ncurl -k --location --request POST \"$KIBANA_URL/api/kibana/dashboards/import\" \\\n --header 'kbn-xsrf: true' \\\n --header 'Content-Type: text/plain' -d @- \\\n | jq\n

Protecting APIs with NGINX App Protect

ABOUT DEVCENTRAL

RESOURCES

SUPPORT

PARTNERS

ABOUT DEVCENTRAL

RESOURCES

SUPPORT

PARTNERS