NGINX Plus Request body Rate Limit with the NJS module and javascript

The nginx njs module allows javascript to process the code or as they call it on the backend nodejs. The module is dynamic for Nginx Plus https://docs.nginx.com/nginx/admin-guide/dynamic-modules/dynamic-modules/ while for the community nginx it needs to be compiled.

 

The code and nginx configuration are also present at:

https://github.com/Nikoolayy1/nginx_njs_request_body_limit/tree/main

 

I have used the example rate limiter from https://github.com/nginx/njs-examples and https://clouddocs.f5.com/training/community/nginx/html/class3/class3.html and modified rate limit example to be based on the request body.

It works as expected. The "r.internalRedirect('@app-backend');" internal redirect is needed as nginx by default does not populate or save the request body and this is why the request needs to pass 2 times in nginx proxy for the body variable to be properly populated!

 

The nginx plus rootless container is a great option for F5 XC RE where root containers are not accepted and for Nginx on XC RE I have made another article at F5 XC vk8s open source nginx deployment on RE | DevCentral

 

NJS "main" file code:

const defaultResponse = "0";
const user = 'username';
const pass = 'username';

function ratelimit(r) {


      switch (r.method) {


      case 'POST':

         var body  = r.requestText;

          r.log(`body: ${body}`);

     if (r.headersIn['Content-Type'] != 'application/x-www-form-urlencoded' || !body.length)

          {

              r.internalRedirect('@app-backend');

              return;

          }

           var result_user = body.includes(user);
           var result_pass = body.includes(pass);

                      if (!result_user) {

                              r.internalRedirect('@app-backend');

                             return;

                      }

    const zone = r.variables['rl_zone_name'];

    const kv = zone && ngx.shared && ngx.shared[zone];

    if (!kv) {

        r.log(`ratelimit: ${zone} js_shared_dict_zone not found`);

        r.internalRedirect('@app-backend');

        return;

    }

    const key = r.variables['rl_key'] || r.variables['remote_addr'];

    const window = Number(r.variables['rl_windows_ms']) || 60000;

    const limit = Number(r.variables['rl_limit']) || 10;

    const now = Date.now();


    let requestData = kv.get(key);

    if (requestData === undefined || requestData.length === 0) {

        requestData = { timestamp: now, count: 1 }

        kv.set(key, JSON.stringify(requestData));

        r.internalRedirect('@app-backend');

        return;

    }

    try {

        requestData = JSON.parse(requestData);

    } catch (e) {

        requestData = { timestamp: now, count: 1 }

        kv.set(key, JSON.stringify(requestData));

        r.internalRedirect('@app-backend');

        return;

    }

    if (!requestData) {

        requestData = { timestamp: now, count: 1 }

        kv.set(key, JSON.stringify(requestData));

        r.internalRedirect('@app-backend');

        return;

    }

    if (now - requestData.timestamp >= window) {

        requestData.timestamp = now;

        requestData.count = 1;

    } else {

        requestData.count++;

    }

    const elapsed = now - requestData.timestamp;

    r.log(`limit: ${limit} window: ${window} elapsed: ${elapsed}  count: ${requestData.count} timestamp: ${requestData.timestamp}`)

    let retryAfter = 0;

    if (requestData.count > limit) {

        retryAfter = 1;

    }
    kv.set(key, JSON.stringify(requestData));

   if (retryAfter) {

            r.return(401, "Unauthorized\n");

           return;

   }

      default:

           r.internalRedirect('@app-backend');

           return;

      }

  }

export default {sub, header, ratelimit, parseRequestBody, log};

 

Nginx nginx.conf file:

 

 server {
    listen       80 default_server;
    server_name  localhost;
   access_log  /var/log/nginx/host.access.log  main;

   js_var $rl_zone_name kv;          # shared dict zone name; requred variable
    js_var $rl_windows_ms 30000;      # optional window in miliseconds; default 1 minute window if not set
    js_var $rl_limit 3;              # optional limit for the window; default 10 requests if not set
    js_var $rl_key $remote_addr;      # rate limit key; default remote_addr if not set
    js_set $rl_result main.ratelimit; # call ratelimit function that returns retry-after value if limit is exceeded

    root /var/www/html;

    index  index.html;
    include       /etc/nginx/mime.types;

error_log  /var/log/nginx/host.error_log  debug;

if ($target) {
    return 401;
}

    location / {
       js_content main.ratelimit;

    }

     location @app-backend {
          internal;
          proxy_pass http://backend;
      }

location /backend {
   internal;
   proxy_set_header Host httpforever.com;
   proxy_pass http://backend/;

            }

Summary:

There is another example how to populate the internal request body variable using that is needed by the njs module using the " mirror " option, shown at https://www.f5.com/company/blog/nginx/deploying-nginx-plus-as-an-api-gateway-part-2-protecting-backend-services  but it did not work for me, so I used the " internal " option with "r.internalRedirect(uri)" https://nginx.org/en/docs/njs/reference.html

 

Nginx njs feature r.subrequest can be used to populate response headers and body but mainly it is for logging and not for rate limiting and I think making a real http subrequest using javascript is not optimal and will not scale well, so I will not recommend this option as rate limiters are best left to be request based. Also I saw strange bug that the subrequest changes the content type header of the response and I had use "js_header_filter" to again change the response header.Nginx App Protect has the BD process from F5 BIG-IP AWAF/ASM that has DOS protections that can monitor the Server's response latency dynamically and make auto thresholds!