Mitigate Unplanned Scale Issues with an iRule Waiting Room

JRahm · ‎14-Apr-2020

This solution originates on Tuncay Sahin’s website and is shared with minor updates with permission here. F5er Tim Wagner reached out after hearing news of the flood of unemployment claims causing site crashes to see if we’d take a look at this iRule. The iRule as-is performs the following functions:

Tracks connections by IP address using the table command (session table accessor)
In the event the visitor count exceeds the max visitors, send a service unavailable error (503) with a meta refresh at a 60 second interval
Upon retry, if the visitor count slips below the max visitors, forward to the server, otherwise, send the 503 again.
For explicit URIs, there is a stats or bypass mechanism

Note that it works as is as long as you add the appropriate data-group. Before getting into the small updates I chose to make, consider this solution to be a fancy sorry page. You can find several solutions on that front in the codeshare. The updates I made are simple:

Eliminate the management tool table and configuration to reduce complexity. I keep the status page details for troubleshooting (/getcount) however.
Rename all the VIP stuff to BYPASS to improve clarity. VIP has a special meaning for BIG-IP, keeping it would be confusing
Remove the embedded images and html pages and use iFiles instead to increase readability. I kept most of the variables for this reason as well, though in a highly performant situation I’d provide a clarity map in comments and seek to optimize for performance instead.
Added a js canvas drawing to reduce boredom in the waiting room. This is also served as an iFile.

The Solution

I left some optimizations on the floor as an exercise for the reader, let me know how you would further tweak this iRule!

when HTTP_REQUEST {
  ## Check if host name match otherwise exit.
  ## This is needed if you have multiple websites running on same Virtual Server
  if {[HTTP::host] eq "test.test.local"} {
    # waiting room js file, only necessary if you want a canvas animation
    if { [HTTP::uri] eq "/bb.js"} {
      HTTP::respond 200 content [ifile get bb.js]
      TCP::close
      return
    }
    ## Set variable
    #Your website (unique)shortcode, needed to divide multiple online waiting room iRules on same Virtual Server.
    #In this example the shortcode is SITE1
    set OWR SITE1
	# Max visitor count
	# How many concurrent visitors can you serve
	set max_visitors 2
	# Timeout in seconds
	# IdleTimeout value is based your cart ideltimeout value. Must at least be equal to your cart IdleTimeout value.
	set IdleTimeout 60
	set WaitingRoomTimeout 60
	# Decide vistors IP address. Visitors behind a proxy are seen for one visitor.
	if { ([HTTP::header exists "True-Client-IP"]) and ([HTTP::header "True-Client-IP"] != "") } {
	  set Client_IP [HTTP::header "True-Client-IP"]
	} else {
	    set Client_IP [IP::client_addr]
	}
	# Defining Tables
	set VisitorsTable VisitorsTable-$OWR-$max_visitors
	set WaitingRoomTable WaitingRoom-$OWR-$max_visitors
	# Generic
	set unique_id [format "%08d" [expr { int(100000000000 * rand()) }]]
	set request_uri [HTTP::host][HTTP::uri]
	set BYPASS $OWR-bypass-url-list
	# Counters
	set VisitorCount [table keys -subtable $VisitorsTable -count]
	set WaitingRoomCount [table keys -subtable $WaitingRoomTable -count]
	set TotalVisitors [expr {$VisitorCount + $WaitingRoomCount}]
    ## End Variable
    ## Monitoring
    # Allow monitoring from internal IP's or subnets.
    if { ($Client_IP starts_with "192.168.102") && ([HTTP::uri] equals "/getcount") } {
      HTTP::respond 200 content "Total Visitors: \[$TotalVisitors\]
Max Visitors: \[$max_visitors\]
Waiting Room Count: \[$WaitingRoomCount\]"
      TCP::close
      return
	}
    ## Start WaitingRoom
    # Check if the visitor session still exists
    set VisitorSession [table lookup -subtable $VisitorsTable $Client_IP]
    if { $VisitorSession != "" } {
      # We have a valid session... The lookup has reset the timer on it so just finish processing
    } else {
		# No valid session...
		# Check if BYPASS URL
		set bypass_url [class match -value [HTTP::uri] contains $BYPASS]
		if { not ($bypass_url == "") } {
			# BYPASS, do nothing
		} else {
			# NOT BYPASS, Check connection count for displaying WR Page
			# So do we have a free 'slot'?
			if {$VisitorCount < $max_visitors} {
			  # Yes we have a free slot... Allocate it..
			  # Register visitor
			  table add -subtable $VisitorsTable $Client_IP $unique_id $IdleTimeout
			} else {
			    # Max visitors limit reached, show WaitingRoom
				# Insert visitor into WaitingRoomTable
				table add -subtable $WaitingRoomTable $Client_IP $unique_id $WaitingRoomTimeout
				# Show waiting Room HTML
				HTTP::respond 503 content [ifile get waitingroom.html]
			 }
             TCP::close
		}
	}
  }
}

The Result

I set the max visitors and the idle timeout to ridiculously low values (2 and 60, respectively) to make it easy to test. I ran siege from from two linux virtual machines so I could test my desktop browser and sure enough, my third connection resulted in the waiting room:

I also tested the /getcount URI to see what my stats were:

Total Visitors: [3]
Max Visitors: [2]
Waiting Room Count: [1]

And finally, I tested the bypass, using a valid path from the data-group to bypass the waiting room. A query to /bypass_test resulted in a 404 (as that URI is dead link currently) instead of the 503 I should get on all non-bypass URIs, so this was successful as well.

What solutions are you looking at for handling temporary scale issues? Cloud bursting? Dynamic growth and shrinkage in kubernetes deployments? Let me know how you are handing these situations in the comments below. And a hearty shout out again to Tuncay Sahin for the original work here!

JRahm · ‎14-Apr-2020

Let me know if you have any questions, or any optimizations you'd make to the original or my update!

PeteWhite · ‎15-Apr-2020

Looks great Jason. If it were me I would simplify deployment by putting it into an iApp and have a boilerplate page with specific link for an icon and a page theme based on either w3.css ( where the admin can specify the main colour theme ) or Bootstrap, again allowing themes.

Rather than using a table based on IP, a cookie on the client with a session ID would offload the task from the BIG-IP and reduce memory usage and speed up execution. Requests with a valid cookie should be preferred over non-cookie requests, to implement some kind of queueing.

JRahm · ‎15-Apr-2020

all good ideas, especially deployment simplification and offloading the compute from the BIG-IP!

lnxgeek · ‎17-Apr-2020

Hi jason

Can you share the bb.js and waitingroom.html?

Thanks.

MichaelOLeary · ‎19-Apr-2020

Nice solution to have in the back pocket, thanks for sharing.

JRahm · ‎20-Apr-2020

waitingroom.html:

<html>
    <head>
        <meta http-equiv="refresh" content="60">
        <title>Online Waiting Room</title>
        <style>
        </style>
        <script type="text/javascript" src="bb.js"></script>
    </head>
    
 
    <body onload="init();">
        <center>
            <h2>Online Waiting Room</h2>
            <h3>Hey there...sorry about the wait!</h3>
            <p>We currently have an exceptionally large number of visitors on the site and you are in the queue.</p>
			<p>Please hold tight, it should only be a few minutes. Make sure you stay on this page and you will be automatically redirected shortly.</p>
            <div>
                <canvas id="canvas" width="800" height="600">
                    <p>Your browser doesn't support canvas.</p>
                </canvas>
            </div>
            <img src="bar.png" id="bar" style="display:none"/>
        </center>
    </body>
</html>

bb.js (found it on the web and updated it, it wasn't sited inline and I forgot to grab the URL so no citation. Sorry!)

var canvas;
var ctx;
var dx = 1;
var dy = 2;
var bar=new Bar(400,500);
var circle=new Circle(400,30,10);
var dxBar=6;
var timer;
var barImg;
function Bar(x,y){
  this.x=x;
  this.y=y;
}
function Circle(x,y,r){
  this.x=x;
  this.y=y;
  this.r=r;
}
function drawBall(c) {
  ctx.beginPath();
  ctx.arc(c.x, c.y, c.r, 0, Math.PI*2, true);
  ctx.fill();
}
function doKeyDown(e){
  if(e.keyCode==37){
    if(bar.x-dxBar>0)
      bar.x-=dxBar;
  }
  else if(e.keyCode==39){
    if(bar.x+dxBar<canvas.width)
      bar.x+=dxBar;
  }
}
function init() {
  window.addEventListener("keydown",doKeyDown,false);
  barImg=document.getElementById("bar");
  canvas = document.getElementById("canvas");
  ctx = canvas.getContext("2d");
  timer=setInterval(draw, 3);
  return timer;
}
function draw() {
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  ctx.fillStyle = "#FAF7F8";
  ctx.fillRect(0,0,canvas.width,canvas.height);
  ctx.fillStyle = "#003300";
  drawBall(circle);
  if (circle.x +dx > canvas.width || circle.x +dx < 0)
    dx=-dx;
  if(circle.y+dy>bar.y && circle.x>bar.x && circle.x<bar.x+barImg.width)
    dy=-dy;
  if (circle.y +dy > canvas.height || circle.y +dy < 0)
    dy=-dy;
  circle.x += dx;
  circle.y += dy;
  ctx.drawImage(barImg,bar.x,bar.y);
  if(circle.y>bar.y){
    clearTimeout(timer);
    ctx.clearRect(0, 0, canvas.width, canvas.height);
    alert("Game Over");
  }
}

JRahm · ‎20-Apr-2020

I also had another version similar to the one I used in my iFile article years back, updated for the new "Jimmy Packets" mascot. The JS is embedded in this one though, I'd recommend stripping it out into it's own file if you go this route. I used a 50x50 pixel image.

<html>
    <head>
        <meta http-equiv="refresh" content="60">
        <title>Online Waiting Room</title>
        <style>
        </style>
    </head>
    <script>
        var surface;
        var happy;
        var x = 25;
        var y = 25;
        var dirX = 1;
        var dirY = 1;
 
        function drawCanvas() {
            // Get our Canvas element
            surface = document.getElementById("myCanvas");
           
            if (surface.getContext) {
                // If Canvas is supported, load the image
                dcjp = new Image();
                dcjp.onload = loadingComplete;
                dcjp.src="dcjp_50px.png";
            }
        }
 
        function loadingComplete(e) {
            // When the image has loaded begin the loop
            setInterval(loop, 5);
        }
 
        function loop() {
            // Each loop we move the image by altering its x/y position
           
            // Grab the context
            var surfaceContext = surface.getContext('2d');
           
            // Draw the image
            surfaceContext.drawImage(dcjp, x, y);
 
            x += dirX;
            y += dirY;
 
            if (x <= 0 || x > 700 - 25) {
                dirX = -dirX;
            }
            if (y <= 0 || y > 350 - 40) {
                dirY = -dirY;
            }
        }
    </script>
 
    <body onload="drawCanvas();">
        <center>
            <h2>Online Waiting Room</h2>
            <h3>Hey there...sorry about the wait!</h3>
            <p>We currently have an exceptionally large number of visitors on the site and you are in the queue.</p>
			<p>Please hold tight, it should only be a few minutes. Make sure you stay on this page. Be mesmerized below, and you will be automatically redirected shortly.</p>
            <div>
                <canvas id="myCanvas" width="700" height="350">
                    <p>Your browser doesn't support canvas.</p>
                </canvas>
            </div><br>
        </center>
    </body>
</html>

DevCentral

Mitigate Unplanned Scale Issues with an iRule Waiting Room

The Solution

The Result