AWS Auto Scale Group to F5 BIG-IP via a webhook

Polling vs. Pushing

Traditionally a Networking Engineer/Operator makes configuration changes on a BIG-IP. In a cloud environment, like Amazon Web Services (AWS), we expect that the infrastructure will propagate the changes without the need for manual intervention.

The following article describes how to use Amazon Web Services (AWS) Auto Scale and AWS Simple Notification Service (SNS) to trigger a dynamic update of a server pool on the F5 BIG-IP. A summary of the process will be; AWS Auto Scale group will notify AWS SNS of any changes, SNS will send a notification to an iRuleLX process on the BIG-IP that will trigger an update of the pool on the BIG-IP.

TMOS has supported the ability to configure a pool from an AWS Auto Scale group since version 12.1.0. The updates are triggered by periodically polling the AWS API (default 10 minutes). The workflow is the following:

BIG-IP contacts the AWS API to check for changes to an autoscale group
BIG-IP updates pool with members from AWS Auto Scale group
Wait 10 minutes

In some cases you may want to trigger an update of a pool when an AWS Auto Scale group launches or terminates a new instance. Ideally something like (implemented later in this article)

AWS Auto Scale Group changes
Triggers notification to BIG-IP
BIG-IP contacts the AWS API to check for changes to an Auto Scale group
BIG-IP updates pool with members from AWS Auto Scale group

This changes the workflow from a polling update to a push model. To make this work we’ll have to take advantage of a combination of F5 and AWS solutions. The following are a set of the leveraged technologies.

Technologies

Webhook

This term refers to a mechanism to provide an alternative to polling for an update. A service will subscribe for notifications and when a notification is ready it will push the update back to the URL that was specified to receive the update. Common use-cases are used with services like GitHub / Slack for handling things like notification when code is committed and sending a message to a Slack channel. A precursor to the Slack example would have been an IRC bot.

AWS Auto Scaling groups

AWS Auto Scale Groups provide an option for an administrator to maintain a group of servers that can be dynamically scaled up or down. When changes are made to an AWS Auto Scale group an administrator can opt to send a notification to the AWS Simple Notification Service (SNS).

AWS Simple Notification Service

AWS SNS provides a mechanism for receiving events/topic notifications and sending messages to subscribers of those topics. SNS can send an e-mail notification or in the case of this article we will use HTTP/HTTPS to send a message to a BIG-IP.

SNS requires that when you configure an HTTP/HTTPS message that you confirm via a subscription URL that is sent with the initial request.

F5 iRuleLX

iRuleLX offers the ability to extend the TCL based iRule and use a language like JavaScript to perform more advanced functionality. In this article we’ll use a 3^rd party Node.JS (JavaScript runtime) library to validate a SNS message. Something that would be challenging in TCL but easy in JavaScript. See also “Getting Started with iRules LX”.

F5 iCall

iCall provides the ability to trigger TMSH commands based on events. For the TMOS 12.1.0 AWS Auto Scale Group support the BIG-IP uses iCall to run a BASH script every 10 minutes. iCall can use other event triggers including an update to iStats.

F5 iStats

iStats provides the ability to create custom data counters (i.e. track usage of different SSL ciphers). For this article it will be used to trigger the update of the BIG-IP pool. This idea is taken from the following article.

Step-by-step

Now that we’ve reviewed the technologies involved let’s take a look at how to stich these together to make a dynamic solution.

Create Auto Scale configuration

Start by following the directions for Auto Scaling Application Servers in AWS.

Create SNS Topic

Create a unique topic that will be used only for the BIG-IP. Configure it to send either an HTTP/HTTPS request to an Elastic IP (public). This EIP will have the iRuleLX that will trigger the update.

Create Auto Scale group Notification

From the Auto Scale Notifications tab configure a notification to be sent on launch/terminate to the SNS topic that you previously created.

Create iRuleLX

iRuleLX has two components. A TCL script (a.k.a. plain old iRule) and JavaScript. The iRule part is the following.

# 
# aws_sns_irule
#
# Receive SNS Message at a specific URI.
# Capture POST data and send to iRuleLX process
# When Notification message is received update iStats object
#
when HTTP_REQUEST {
  # only allow requests to specific/magic URI.  SNS requests will originate from
  # AWS IP space.  Ideally also use SSL.
  
  set magic_uri "/unicorns_and_rainbows_2016"
  
  # grab POST data from body of request
  if {[HTTP::method] eq "POST" && [HTTP::uri] eq $magic_uri }{
    # Trigger collection for up to 1MB of data
    if {[HTTP::header "Content-Length"] ne "" && [HTTP::header "Content-Length"] <= 1048576}{
      set content_length [HTTP::header "Content-Length"]
    } else {
      set content_length 1048576
    }
    # Check if $content_length is not set to 0
    if { $content_length > 0} {
      HTTP::collect $content_length
    }
  } else {
  # only accept POST requests 
    reject
  }
}

when HTTP_REQUEST_DATA {
  # process POST body (JSON)
  set payload [HTTP::payload]
  set RPC_HANDLE [ILX::init aws_sns_plugin aws_sns_nodejs]
  set rpc_response [ILX::call $RPC_HANDLE process_aws_sns $payload]
  if { $rpc_response eq "Notification" } {
    log local0. "Updating AWS Auto Scale via webhook"
    ISTATS::set "AWS autoscale string erchen-autoscale-group" 1
  } elseif { $rpc_response eq "Error"} {
    log local0. "Bad request"
    HTTP::respond 500 content "Error"
    return
  }
  HTTP::respond 200 content "Success"
}

The iRule code receives a POST request from SNS and sends the body of the request (JSON payload) to iRuleLX to process the payload. After processing the payload in iRuleLX it will update an iStats key that will be used to trigger an iCall script that invokes the BASH script that updates the BIG-IP pool.

The JavaScript part (index.js) is the following.

/* Import modules. 
 * f5-nodejs: required
 * https: used to fetch subscription URL
 * sns-validator: used to verify SNS message
 */
var f5 = require('f5-nodejs');
var https = require('https');
var MessageValidator = require('sns-validator'),
    validator = new MessageValidator();

/* Create a new rpc server for listening to TCL iRule calls. */
var ilx = new f5.ILXServer();

ilx.addMethod('process_aws_sns', function(req, res) {
/* parse message body as JSON.  Error out if invalid */
    try {
        var payload = JSON.parse(req.params(0));
    } catch (err) {
        res.reply('Error');
        return;
    }
/* Use sns-validator to ensure valid SNS message */
    validator.validate(payload, function(err, payload) {
        if (err) {
            console.error(err);
            res.reply('Error');
            return;
        }
/* in case of Subscription message.  Send GET request. */
        if (payload.Type == 'SubscriptionConfirmation') {
            console.log('Subscribing to SNS Topic');
            console.log(payload.SubscribeURL);
            https.get(payload.SubscribeURL);
        }
        /* return SNS message Type */
        res.reply(payload.Type);
    });

});

/* Start listening for ILX::call and ILX::notify events. */
ilx.listen();

The JavaScript code loads the “https” module for making a subscription request and the “sns-validator” module (use NPM to install) for validating the SNS message. When a message is received from the iRule it first parses the message into JSON and validates the message. Any errors are logged. In the case that the message includes a subscription request the BIG-IP will make a GET request to the URL that is embedded in the message.

Modify iCall

The iRule triggers an update by updating an iStats object. To reset the trigger you need to modify the iCall script to unset the iStats object.

sys icall script autoscale {
    app-service none
    definition {
        exec /usr/libexec/aws/autoscale/aws-autoscale-pool-manager.sh
        exec istats remove "AWS autoscale string erchen-autoscale-group"
    }
    description none
    events none
}

This can be modified via TMSH running “edit /sys icall script autoscale”.

Attach iRuleLX to Virtual Server

Create a VIrtual Server that will host the webhook. You could dedicate a separate Virtual Server or allocate a non-standard port. For security you should utilize SSL, but for testing you can use a non-SSL VS. SNS needs to be able to communicate with a public IP. Make sure that you have an Elastic IP attached to the VS that you choose to use and an appropriate security group to allow communication.

Create Subscription

Now that you have the BIG-IP setup to receive a webhook you can go back into AWS SNS to create a subscription to the topic that you created earlier. Specify the Elastic IP / URI that you configured previously (i.e. http(s)://[Elastic IP]/[configured URI]).

All Together

Once everything has been plumbed together you can test it out by making a change to your Auto Scale group. The following are some sample logs of the initial subscription to the SNS Topic and receiving a notification from the Auto Scale group.

Nov 23 07:08:49 ip-10-1-1-59 info sdmd[6129]: 018e0017:6: pid[27803] plugin[/Common/aws_sns_plugin.aws_sns_nodejs] Subscribing to SNS Topic
Nov 23 07:08:49 ip-10-1-1-59 info sdmd[6129]: 018e0017:6: pid[27803] plugin[/Common/aws_sns_plugin.aws_sns_nodejs] https://sns.us-east-1.amazonaws.com/?Action=ConfirmSubscription&TopicArn=arn:aws:sns:us-east-1:XXX:trigger-f5-aws-autoscale-pool-manager&Token=XXX
Nov 23 07:09:50 ip-10-1-1-59 info tmm[10437]: Rule /Common/aws_sns_plugin/aws_sns_irule : Updating AWS Auto Scale via webhook
Nov 23 07:09:51 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Starting.
Nov 23 07:09:52 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Using region us-east-1
Nov 23 07:09:52 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Using AutoScaling Url http://autoscaling.us-east-1.amazonaws.com
Nov 23 07:09:52 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Updating pool : erchen-autoscale-group_pool with instances from Auto Scale Group : erchen-autoscale-group
Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Auto Scale Group : erchen-autoscale-group contains 2 instances
Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : erchen-autoscale-group : 10.1.20.210 10.1.20.105
Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : erchen-autoscale-group_pool contains 1 instances
Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : erchen-autoscale-group_pool : 10.1.20.210
Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Adding ip-10_1_20_105 with ip 10.1.20.105 to pool erchen-autoscale-group_pool

Compared to the Auto Scale group log you can see that the BIG-IP picked up the change very quickly.