AWS Auto Scale Group to F5 BIG-IP via a webhook
Polling vs. Pushing Traditionally a Networking Engineer/Operator makes configuration changes on a BIG-IP. In a cloud environment, like Amazon Web Services (AWS), we expect that the infrastructure will propagate the changes without the need for manual intervention. The following article describes how to use Amazon Web Services (AWS) Auto Scale and AWS Simple Notification Service (SNS) to trigger a dynamic update of a server pool on the F5 BIG-IP. A summary of the process will be; AWS Auto Scale group will notify AWS SNS of any changes, SNS will send a notification to an iRuleLX process on the BIG-IP that will trigger an update of the pool on the BIG-IP. TMOS has supported the ability to configure a pool from an AWS Auto Scale group since version 12.1.0. The updates are triggered by periodically polling the AWS API (default 10 minutes). The workflow is the following: BIG-IP contacts the AWS API to check for changes to an autoscale group BIG-IP updates pool with members from AWS Auto Scale group Wait 10 minutes In some cases you may want to trigger an update of a pool when an AWS Auto Scale group launches or terminates a new instance. Ideally something like (implemented later in this article) AWS Auto Scale Group changes Triggers notification to BIG-IP BIG-IP contacts the AWS API to check for changes to an Auto Scale group BIG-IP updates pool with members from AWS Auto Scale group This changes the workflow from a polling update to a push model. To make this work we’ll have to take advantage of a combination of F5 and AWS solutions. The following are a set of the leveraged technologies. Technologies Webhook This term refers to a mechanism to provide an alternative to polling for an update. A service will subscribe for notifications and when a notification is ready it will push the update back to the URL that was specified to receive the update. Common use-cases are used with services like GitHub / Slack for handling things like notification when code is committed and sending a message to a Slack channel. A precursor to the Slack example would have been an IRC bot. AWS Auto Scaling groups AWS Auto Scale Groups provide an option for an administrator to maintain a group of servers that can be dynamically scaled up or down. When changes are made to an AWS Auto Scale group an administrator can opt to send a notification to the AWS Simple Notification Service (SNS). AWS Simple Notification Service AWS SNS provides a mechanism for receiving events/topic notifications and sending messages to subscribers of those topics. SNS can send an e-mail notification or in the case of this article we will use HTTP/HTTPS to send a message to a BIG-IP. SNS requires that when you configure an HTTP/HTTPS message that you confirm via a subscription URL that is sent with the initial request. F5 iRuleLX iRuleLX offers the ability to extend the TCL based iRule and use a language like JavaScript to perform more advanced functionality. In this article we’ll use a 3 rd party Node.JS (JavaScript runtime) library to validate a SNS message. Something that would be challenging in TCL but easy in JavaScript. See also “Getting Started with iRules LX”. F5 iCall iCall provides the ability to trigger TMSH commands based on events. For the TMOS 12.1.0 AWS Auto Scale Group support the BIG-IP uses iCall to run a BASH script every 10 minutes. iCall can use other event triggers including an update to iStats. F5 iStats iStats provides the ability to create custom data counters (i.e. track usage of different SSL ciphers). For this article it will be used to trigger the update of the BIG-IP pool. This idea is taken from the following article. Step-by-step Now that we’ve reviewed the technologies involved let’s take a look at how to stich these together to make a dynamic solution. Create Auto Scale configuration Start by following the directions for Auto Scaling Application Servers in AWS. Create SNS Topic Create a unique topic that will be used only for the BIG-IP. Configure it to send either an HTTP/HTTPS request to an Elastic IP (public). This EIP will have the iRuleLX that will trigger the update. Create Auto Scale group Notification From the Auto Scale Notifications tab configure a notification to be sent on launch/terminate to the SNS topic that you previously created. Create iRuleLX iRuleLX has two components. A TCL script (a.k.a. plain old iRule) and JavaScript. The iRule part is the following. # # aws_sns_irule # # Receive SNS Message at a specific URI. # Capture POST data and send to iRuleLX process # When Notification message is received update iStats object # when HTTP_REQUEST { # only allow requests to specific/magic URI. SNS requests will originate from # AWS IP space. Ideally also use SSL. set magic_uri "/unicorns_and_rainbows_2016" # grab POST data from body of request if {[HTTP::method] eq "POST" && [HTTP::uri] eq $magic_uri }{ # Trigger collection for up to 1MB of data if {[HTTP::header "Content-Length"] ne "" && [HTTP::header "Content-Length"] <= 1048576}{ set content_length [HTTP::header "Content-Length"] } else { set content_length 1048576 } # Check if $content_length is not set to 0 if { $content_length > 0} { HTTP::collect $content_length } } else { # only accept POST requests reject } } when HTTP_REQUEST_DATA { # process POST body (JSON) set payload [HTTP::payload] set RPC_HANDLE [ILX::init aws_sns_plugin aws_sns_nodejs] set rpc_response [ILX::call $RPC_HANDLE process_aws_sns $payload] if { $rpc_response eq "Notification" } { log local0. "Updating AWS Auto Scale via webhook" ISTATS::set "AWS autoscale string erchen-autoscale-group" 1 } elseif { $rpc_response eq "Error"} { log local0. "Bad request" HTTP::respond 500 content "Error" return } HTTP::respond 200 content "Success" } The iRule code receives a POST request from SNS and sends the body of the request (JSON payload) to iRuleLX to process the payload. After processing the payload in iRuleLX it will update an iStats key that will be used to trigger an iCall script that invokes the BASH script that updates the BIG-IP pool. The JavaScript part (index.js) is the following. /* Import modules. * f5-nodejs: required * https: used to fetch subscription URL * sns-validator: used to verify SNS message */ var f5 = require('f5-nodejs'); var https = require('https'); var MessageValidator = require('sns-validator'), validator = new MessageValidator(); /* Create a new rpc server for listening to TCL iRule calls. */ var ilx = new f5.ILXServer(); ilx.addMethod('process_aws_sns', function(req, res) { /* parse message body as JSON. Error out if invalid */ try { var payload = JSON.parse(req.params(0)); } catch (err) { res.reply('Error'); return; } /* Use sns-validator to ensure valid SNS message */ validator.validate(payload, function(err, payload) { if (err) { console.error(err); res.reply('Error'); return; } /* in case of Subscription message. Send GET request. */ if (payload.Type == 'SubscriptionConfirmation') { console.log('Subscribing to SNS Topic'); console.log(payload.SubscribeURL); https.get(payload.SubscribeURL); } /* return SNS message Type */ res.reply(payload.Type); }); }); /* Start listening for ILX::call and ILX::notify events. */ ilx.listen(); The JavaScript code loads the “https” module for making a subscription request and the “sns-validator” module (use NPM to install) for validating the SNS message. When a message is received from the iRule it first parses the message into JSON and validates the message. Any errors are logged. In the case that the message includes a subscription request the BIG-IP will make a GET request to the URL that is embedded in the message. Modify iCall The iRule triggers an update by updating an iStats object. To reset the trigger you need to modify the iCall script to unset the iStats object. sys icall script autoscale { app-service none definition { exec /usr/libexec/aws/autoscale/aws-autoscale-pool-manager.sh exec istats remove "AWS autoscale string erchen-autoscale-group" } description none events none } This can be modified via TMSH running “edit /sys icall script autoscale”. Attach iRuleLX to Virtual Server Create a VIrtual Server that will host the webhook. You could dedicate a separate Virtual Server or allocate a non-standard port. For security you should utilize SSL, but for testing you can use a non-SSL VS. SNS needs to be able to communicate with a public IP. Make sure that you have an Elastic IP attached to the VS that you choose to use and an appropriate security group to allow communication. Create Subscription Now that you have the BIG-IP setup to receive a webhook you can go back into AWS SNS to create a subscription to the topic that you created earlier. Specify the Elastic IP / URI that you configured previously (i.e. http(s)://[Elastic IP]/[configured URI]). All Together Once everything has been plumbed together you can test it out by making a change to your Auto Scale group. The following are some sample logs of the initial subscription to the SNS Topic and receiving a notification from the Auto Scale group. Nov 23 07:08:49 ip-10-1-1-59 info sdmd[6129]: 018e0017:6: pid[27803] plugin[/Common/aws_sns_plugin.aws_sns_nodejs] Subscribing to SNS Topic Nov 23 07:08:49 ip-10-1-1-59 info sdmd[6129]: 018e0017:6: pid[27803] plugin[/Common/aws_sns_plugin.aws_sns_nodejs] https://sns.us-east-1.amazonaws.com/?Action=ConfirmSubscription&TopicArn=arn:aws:sns:us-east-1:XXX:trigger-f5-aws-autoscale-pool-manager&Token=XXX Nov 23 07:09:50 ip-10-1-1-59 info tmm[10437]: Rule /Common/aws_sns_plugin/aws_sns_irule : Updating AWS Auto Scale via webhook Nov 23 07:09:51 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Starting. Nov 23 07:09:52 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Using region us-east-1 Nov 23 07:09:52 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Using AutoScaling Url http://autoscaling.us-east-1.amazonaws.com Nov 23 07:09:52 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Updating pool : erchen-autoscale-group_pool with instances from Auto Scale Group : erchen-autoscale-group Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Auto Scale Group : erchen-autoscale-group contains 2 instances Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : erchen-autoscale-group : 10.1.20.210 10.1.20.105 Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : erchen-autoscale-group_pool contains 1 instances Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : erchen-autoscale-group_pool : 10.1.20.210 Nov 23 07:09:56 ip-10-1-1-59 notice logger: aws-autoscale-pool-manager.sh : Adding ip-10_1_20_105 with ip 10.1.20.105 to pool erchen-autoscale-group_pool Compared to the Auto Scale group log you can see that the BIG-IP picked up the change very quickly. Programmable Proxy Both AWS and F5 provide environments that are highly programmable. Please share your ideas on how you’re building a programmable proxy in AWS.
F5 BIG-IP Instance Migration in AWS
The f5-aws-migrate.py is a Python 2.7 script that automates the migration ofan existing BIG-IP instance to a new instance using a different BIG-IP image in AWS while keeping all configurations identical. For a primer on F5 in AWS, check out these three excellent articles written by Chris Mutzel: F5 in AWS Part 1 - AWS Networking Basics F5 in AWS Part 2 - Running BIG-IP in an EC2 Virtual Private Cloud F5 in AWS Part 3 - Advanced Topologies and More on Highly Available Services As discussed in the articleF5 in AWS Part 2, there are two ways you can run BIG-IP in AWS: subscription (hourly or annual subscriptions) or bring your own license (BYOL). You might be running a BIG-IP instance in AWS on an hourly subscription and then decide to convert to an annual subscription instead. Or you might decide to convert the BIG-IP subscription instance to a BYOL instance instead after obtaining a F5 software license. To achieve this conversion (prior to the script) you must manually create a new BIG-IP instance with either an annual subscription or a F5 software license. Additionally, you may want to move BIG-IP instances for other reasons. For example, if you are trying to perform a complete mitigation on a BIG-IP instance impacted byCVE-2016-2084. According to the following Security Advisory on AskF5, anew BIG-IP instance needs to be created to replace the vulnerable instance (SOL 11772107: BIG-IP and BIG-IQ cloud image vulnerability CVE-2016-2084has more information on this vulnerability). The challenge with any of these scenarios is ensuring the new BIG-IP instance has the identical configuration as the old instance being migrated. This involves two major tasks. AWS configuration The new instance must be configured with the same ENIs (Elastic Network Interfaces), Security Groups, and Tags as the old instance. All of these settings need to be gathered from the old instance for use in configuring the new instance. In order to reuse the ENIs from the old instance, the DeleteOnTermination setting on each ENI needs to be set to False. The old instance needs to be terminated allowing detachment of the ENIs. Then you would create the new instance from the desiredBIG-IP image and manually configure it with the identical settings gathered from the old instance. BIG-IP configuration On the new BIG-IP instance, if using BYOL license, you must complete the licensing process. The BIG-IP UCS (User Configuration Set; a backup) file saved from the old instance needs to be restored onto the new instance. The result is a new instance created from the selected BIG-IP image with identical configuration as the old terminated instance. Performing all of these steps manually can be tedious and error prone. A solution automating the two major tasks described above is now available. F5 has created a Python 2.7 script, f5-aws-migrate.py, which automates the migration of one BIG-IP instance to another instance in AWS for the two types of BIG-IP images available on the AWS Marketplace. The script begins by gathering a BIG-IP UCS file and polling AWS to gather instance configuration details. It then terminates the original instance and launches a new, identical instance using the AMI image you specify. Finally, the script performs automated licensing and installs the UCS file from the original instance with a no-license flag to avoid overwriting the new license. The script can also perform complete BIG-IP mitigation steps forCVE-2016-2084. For more information, and to download the software see the ReadMe file on our F5 DevCentral Github repositoryhttps://github.com/f5devcentral/f5-aws-migrate
