Understanding iApps
Understanding iApps iApps are powerful tools. Used by roughly one third of all F5 customers, they perform input validation and apply complex business logic for configuring a wide variety of applications. They hide complexity, sometimes driving hundreds of configuration parameters with just a handful of input values. They also provide deployment guidance, reducing the need for documentation and training. But iApps are often misunderstood and misused, sometimes with frustrating consequences. iApp, not Wizard It is helpful to understand the difference between an iApp and a wizard. A wizard is usually a script with a GUI. It is a tool that accepts a set of user inputs and performs a one-time procedure. When run twice, a wizard performs exactly the same steps during its second run as it did during its first. Although an iApp is also a script with a GUI, it behaves differently on re-entry. Unlike a wizard, an iApp maintains a relationship to the configuration that it generates. iApps have 5 critical properties: iApps always act atomically. The result of deployment is always either the entire intended configuration or none at all. iApp-driven configuration objects are marked so the iApp can track them throughout its lifecycle. A visualization of these configuration objects is presented in the popular TMUI “Component View.” iApps protect the configuration from accidental changes. iApp-driven elements may not be changed via the UI or CLI except through the iApp. iApps support re-entry. Since the iApp framework tracks the configuration objects that it manages, it can be intelligent about which elements are touched during reconfiguration. iApps automatically perform cleanup on deletion. The housekeeping is automatic, and the iApp author does not need to (and is not allowed to) write any delete-time code. Mark and Sweep On initial deployment, an iApp builds a configuration as directed by the TMSH commands in the iApp’s implementation section. On reconfiguration, the iApp framework again runs the implementation, but does not immediately submit the result to BIG-IP. Instead, it first compares the desired (new) configuration with the existing (old) configuration. All similarities are dropped from the workload. If existing elements are missing from the desired configuration, the framework automatically prepares delete requests for those elements. In the end, the changes submitted to BIG-IP are the minimum necessary to accomplish the desired configuration. TMSH “create” commands used to build config on the initial deployment are automatically changed to “modify” on re-entry. This mechanism, known as “mark-and-sweep,” reduces a scripted change set to its net effect, often allowing iApps to reconfigure BIG-IP with no disruption to data plane traffic. Strict Updates As noted above, an iApp tags all of the configuration objects under its control and BIG-IP prevents users from modifying these objects outside the iApp. This restriction, called “strict-updates”, can be disabled. The negative consequences of this action are often underestimated. To illustrate the typical frustration, consider a simple iApp that accepts 2 inputs and constructs a virtual server with a custom HTTP profile: presentation { section virtual { string ip required choice xff { “enabled”, “disabled” } } } implementation { tmsh::create ltm profile http ${tmsh::app_name}_http \ insert-xforwarded-for $::virtual__xff tmsh::create ltm virtual ${tmsh::app_name}_vs \ destination $::virtual__ip:80 \ profiles add \{ ${tmsh::app_name}_http \} } Suppose you deploy this iApp with X-Forwarded-For enabled, then disable strict-updates and edit the HTTP profile directly as shown: Now, re-enter and re-deploy the iApp without changing any inputs. What is the result? You might expect your manual customization of response chunking to remain in place, because there is no mention of response chunking in the iApp code. In this case, however, iApp re-deployment causes chunking to be set back to its default value. Consider another change, this time to the virtual server: Disable address translation, then re-enter and re-deploy the iApp again. What is the result this time? You might expect your customization to be overwritten as it was in the previous example. In this case, however, address translation remains disabled. The TMSH “modify” command does not handle defaults consistently across all key-value pairs. This makes tampering with an iApp-enforced configuration especially hazardous. Right Use Hopefully, this article provides some insights that will help you use iApps to your best advantage. An iApp is a scripted management tool that is dedicated to an application for life. If you find yourself wishing to disengage the iApp and tamper with its configuration, instead consider modifying the iApp code to accomplish your goals. If you absolutely must disable strict-updates, do so with crystal-clear expectations for managing that application going forward.
F5 Automated Backups - The Right Way
Hi all, Often I've been scouring the devcentral fora and codeshares to find that one piece of handywork that will drastically simplify my automated backup needs on F5 devices. Based on the works of Jason Rahm in his post "Third Time's the Charm: BIG-IP Backups Simplified with iCall" on the 26th of June 2013, I went ahead and created my own iApp that pretty much provides the answers for all my backup-needs. Here's a feature list of this iApp: It allows you to choose between both UCS or SCF as backup-types. (whilst providing ample warnings about SCF not being a very good restore-option due to the incompleteness in some cases) It allows you to provide a passphrase for the UCS archives (the standard GUI also does this, so the iApp should too) It allows you to not include the private keys (same thing: standard GUI does it, so the iApp does it too) It allows you to set a Backup Schedule for every X minutes/hours/days/weeks/months or a custom selection of days in the week It allows you to set the exact time, minute of the hour, day of the week or day of the month when the backup should be performed (depending on the usefulness with regards to the schedule type) It allows you to transfer the backup files to external devices using 4 different protocols, next to providing local storage on the device itself SCP (username/private key without password) SFTP (username/private key without password) FTP (username/password) SMB (using smbclient, with username/password) Local Storage (/var/local/ucs or /var/local/scf) It stores all passwords and private keys in a secure fashion: encrypted by the master key of the unit (f5mku), rendering it safe to store the backups, including the credentials off-box It has a configurable automatic pruning function for the Local Storage option, so the disk doesn't fill up (i.e. keep last X backup files) It allows you to configure the filename using the date/time wildcards from the tcl [clock] command, as well as providing a variable to include the hostname It requires only the WebGUI to establish the configuration you desire It allows you to disable the processes for automated backup, without you having to remove the Application Service or losing any previously entered settings For the external shellscripts it automatically generates, the credentials are stored in encrypted form (using the master key) It allows you to no longer be required to make modifications on the linux command line to get your automated backups running after an RMA or restore operation It cleans up after itself, which means there are no extraneous shellscripts or status files lingering around after the scripts execute I wasn't able to upload the iApp template to this article, so I threw it on pastebin: http://pastebin.com/YbDj3eMN Enjoy! Thomas Schockaert
Creating, Importing and Assigning a CA Certificate Bundle
Within this article, I will be using a personal and relative use case to my own customers. While many organizations may only have one or two Root CA's to identify, the US Department of Defense has numerous CA's sometimes making it difficult for new F5 admins to grasp the concept of a certificate bundle and where to use it. In this article I wanted to take just a few minutes to walk you through the creation of a CA bundle, importing it into the BIG-IP and where you would apply the bundle to perform functions such as smart card authentication. If you would like to attempt to use the cert bundle iApp created by F5, the iApp deployment guide can be found using the link below though deploying that iApp is outside the scope of this document. With all of that, let's begin. https://f5.com/solutions/deployment-guides/ca-bundle-iapp-big-ip-v115-v12 Obtaining all CA's to Include in the Bundle For DoD customers, navigate to https://iase.disa.mil/pki-pke/Pages/index.aspx Select For Administrators, Integrators and Developers Select Tools and continue to browse until you locate PKI CA Certificate Bundles: PKCS#7 Select and download the certificate bundle that is appropriate for your organization though as an example I have selected For DoD PKI Only. Locate the Zip file in the directory you downloaded it to, right click and select extract. Copy Required Files to the BIG-IP After you have extracted all of the files within the .zip file, using your preferred SCP application, copy the file name with the extension of .pem.p7b as shown in this example Certificates_PKCS7_v5.3_DoD.pem.p7b. Note: For Windows users PowerShell can be used to SCP files though if you prefer WinSCP it is certainly acceptable. Run the command scp -p @: Convert the PKCS Certificate to PEM format SSH to the BIG-IP Navigate to the directory you placed the .p7b file during the SCP process above. Run the command openssl pkcs7 -in .p7b -text -out .pem -print_certs to convert the file from the p7b file format to pem. Once you have converted the file from p7b to pem format run the command tmsh install sys crypto cert from-local-file to import the file into the BIG-IP certificate store. To save the config, run the command save sys config Validate Cert Bundle was Imported Log into the BIG-IP TMUI >> System >> Certificate Management >> Traffic Certificate Management >> SSL Certificate List Locate the certificate name that you used in the previous step to import the pem file into the certificate store. To view the certificates included in that bundle, click the certificate bundle and within the certificate tab you can view all certificates. Assign the CA Bundle to a SSL Client Profile This task is most commonly used in SSL client profiles assigned to applications performing smart card or user certificate based authentication. Navigate to Local Traffic >> Profiles >> SSL >> Client Select the profile that will be used for client authentication Scroll until you reach Client Authentication From the drop down menu for Trusted Certificate Authorities, select the bundle created in the previous steps. From the drop down menu for Advertised Certificate Authorities, select the bundle created in the previous steps. Click Update The Trusted Certificate Authorities setting is required only if the BIG-IP system performs Client Certificate Authentication. This setting is specifies the BIG-IP system's Trusted Certificate Authorities store (the CAs that the BIG-IP system trusts when the system verifies a client certificate that is presented during Client Certificate Authentication). The Advertised Certificate Authorities setting is optional. You can use it to specify the CAs that the BIG-IP system advertises as trusted when soliciting a client certificate for client certificate authentication. If the Client Certificate setting is configured to Require or Request, you can configure the Advertised Certificate Authorities setting to send clients a list of CAs that the server is likely to trust. At this point you have successfully created, imported and assigned your new certificate bundle. If you would like to view a complete guide on configuring smart card authentication, please view my articles on DevCentral. Until next time!BIG-IP Interface Stats in Real Time with a TMSH Script
For the savants among us, calculating bits in and bits out over a delta from two snapshots of the interface counters is a walk in the park. For the rest of us, it's nice to have a tool to look at the current traffic load on an interface while working in the command line interface. This article will walk you through creating a TMSH script to do just that. Source Data You can get at interface data via snmp and icontrol, but is also available with the tmsh show net interface command. --------------------------------------------------------- Net::Interface Name Status Bits Bits Pkts Pkts Drops Errs Media In Out In Out --------------------------------------------------------- 1.1 up 59.4T 5.0T 6.2G 2.4G 0 0 none Yep, that's data. But when you get to terabits, the dial doesn't move quite so quickly, so taking a diff every few seconds won't amount to much. Specifying the raw option on the show net interface command helps out in that regard. (raw) ----------------------------------------------------------------------------------------- Net::Interface Name Status Bits Bits Pkts Pkts Drops Errs Media In Out In Out ----------------------------------------------------------------------------------------- 1.1 up 59485486972968 5080727699544 6291600606 2488751052 0 0 none That's better, but a little more challenging to parse than adding the field-fmt option, which puts it in a nice key value pair list. The bits-in and bits-out counters are the focus of this script. net interface 1.1 { counters.bits-in 59486479580896 counters.bits-out 5080875828888 counters.drops-all 0 counters.errors-all 0 counters.pkts-in 6291722759 counters.pkts-out 2488812198 media-active none name 1.1 status up } Now that we have key value pairs, and already separated by whitespace, this is a simple extraction once we split the entire string by newline. % split $x "\n" net\ interface\ 1.1\ \{ \ { counters.bits-in 59500356294368} \ { counters.bits-out 5082163022832} \ { counters.drops-all 0} \ { counters.errors-all 0} \ { counters.pkts-in 6293231170} \ { counters.pkts-out 2489470246} \ { media-active none} \ { name 1.1} \ { status up} \} \ {} % lindex [split $x "\n"] 1 counters.bits-in 59500356294368 % lindex [split $x "\n"] 2 counters.bits-out 5082163022832 % lindex [lindex [split $x "\n"] 1] 1 59500356294368 % lindex [lindex [split $x "\n"] 2] 1 5082163022832 Now that the data is extracted in proper form, we can move on to the script! Goals & Workflow The goals for this script are simple: take the values from counters.bits-in and counters.bits-out from a specified interface and display them at a specified refresh interval. We'll get from goals to a script by first working through some workflow: The Script Since we need to get data from the user (interface and interval specifications), let's start with the standard input. We'll use the getFeedback proc below. proc getFeedback { question } { puts -nonewline $question flush stdout return [gets stdin] } This proc pulls is then used in the initial script setup as shown next. tmsh::clear_screen if { $tmsh::argc == 1 } { set int [getFeedback "Please enter the interface number (ie, 1.1): "] } else { set int [lindex $tmsh::argv 1] } set l1 [] set l2 [] set interval [getFeedback "Please enter refresh rate for the stats (in seconds): "] set delay [expr $interval * 1000] Here we see the screen has been cleared, and then if the only argument in the script initialization is the script itself, then we ask for the interface name. Otherwise, we take the second argument value and set it as the interface name. Then, we initialize the l1 and l2 variables as lists. Finally, we ask for the desired refresh interval and set that delay for the after command use as it's argument is in milliseconds, not seconds. Next, we need to go ahead and take the data and dump it into the l1 variable we initialized: lappend l1 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 1] 1] lappend l1 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 2] 1] It looks a little scary, but this is an exact copy of the structure shown above in the Tcl shell except that we're using the TMSH command output instead of the static "x" variable we used to get the syntax necessary to extract the data. This results in l1 having a list with the bits-in and bits-out values in indexes 0 and 1 respectively. Now, the loop that allows this script to display the bit rate real time. while { true } { after $delay lappend l2 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 1] 1] lappend l2 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 2] 1] tmsh::clear_screen set statsIn [expr ([lindex $l2 0] - [lindex $l1 0]) / $interval] set statsOut [expr ([lindex $l2 1] - [lindex $l1 1]) / $interval] puts "Interface\t\tInbound (bps)\t\tOutbound (bps)" puts "$int\t\t\t$statsIn\t\t\t$statsOut" set l1 $l2 unset l2 } This loop will continue until you break it with a ctrl-c. We start the loop condition with our specified delay, then do with the l2 variable what we did with the l1 variable: take a snapshot of the bits-in and bits-out of the interface. After again clearing the screen, now we take the delta of the new snapshot and the old snapshot, and divide by the interval to get the bits transferred in and out on that interface, per second. Next, we display that to the screen with the puts command. Finally, in order to maintain the latest snapshot for the next interval, we set the l2 data to the l1 variable and unset the l2 variable. And that's it. Not that complicated, right? Going Forward This is a very simple throwaway script that needs a lot of work to have "arrived." Error checking, extensibility, etc, are missing, and are all left to the reader to develop for those purposes. This met a very specific troubleshooting need in my environment, and I would be remiss if I didn't share. I'd love to see someone take on error checking, or maybe displaying the bitrates for all interfaces if none is specified, or going a step further, summarizing all interfaces per vlan and showing vlan bitrates. Any takers? The script in its entirety is here in the TMSH codeshare.Scheduling BIG-IP Configuration Backups via the GUI with an iApp
Beginning with BIG-IP version 11, the idea of templates has not only changed in amazing and powerful ways, it has been extended to be far more than just templates. The replacement for templates is called iApp TM . But to call the iApp TM just a template would be woefully inaccurate and narrow. It does templates well, and takes the concept further by allowing you to re-enter a templated application and make changes. Previously, deploying an application via a template was sort of like the Ron Popeil rotisserie: “Set it, and forget it!” Once it was executed, the template process was over, it was up to you to track and potentially clean up all those objects. Now, the application service you create based on an iApp TM template effectively “owns” all the objects it created, so any change to the deployment adds/changes/deletes objects as necessary. The other exciting change from the template perspective is the idea of strictness. Once an application service is configured, any object created that is owned by that service cannot be changed outside of the service itself. This means that if you want to add a pool member, it must be done within the application service, not within the pool. You can turn this off, but what a powerful protection of your services! Update for v11.2 - https://devcentral.f5.com/s/Tutorials/TechTips/tabid/63/articleType/ArticleView/articleId/1090565/Archiving-BIG-IP-Configurations-with-an-iApp-in-v112.aspx The Problem I received a request from one of our MVPs that he’d really like to be able to allow his users to schedule configuration backups without dropping to the command line. Knowing that the iApp TM feature was releasing soon with version 11, I started to see how I might be able to coax a command line configuration from the GUI. In training, I was told that “anything you can do in tmsh, you can do with an iApp TM .” This is excellent, and the basis for why I think they are going to be incredibly popular for not only controlling and managing applications, but also for extending CLI functions to the GUI. Anyway, so in order to schedule a configuration backup, I need: A backup script A cron job to call said script That’s really all there is to it. The Solution Thankfully, the background work is already done courtesy of a config backup codeshare entry by community user Colin Stubbs in the Advanced Design & Config Wiki. I did have to update the following bigpipe lines from the script: bigpipe export oneline “${SCF_FILE}” to tmsh save /sys config one-line file “${SCF_FILE}” bigpipe export “${SCF_FILE}” to tmsh save /sys config file "${SCF_FILE}" bigpipe config save “${UCS_FILE}” passphrase “${UCS_PASSPHRASE}” to tmsh save /sys ucs "${UCS_FILE}" passphrase "${UCS_PASSPHRASE}" bigpipe config save “${UCS_FILE}” to tmsh save /sys ucs "${UCS_FILE}" Also, I created (according the script comments from the codeshare entry) a /var/local/bin directory to place the script and a /var/local/backups directory for the script to dump the backup files in. These are optional and can be changed as necessary in your deployment, you’ll just need to update the script to reflect your file system preferences. Now that I have everything I need to support a backup, I can move on to the iApp TM template configuration. iApp TM Components A template consists of three parts: implementation, presentation, and help. You can create an empty template, or just start with presentation or help if you like. The implementation is tmsh script language, based on the Tcl language so loved by all of us iRulers. Please reference the tmsh wiki for the available tmsh extensions to the Tcl language. The presentation is written with the Application Presentation Language, or APL, which is new and custom-built for templates. It is defined on the APL page in the iApp TM wiki. The help is written in HTML, and is used to guide users in the use of the template. I’ll focus on the presentation first, and then the implementation. I’ll forego the help section in this article. Presentation The reason I’m starting with the presentation section of the template is that the implementation section’s Tcl variables reflect the presentation methods naming conventions. I want to accomplish a few things in the template presentation: Ask users for the frequency of backups (daily, weekly, monthly) If weekly, ask for the day of the week If monthly, ask for the day of the month and provide a warning about days 29-31 For all frequencies, ask for the hour and minute the backup should occur The APL code for this looks like this: section time_select { choice day_select display "large" { "Daily", "Weekly", "Monthly" } optional ( day_select == "Weekly" ) { choice dow_select display "medium" { "Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday" } } optional ( day_select == "Monthly" ) { message dom_warning "The day of the month should be the 1st-28th. Selecting the 29th-31st will result in missed backups on some months." choice dom_select display "medium" tcl { for { set x 1 } { $x < 32 } { incr x } { append dom "$x\n" } return $dom } } choice hr_select display "medium" tcl { for { set x 0 } { $x < 24 } { incr x} { append hrs "$x\n" } return $hrs } choice min_select display "medium" tcl { for { set x 0 } { $x < 60 } { incr x } { append mins "$x\n" } return $mins } } text { time_select "Backup Schedule" time_select.day_select "Choose the frequency the backup should occur:" time_select.dow_select "Choose the day of the week the backup should occur:" time_select.dom_warning "WARNING: " time_select.dom_select "Choose the day of the month the backup should occur:" time_select.hr_select "Choose the hour the backup should occur:" time_select.min_select "Choose the minute the backup should occur:" } A few things to point out. First, the sections (which can’t be nested) provide a way to set apart functional differences in your form. I only needed one here, but it’s very useful if I were to build on and add options for selecting a UCS or SCF format, or specifying a mail address for the backups to be mailed to. Second, order matters. The objects will be displayed in the template as you define them. Third, the optional command allows me to hide questions that wouldn’t make sense given previous answers. If you dig into some of the canned templates shipping with v11, you’ll also see another use case for the optional command. Fourth, you can use Tcl commands to populate fields for you. This can be generated data like I did above, or you can loop through configuration objects to present in the template as well. Finally, the text section is where you define the language you want to appear with each of your objects. The nomenclature here is section.variable. To give you an idea what this looks like, here is a screenshot of a monthly backup configuration: Once my template (f5.archiving) is saved, I can configure it in the Application Services section by selecting the template. At this point, I have a functioning presentation, but with no implementation, it’s effectively useless. Implementation Now that the presentation is complete, I can move on to an implementation. I need to do a couple things in the implementation: Grab the data entered into the application service Convert the day of week information from long name to the appropriate 0-6 (or 1-7) number for cron Use that data to build a cron file (statically assigned at this point to /etc/cron.d/f5backups) Here is the implementation section: array set dow_map { Sunday 0 Monday 1 Tuesday 2 Wednesday 3 Thursday 4 Friday 5 Saturday 6 } set hr $::time_select__hr_select set min $::time_select__min_select set infile [open "/etc/cron.d/f5backups" "w" "0755"] puts $infile "SHELL=\/bin\/bash" puts $infile "PATH=\/sbin:\/bin:\/usr\/sbin:\/usr\/bin" puts $infile "#MAILTO=user@somewhere" puts $infile "HOME=\/var\/tmp\/" if { $::time_select__day_select == "Daily" } { puts $infile "$min $hr * * * root \/bin\/bash \/var\/local\/bin\/f5backup.sh 1>\/var\/tmp\/f5backup.log 2>\&1" } elseif { $::time_select__day_select == "Weekly" } { puts $infile "$min $hr * * $dow_map($::time_select__dow_select) root \/bin\/bash \/var\/local\/bin\/f5backup.sh 1>\/var\/tmp\/f5backup.log 2>\&1" } elseif { $::time_select__day_select == "Monthly" } { puts $infile "$min $hr $::time_select__dom_select * * root \/bin\/bash \/var\/local\/bin\/f5backup.sh 1>\/var\/tmp\/f5backup.log 2>\&1" } close $infile A few notes: The dow_map array is to convert the selected day (ie, Saturday) to a number for cron (6). The variables in the implementation section reference the data supplied from the presentation like so: $::<section>__<presentation variable name> (Note the double underscore between them. As such, DO NOT use double underscores in your presentation variables) tmsh special characters need to be escaped if you’re using them for strings. A succesful configuration of the application service results in this file configuration for /etc/cron.d/f5backups: SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin #MAILTO=user@somewhere HOME=/var/tmp/ 54 15 * * * root /bin/bash /var/local/bin/f5backup.sh 1>/var/tmp/f5backup.log 2>&1 So this backup is scheduled to run daily at 15:54. This is confirmed with this directory listing on my BIG-IP: [root@golgotha:Active] bin # ls -las /var/local/backups total 2168 8 drwx------ 2 root root 4096 Aug 16 15:54 . 8 drwxr-xr-x 9 root root 4096 Aug 3 14:44 .. 1076 -rw-r--r-- 1 root root 1091639 Aug 15 15:54 f5backup-golgotha.test.local-20110815155401.tar.bz2 1076 -rw-r--r-- 1 root root 1092259 Aug 16 15:54 f5backup-golgotha.test.local-20110816155401.tar.bz2 Conclusion This is just scratching the surface of what can be done with the new iApp TM feature in v11. I didn’t even cover the ability to use presentation and implementation libraries, but that will be covered in due time. If you’re impatient, there are already several examples (including this one here) in the codeshare. Related Articles F5 DevCentral > Community > Group Details - iApp iApp Wiki Home - DevCentral Wiki F5 Agility 2011 - James Hendergart on iApp iApp Codeshare - DevCentral Wiki iApp Lab 5 - Priority Group Activation - DevCentral Wiki iApp Template Development Tips and Techniques - DevCentral Wiki Lori MacVittie - iApp
F5 in AWS Part 5 - Cloud-init, Single-NIC, and Auto Scale Out in BIG-IP
Updated for Current Versions and Documentation Part 1 : AWS Networking Basics Part 2: Running BIG-IP in an EC2 Virtual Private Cloud Part 3: Advanced Topologies and More on Highly-Available Services Part 4: Orchestrating BIG-IP Application Services with Open-Source Tools Part 5: Cloud-init, Single-NIC, and Auto Scale Out of BIG-IP in v12 The following article covers features and examplesin the 12.1 AWS Marketplace release, discussed in the following documentation: Amazon Web Services: Single NIC BIG-IP VE Amazon Web Services: Auto Scaling BIG-IP VE You can find the BIG-IP Hourly and BYOL releases in the Amazon marketplace here. BIG-IP utility billing images are available, which makes it a great time to talk about some of the functionality. So far in Chris’s series, we have discussed some of the highly-available deployment footprints of BIG-IP in AWS and how these might be orchestrated. Several of these footprints leverage BIG-IP's Device Service Clustering (DSC) technology for configuration management across devices and also lend themselves to multi-app or multi-tenant configurations in a shared-service model. But what if you want to deploy BIG-IP on a per-app or per-tenant basis, in a horizontally scalable footprint that plays well with the concepts of elasticity and immutability in cloud? Today we have just the option for you. Before highlighting these scalable deployment models in AWS, we need to cover cloud-init and single-NIC configurations; two important additions to BIG-IP that enable an Auto Scaling topology. Elasticity Elastiity is obviously one of the biggest promises/benefits of cloud. By leveraging cloud, we are essentially tapping into the "unlimited" (at least relative to our own datacenters) resources large cloud providers have built. In actual practice, this means adopting new methodologies and approaches to truely deliver this. Immutablity In traditional operational model of datacenters, everything was "actively" managed. Physical Infrastructure still tends to lend itself to active management but even virtualized services and applications running on top of the infrastructure were actively managed. For example, servers were patched, code was live upgraded inplace, etc. However, to achieve true elasticity, where things are spinning up or down and more ephemeral in nature, it required a new approach. Instead of trying to patch or upgrade individual instances, the approach was treating them as disposable which meant focusing more on the build process itself. ex. Netflix's famous Building with Legos approach. Yes, the concept of golden images/snapshots existed since virtualization but cloud, with self-service, automation and auto scale functionality, forced this to a new level. Operations focus shifted more towards a consistent and repeatable "build" or "packaging" effort, with final goal of creating instances that didn't need to be touched, logged into, etc. In the specific context of AWS's Auto Scale groups, that means modifying the Auto Scale Group's "launch config". The steps for creating the new instances involve either referencing an entirely new image ID or maybe modification to a cloud-init config. Cloud-init What is it? First, let’s talk about cloud-init as it is used with most Linux distributions. Most of you who are evaluating or operating in the cloud have heard of it. For those who haven’t, cloud-init is an industry standard for bootstrapping machines at startup. It provides a simple domain specific language for common infrastructure provisioning tasks. You can read the official docs here. For the average linux or systems engineer, cloud-init is used to perform tasks such as installing a custom package, updating yum repositories or installing certificates to perform final customizations to a "base" or “golden” image. For example, the Security team might create an approved hardened base image and various Dev teams would use cloud-init to customize the image so it booted up with an ‘identity’ if you will – an application server with an Apache webserver running or a database server with MySQL provisioned. Let’s start with the most basic "Hello World" use of cloud-init, passing in User Data (in this case a simple bash script). If launching an instance via the AWS Console, on the Configure Instance page, navigate down the “Advanced Details”: Figure 1: User Data input field - bash However, User Data is limited to < 16KBs and one of the real powers of cloud-init came from extending functionality past this boundry and providing a standardized way to provision, or ah humm, "initialize" instances.Instead of using unwieldy bash scripts that probed whether it was an Ubuntu or a Redhat instance and used this OS method or that OS method (ex. use apt-get vs. rpm) to install a package, configure users, dns settings, mount a drive, etc. you could pass a yaml file starting with #cloud-config file that did a lot of this heavy lifting for you. Figure 2: User Data input field - cloud-config Similar to one of the benefits of Chef, Puppet, Salt or Ansible, it provided a reliable OS or distribution abstraction but where those approaches require external orchestration to configure instances, this was internally orchestrated from the very first boot which was more condusive to the "immutable" workflow. NOTE: cloud-init also compliments and helps boot strap those tools for more advanced or sophisticated workflows (ex. installing Chef/Puppet to keep long running non-immutable services under operation/management and preventing configuration drift). This brings us to another important distinction. Cloud-init originated as a project from Canonical (Ubuntu) and was designed for general purpose OSs. The BIG-IP's OS (TMOS) however is a highly customized, hardened OS so most of the modules don't strictly apply. Much of the Big-IP's configuration is consumed via its APIs (TMSH, iControl REST, etc.) and stored in it's database MCPD. We can still achieve some of the benefits of having cloud-init but instead, we will mostly leverage the simple bash processor. So when Auto Scaling BIG-IPs, there are a couple of approaches. 1) Creating a custom image as described in the official documentation. 2) Providing a cloud-init configuration This is a little lighter weight approach in that it doesn't require the customization work above. 3) Using a combination of the two, creating a custom image and leveraging cloud-init. For example, you may create a custom image with ASM provisioned, SSL certs/keys installed, and use cloud-init to configure additional environment specific elements. Disclaimer: Packaging is an art, just look at the rise of Docker and new operating systems. Ideally, the more you bake into the image upfront, the more predictable it will be and faster it deploys. However, the less you build-in, the more flexible you can be. Things like installing libraries, compiling, etc. are usually worth building in the image upfront. However, the BIG-IP is already a hardened image and things like installing libraries is not something required or recommended so the task is more about addressing the last/lighter weight configuration steps. However, depending on your priorities and objectives,installing sensitive keying material, setting credentials, pre-provisioning modules, etc. might make good candidates for investing in building custom images. Using Cloud-init with CloudFormation templates Remember when we talked about how you could use CloudFormation templates in a previous post to setup BIG-IP in a standard way? Because the CloudFormation service by itself only gave us the ability to lay down the EC2/VPC infrastructure, we were still left with remaining 80% of the work to do; we needed an external agent or service (in our case Ansible) to configure the BIG-IP and application services. Now, with cloud-init on the BIG-IP (using version 12.0 or later), we can perform that last 80% for you. Using Cloud-init with BIG-IP As you can imagine, there’s quite a lot you can do with just that one simple bash script shown above. However, more interestingly, we also installed AWS’s Cloudformation Helper scripts. http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-helper-scripts-reference.html to help extend cloud-init and unlock a larger more powerful set of AWS functionality. So when used with Cloudformation, our User Data simply changes to executing the following AWS Cloudformation helper script instead. "UserData": { "Fn::Base64": { "Fn::Join": [ "", [ "#!/bin/bash\n", "/opt/aws/apitools/cfn-init-1.4-0.amzn1/bin/cfn-init -v -s ", { "Ref": "AWS::StackId" }, " -r ", "Bigip1Instance", " --region ", { "Ref": "AWS::Region" }, "\n" ] ] } } This allows us to do things like obtaining variables passed in from Cloudformation environment, grabbing various information from the metadata service, creating or downloading files, running particular sequence of commands, etc. so once BIG-IP has finishing running, our entire application delivery service is up and running. For more information, this page discusses how meta-data is attached to an instance using CloudFormation templates: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-init.html#aws-resource-init-commands. Example BYOL and Utility CloudFormation Templates We’ve posted several examples on github to get you started. https://github.com/f5networks/f5-aws-cloudformation In just a few short clicks, you can have an entire BIG-IP deployment up and running. The two examples belowwill launch an entire reference stack complete with VPCs, Subnets, Routing Tables, sample webserver, etc. andshow the use of cloud-init to bootstrap a BIG-IP. Cloud-init is used to configure interfaces, Self-IPs, database variables, a simple virtual server, and in the case of of the BYOL instance, to license BIG-IP. Let’s take a closer look at the BIG-IP resource created in one of these to see what’s going on here: "Bigip1Instance ": { "Metadata ": { "AWS::CloudFormation::Init ": { "config ": { "files ": { "/tmp/firstrun.config ": { "content ": { "Fn::Join ": [ " ", [ "#!/bin/bash\n ", "HOSTNAME=`curl http://169.254.169.254/latest/meta-data/hostname`\n ", "TZ='UTC'\n ", "BIGIP_ADMIN_USERNAME=' ", { "Ref ": "BigipAdminUsername " }, "'\n ", "BIGIP_ADMIN_PASSWORD=' ", { "Ref ": "BigipAdminPassword " }, "'\n ", "MANAGEMENT_GUI_PORT=' ", { "Ref ": "BigipManagementGuiPort " }, "'\n ", "GATEWAY_MAC=`ifconfig eth0 | egrep HWaddr | awk '{print tolower($5)}'`\n ", "GATEWAY_CIDR_BLOCK=`curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/${GATEWAY_MAC}/subnet-ipv4-cidr-block`\n ", "GATEWAY_NET=${GATEWAY_CIDR_BLOCK%/*}\n ", "GATEWAY_PREFIX=${GATEWAY_CIDR_BLOCK#*/}\n ", "GATEWAY=`echo ${GATEWAY_NET} | awk -F. '{ print $1\ ".\ "$2\ ".\ "$3\ ".\ "$4+1 }'`\n ", "VPC_CIDR_BLOCK=`curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/${GATEWAY_MAC}/vpc-ipv4-cidr-block`\n ", "VPC_NET=${VPC_CIDR_BLOCK%/*}\n ", "VPC_PREFIX=${VPC_CIDR_BLOCK#*/}\n ", "NAME_SERVER=`echo ${VPC_NET} | awk -F. '{ print $1\ ".\ "$2\ ".\ "$3\ ".\ "$4+2 }'`\n ", "POOLMEM=' ", { "Fn::GetAtt ": [ "Webserver ", "PrivateIp " ] }, "'\n ", "POOLMEMPORT=80\n ", "APPNAME='demo-app-1'\n ", "VIRTUALSERVERPORT=80\n ", "CRT='default.crt'\n ", "KEY='default.key'\n " ] ] }, "group ": "root ", "mode ": "000755 ", "owner ": "root " }, "/tmp/firstrun.utils ": { "group ": "root ", "mode ": "000755 ", "owner ": "root ", "source ": "http://cdn.f5.com/product/templates/utils/firstrun.utils " } "/tmp/firstrun.sh ": { "content ": { "Fn::Join ": [ " ", [ "#!/bin/bash\n ", ". /tmp/firstrun.config\n ", ". /tmp/firstrun.utils\n ", "FILE=/tmp/firstrun.log\n ", "if [ ! -e $FILE ]\n ", " then\n ", " touch $FILE\n ", " nohup $0 0<&- &>/dev/null &\n ", " exit\n ", "fi\n ", "exec 1<&-\n ", "exec 2<&-\n ", "exec 1<>$FILE\n ", "exec 2>&1\n ", "date\n ", "checkF5Ready\n ", "echo 'starting tmsh config'\n ", "tmsh modify sys ntp timezone ${TZ}\n ", "tmsh modify sys ntp servers add { 0.pool.ntp.org 1.pool.ntp.org }\n ", "tmsh modify sys dns name-servers add { ${NAME_SERVER} }\n ", "tmsh modify sys global-settings gui-setup disabled\n ", "tmsh modify sys global-settings hostname ${HOSTNAME}\n ", "tmsh modify auth user admin password \ "'${BIGIP_ADMIN_PASSWORD}'\ "\n ", "tmsh save /sys config\n ", "tmsh modify sys httpd ssl-port ${MANAGEMENT_GUI_PORT}\n ", "tmsh modify net self-allow defaults add { tcp:${MANAGEMENT_GUI_PORT} }\n ", "if [[ \ "${MANAGEMENT_GUI_PORT}\ " != \ "443\ " ]]; then tmsh modify net self-allow defaults delete { tcp:443 }; fi \n ", "tmsh mv cm device bigip1 ${HOSTNAME}\n ", "tmsh save /sys config\n ", "checkStatusnoret\n ", "sleep 20 \n ", "tmsh save /sys config\n ", "tmsh create ltm pool ${APPNAME}-pool members add { ${POOLMEM}:${POOLMEMPORT} } monitor http\n ", "tmsh create ltm policy uri-routing-policy controls add { forwarding } requires add { http } strategy first-match legacy\n ", "tmsh modify ltm policy uri-routing-policy rules add { service1.example.com { conditions add { 0 { http-uri host values { service1.example.com } } } actions add { 0 { forward select pool ${APPNAME}-pool } } ordinal 1 } }\n ", "tmsh modify ltm policy uri-routing-policy rules add { service2.example.com { conditions add { 0 { http-uri host values { service2.example.com } } } actions add { 0 { forward select pool ${APPNAME}-pool } } ordinal 2 } }\n ", "tmsh modify ltm policy uri-routing-policy rules add { apiv2 { conditions add { 0 { http-uri path starts-with values { /apiv2 } } } actions add { 0 { forward select pool ${APPNAME}-pool } } ordinal 3 } }\n ", "tmsh create ltm virtual /Common/${APPNAME}-${VIRTUALSERVERPORT} { destination 0.0.0.0:${VIRTUALSERVERPORT} mask any ip-protocol tcp pool /Common/${APPNAME}-pool policies replace-all-with { uri-routing-policy { } } profiles replace-all-with { tcp { } http { } } source 0.0.0.0/0 source-address-translation { type automap } translate-address enabled translate-port enabled }\n ", "tmsh save /sys config\n ", "date\n ", "# typically want to remove firstrun.config after first boot\n ", "# rm /tmp/firstrun.config\n " ] ] }, "group ": "root ", "mode ": "000755 ", "owner ": "root " } }, "commands ": { "b-configure-Bigip ": { "command ": "/tmp/firstrun.sh\n " } } } } }, "Properties ": { "ImageId ": { "Fn::FindInMap ": [ "BigipRegionMap ", { "Ref ": "AWS::Region " }, { "Ref ": "BigipPerformanceType " } ] }, "InstanceType ": { "Ref ": "BigipInstanceType " }, "KeyName ": { "Ref ": "KeyName " }, "NetworkInterfaces ": [ { "Description ": "Public or External Interface ", "DeviceIndex ": "0 ", "NetworkInterfaceId ": { "Ref ": "Bigip1ExternalInterface " } } ], "Tags ": [ { "Key ": "Application ", "Value ": { "Ref ": "AWS::StackName " } }, { "Key ": "Name ", "Value ": { "Fn::Join ": [ " ", [ "BIG-IP: ", { "Ref ": "AWS::StackName " } ] ] } } ], "UserData ": { "Fn::Base64 ": { "Fn::Join ": [ " ", [ "#!/bin/bash\n ", "/opt/aws/apitools/cfn-init-1.4-0.amzn1/bin/cfn-init -v -s ", { "Ref ": "AWS::StackId " }, " -r ", "Bigip1Instance ", " --region ", { "Ref ": "AWS::Region " }, "\n " ] ] } } }, "Type ": "AWS::EC2::Instance " } Above may look like a lot at first but high level, we start by creating some files "inline" as well as “sourcing” some files from a remote location. /tmp/firstrun.config - Here we create a file inline, laying down variables from the Cloudformation Stack deployment itself and even the metadata service (http://169.254.169.254/latest/meta-data/). Take a look at the “Ref” stanzas. When this file is laid down on the BIG-IP disk itself, those variables will be interpolated and contain the actual contents. The idea here is to try to keep config and execution separate. /tmp/firstrun.utils – These are just some helper functions to help with initial provisioning. We use those to determine when the BIG-IP is ready for this particular configuration (ex. after a licensing or provisioning step). Note that instead of creating the file inline like the config file above, we simply “source” or download the file from a remote location. /tmp/firstrun.sh – This file is created inline as well and where it really all comes together. The first thing we do is load config variables from the firstrun.conf file and load the helper functions from firstrun.utils. We then create separate log file (/tmp/firstrun.log) to capture the output of this particular script. Capturing the output of these various commands just helps with debugging runs. Then we run a function called checkF5Ready (that we loaded from that helper function file) to make sure BIG-IP’s database is up and ready to accept a configuration. The rest may look more familiar and where most of the user customization takes place. We use variables from the config file to configure the BIG-IP using familiar methods like TMSH and iControl REST. Technically, you could lay down an entire config file (like SCF) and load it instead. We use tmsh here for simplicity. The possibilities are endless though. Disclaimer: the specific implementation above will certainly be optimized and evolve but the most important take away is we can now leverage cloud-init and AWS's helper libraries to help bootstrap the BIG-IP into a working configuration from the very first boot! Debugging Cloud-init What if something goes wrong? Where do you look for more information? The first place you might look is in various cloud-init logs in /var/log (cloud-init.log, cfn-init.log, cfn-wire.log): Below is an example for the CFTs below: [admin@ip-10-0-0-205:NO LICENSE:Standalone] log # tail -150 cfn-init.log 2016-01-11 10:47:59,353 [DEBUG] CloudFormation client initialized with endpoint https://cloudformation.us-east-1.amazonaws.com 2016-01-11 10:47:59,353 [DEBUG] Describing resource BigipEc2Instance in stack arn:aws:cloudformation:us-east-1:452013943082:stack/as-testing-byol-bigip/07c962d0-b893-11e5-9174-500c217b4a62 2016-01-11 10:47:59,782 [DEBUG] Not setting a reboot trigger as scheduling support is not available 2016-01-11 10:47:59,790 [INFO] Running configSets: default 2016-01-11 10:47:59,791 [INFO] Running configSet default 2016-01-11 10:47:59,791 [INFO] Running config config 2016-01-11 10:47:59,792 [DEBUG] No packages specified 2016-01-11 10:47:59,792 [DEBUG] No groups specified 2016-01-11 10:47:59,792 [DEBUG] No users specified 2016-01-11 10:47:59,792 [DEBUG] Writing content to /tmp/firstrun.config 2016-01-11 10:47:59,792 [DEBUG] No mode specified for /tmp/firstrun.config 2016-01-11 10:47:59,793 [DEBUG] Writing content to /tmp/firstrun.sh 2016-01-11 10:47:59,793 [DEBUG] Setting mode for /tmp/firstrun.sh to 000755 2016-01-11 10:47:59,793 [DEBUG] Setting owner 0 and group 0 for /tmp/firstrun.sh 2016-01-11 10:47:59,793 [DEBUG] Running command b-configure-BigIP 2016-01-11 10:47:59,793 [DEBUG] No test for command b-configure-BigIP 2016-01-11 10:47:59,840 [INFO] Command b-configure-BigIP succeeded 2016-01-11 10:47:59,841 [DEBUG] Command b-configure-BigIP output: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 40 0 40 0 0 74211 0 --:--:-- --:--:-- --:--:-- 40000 2016-01-11 10:47:59,841 [DEBUG] No services specified 2016-01-11 10:47:59,844 [INFO] ConfigSets completed 2016-01-11 10:47:59,851 [DEBUG] Not clearing reboot trigger as scheduling support is not available [admin@ip-10-0-0-205:NO LICENSE:Standalone] log # If trying out the example templates above, you can inspect the various files mentioned. Ex. In addition to checking for their general presence: /tmp/firstrun.config= make sure variables were passed as you expected. /tmp/firstrun.utils= Make sure exists and was downloaded /tmp/firstrun.log = See if any obvious errors were outputted. It may also be worth checking AWS Cloudformation Console to make sure you passed the parameters you were expecting. Single-NIC Another one of the important building blocks introduced with 12.0 on AWS and Azure Virtual Editions is the ability to run BIG-IP with just a single network interface. Typically, BIG-IPs were deployed in a multi-interface model, where interfaces were attached to an out-of-band management network and one or more traffic (or "data-plane") networks. But, as we know, cloud architectures scale by requiring simplicity, especially at the network level. To this day, some clouds can only support instances with a single IP on a single NIC. In AWS’s case, although they do support multiple NIC/multiple IP, some of their services like ELB only point to first IP address of the first NIC. So this Single-NIC configuration makes it not only possible but also dramatically easier to deploy in these various architectures. How this works: We can now attach just one interface to the instance and BIG-IP will start up, recognize this, use DHCP to configure the necessary settings on that interface. Underneath the hood, the following DB keys will be set: admin@(ip-10-0-1-65)(cfg-sync Standalone)(Active)(/Common)(tmos)# list sys db provision.1nic one-line sys db provision.1nic { value "enable" } admin@(ip-10-0-1-65)(cfg-sync Standalone)(Active)(/Common)(tmos)# list sys db provision.1nicautoconfig one-line sys db provision.1nicautoconfig { value "enable" } provision.1nic = allows both management and data-plane to use the same interface provision.1nicautoconfig = uses address from DHCP to configure a vlan, Self-IP and default gateway. Ex. network objects automatically configured admin@(ip-10-0-1-65)(cfg-sync Standalone)(Active)(/Common)(tmos)# list net vlan net vlan internal if-index 112 interfaces { 1.0 { } } tag 4094 } admin@(ip-10-0-1-65)(cfg-sync Standalone)(Active)(/Common)(tmos)# list net self net self self_1nic { address 10.0.1.65/24 allow-service { default } traffic-group traffic-group-local-only vlan internal } admin@(ip-10-0-1-65)(cfg-sync Standalone)(Active)(/Common)(tmos)# list net route net route default { gw 10.0.1.1 network default } Note: Traffic Management Shell and the Configuration Utility (GUI) are still available on ports 22 and 443 respectively. If you want to run the management GUI on a higher port (for instance if you don’t have the BIG-IPs behind a Port Address Translation service (like ELB) and want to run an HTTPS virtual on 443), use the following commands: tmsh modify sys httpd ssl-port 8443 tmsh modify net self-allow defaults add { tcp:8443 } tmsh modify net self-allow defaults delete { tcp:443 } WARNING: Now that management and dataplane run on the same interface, make sure to modify your Security Groups to restrict access to SSH and whatever port you use for the Mgmt GUI port to trusted networks. UPDATE: On Single-Nic, Device Service Clustering currently only supports Configuration Syncing (Network Failover is restricted for now due to BZ-606032). In general, the single-NIC model lends itself better to single-tenant or per-app deployments, where you need the advanced services from BIG-IP like content routing policies, iRules scripting, WAF, etc. but don’t necessarily care for maintaining a management subnet in the deployment and are just optimizing or securing a single application. By single tenant we also mean single management domain as you're typically running everything through a single wildcard virtual (ex. 0.0.0.0/0, 0.0.0.0/80, 0.0.0.0/443, etc.) vs. giving each tenant its own Virtual Server (usually with its own IP and configuration) to manage. However, you can still technically run multiple applications behind this virtual with a policy or iRule, where you make traffic steering decisions based on L4-L7 content (SNI, hostname headers, URIs, etc.). In addition, if the BIG-IPs are sitting behind a Port Address Translation service, it also possible to stack virtual services on ports instead. Ex. 0.0.0.0:444 = Virtual Service 1 0.0.0.0:445 = Virtual Service 2 0.0.0.0:446 = Virtual Service 3 We’ll let you get creative here…. BIG-IP Auto Scale Finally, the last component of Auto Scaling BIG-IPs involves building scaling policies via Cloudwatch Alarms. In addition to the built-in EC2 metrics in CloudWatch, BIG-IP can report its own set of metrics, shown below: Figure 3: Cloudwatch metrics to scale BIG-IPs based on traffic load. This can be configured with the following TMSH commands on any version 12.0 or later build: tmsh modify sys autoscale-group autoscale-group-id ${BIGIP_ASG_NAME} tmsh load sys config merge file /usr/share/aws/metrics/aws-cloudwatch-icall-metrics-config These commands tell BIG-IP to push the above metrics to a custom “Namespace” on which we can roll up data via standard aggregation functions (min, max, average, sum). This namespace (based on Auto Scale group name) will appear as a row under in the “Custom metrics” dropdown on the left side-bar in CloudWatch (left side of Figure 1). Once these BIG-IP or EC2 metrics have been populated, CloudWatch alarms can be built, and these alarms are available for use in Auto Scaling policies for the BIG-IP Auto Scale group. (Amazon provides some nice instructions here). Auto Scaling Pool Members and Service Discovery If you are scaling your ADC tier, you are likely also going to be scaling your application as well. There are two options for discovering pool members in your application's auto scale group. 1) FQDN Nodes For example, in a typicalsandwichdeployment, your application members might also be sitting behind an internal ELB so you would simply point your FQDN node at the ELB's DNS. For more information, please see: https://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/ltm-implementations-12-1-0/25.html?sr=56133323 2) BIG-IP's AWS Auto Scale Pool Member Discovery feature (introduced v12.0) This feature polls the Auto Scale Group via API calls and populates the pool based on its membership. For more information, please see: https://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/bigip-ve-autoscaling-amazon-ec2-12-1-0/2.html Putting it all together The high-level steps for Auto Scaling BIG-IP include the following: Optionally* creating an ElasticLoadBalancer group which will direct traffic to BIG-IPs in your Auto Scale group once they become operational. Otherwise, will need Global Server Load Balancing (GSLB). Creating a launch configuration in EC2 (referencing either custom image id and/or using Cloud-init scripts as described above) Creating an Auto Scale group using this launch configuration Creating CloudWatch alarms using the EC2 or custom metrics reported by BIG-IP. Creating scaling policies for your BIG-IP Auto Scale group using the alarms above. You will want to create both scale up and scale down policies. Here are some things to keep in mind when Auto Scaling BIG-IP ● BIG-IP must run in a single-interface configuration with a wildcard listener (as we talked about earlier). This is required because we don't know what IP address the BIG-IP will get. ● Auto Scale Groups themselves consist of utility instances of BIG-IP ● The scale up time for BIG-IP is about 12-20 minutes depending on what is configured or provisioned. While this may seem like a long-time, creating the right scaling policies (polling intervals, thresholds, unit of scale) make this a non-issue. ● This deployment model lends itself toward the themes of stateless, horizontal scalability and immutability embraced by the cloud. Currently, the config on each device is updated once and only once at device startup. The only way to change the config will be through the creation of a new image or modification to the launch configuration. Stay tuned for a clustered deployment which is managed in a more traditional operational approach. If interesting in exploring Auto Scale further, we have put together some examples of scaling the BIG-IP tier here: https://github.com/f5devcentral/f5-aws-autoscale * Above github repository also provides some examples of incorporating BYOL instances in the deployment to help leverage BYOL instances for your static load and using Auto Scale instances for your dynamic load. See the READMEs for more information. CloudFormation templates with Auto Scaled BIG-IP and Application Need some ideas on how you might leverage these solutions? Now that you can completely deploy a solution with a single template, how about building service catalog items for your business units that deploy different BIG-IP services (LB, WAF) or a managed service build on top of AWS that can be easily deployed each time you on-board a new customer?Full examples of iControlREST for device and application service deployment
In a previous article, I highlighted a proof-of-concept where we fully automated the deployment of BIG-IP in AWS using the web interfaces of BIG-IP inconjunction with Ansible. The goal of this article is to focus in more detail on the use of iControlREST within that project, in order to show how it can be extremely useful for automating various aspects of your ITOM workflows. There are four main workflows we execute in order to configure BIG-IP in AWS and to deploy services. This breakown is provided below. Basic System Configuration AWS-specific system configuration Network attachment Application service provisioning For now, we avoid the discussion about which configuration elements are part of the infrastructure deployment or the application deployment. This is an important discussion, but one that can only take place after we understand how to provision the elements that will be a part of either workflow. For each of these workflows, we have provided the log output which shows REST calls and responses. To gain from these examples, it is important to understand the following: These iControlREST calls were scraped from an execution of the aws-deployments code where we captured log output. In that project, we have written a custom Ansible module called 'bigip_config' (see /library/bigip_config.py in the project directory of Github). This module is used to provision config objects within TMOS using iControlREST. In order to make it easier to use this 'bigip_config' Ansible module, we aimed to make itidempotent. This means that we need only identify the resources we wish to create or update, and identify the state of the resource after our module is run. We don't need to worry about whether the object exists when calling our module, or the procedural set of calls that should be made it order to get it there. An example might be like: "create iApp service 'my_app' with parameters X, Y, and Z". We don't care whether 'my_app' already exists.In order to implement such behavior, the module internally does an HTTP GET against the resource or collection it is modifying. Subsequently, if the resource already exists, a PATCH call is made, otherwise a POST call is made. In many cases in the code, we repeat a call until it returns successfully or returns the state we are expecting. You can see this in the examples, where the same call seems to be made repeatedly. Basic system configuration The examples below show how we are configuring basic device settings with REST. First, because BIG-IP was just started, we wait until the first iCR call,"GET mgmt/tm/sys/db",succeeds before we continue. The final step the workflow involves provisioning modules on BIG-IP. Note the 30 second wait between provisioning of AVR and ASM reflected in the timestamps. 2015-11-12 09:46:03 : Disabling Setup Utility in GUI GET mgmt/tm/sys/db "" 2015-11-12 09:46:10 : Disabling Setup Utility in GUI GET mgmt/tm/sys/db "" 2015-11-12 09:46:17 : Disabling Setup Utility in GUI GET mgmt/tm/sys/db "" Method GET mgmt/tm/sys/db returned: {"kind":"tm:sys:db:dbcollectionstate","selfLink":"https://localhost/mgmt/tm/sys/db?ver=11.6.0","items":[....<a whole bunch of database variables>...]} 2015-11-12 09:46:19 : Disabling Setup Utility in GUI PATCH mgmt/tm/sys/db/setup.run?ver=11.6.0 {"value": "false"} Method PATCH mgmt/tm/sys/db/setup.run?ver=11.6.0 returned: {"kind":"tm:sys:db:dbstate","name":"setup.run","fullPath":"setup.run","generation":62,"selfLink":"https://localhost/mgmt/tm/sys/db/setup.run?ver=11.6.0","defaultValue":"true","scfConfig":"false","value":"false","valueRange":"false true"} 2015-11-12 09:46:22 : Configuring NTP servers PATCH mgmt/tm/sys/ntp {"timezone": "America/Los_Angeles", "servers": ["0.pool.ntp.org", "1.pool.ntp.org"]} Method PATCH mgmt/tm/sys/ntp returned: {"kind":"tm:sys:ntp:ntpstate","selfLink":"https://localhost/mgmt/tm/sys/ntp?ver=11.6.0","servers":["0.pool.ntp.org","1.pool.ntp.org"],"timezone":"America/Los_Angeles","restrictReference":{"link":"https://localhost/mgmt/tm/sys/ntp/restrict?ver=11.6.0","isSubcollection":true}} 2015-11-12 09:46:25 : Configuring syslog logging destinations PATCH mgmt/tm/sys/syslog {"include": "destination loghost { udp( 10.0.3.32 port (514));};"} Method PATCH mgmt/tm/sys/syslog returned: {"kind":"tm:sys:syslog:syslogstate","selfLink":"https://localhost/mgmt/tm/sys/syslog?ver=11.6.0","authPrivFrom":"notice","authPrivTo":"emerg","consoleLog":"enabled","cronFrom":"warning","cronTo":"emerg","daemonFrom":"notice","daemonTo":"emerg","include":"destination loghost { udp( 10.0.3.32 port (514));};","isoDate":"disabled","kernFrom":"debug","kernTo":"emerg","local6From":"notice","local6To":"emerg","mailFrom":"notice","mailTo":"emerg","messagesFrom":"notice","messagesTo":"warning","userLogFrom":"notice","userLogTo":"emerg"} 2015-11-12 09:46:28 : Configuring HTTP mgmt access PATCH mgmt/tm/sys/httpd {"allow": ["ALL"]} Method PATCH mgmt/tm/sys/httpd returned: {"kind":"tm:sys:httpd:httpdstate","selfLink":"https://localhost/mgmt/tm/sys/httpd?ver=11.6.0","allow":["ALL"],"authName":"BIG-IP","authPamDashboardTimeout":"off","authPamIdleTimeout":1200,"authPamValidateIp":"on","fastcgiTimeout":300,"hostnameLookup":"off","logLevel":"warn","maxClients":10,"redirectHttpToHttps":"disabled","requestBodyMaxTimeout":0,"requestBodyMinRate":500,"requestBodyTimeout":60,"requestHeaderMaxTimeout":40,"requestHeaderMinRate":500,"requestHeaderTimeout":20,"sslCertfile":"/etc/httpd/conf/ssl.crt/server.crt","sslCertkeyfile":"/etc/httpd/conf/ssl.key/server.key","sslCiphersuite":"DEFAULT:!aNULL:!eNULL:!LOW:!RC4:!MD5:!EXP","sslOcspDefaultResponder":"http://127.0.0.1","sslOcspEnable":"off","sslOcspOverrideResponder":"off","sslOcspResponderTimeout":300,"sslOcspResponseMaxAge":-1,"sslOcspResponseTimeSkew":300,"sslProtocol":"all -SSLv2 -SSLv3","sslVerifyClient":"no","sslVerifyDepth":10} 2015-11-12 09:46:31 : Configuring SSH mgmt access PATCH mgmt/tm/sys/sshd {"allow": ["ALL"]} Method PATCH mgmt/tm/sys/sshd returned: {"kind":"tm:sys:sshd:sshdstate","selfLink":"https://localhost/mgmt/tm/sys/sshd?ver=11.6.0","allow":["ALL"],"banner":"disabled","inactivityTimeout":0,"logLevel":"info","login":"enabled"} 2015-11-12 09:46:33 : Configuring SNMP access PATCH mgmt/tm/sys/snmp {"allowedAddresses": ["172.16.0.0/16"]} Method PATCH mgmt/tm/sys/snmp returned: {"kind":"tm:sys:snmp:snmpstate","selfLink":"https://localhost/mgmt/tm/sys/snmp?ver=11.6.0","agentAddresses":["tcp6:161","udp6:161"],"agentTrap":"enabled","allowedAddresses":["172.16.0.0/16"],"authTrap":"disabled","bigipTraps":"enabled","l2forwardVlan":"none","loadMax1":12,"loadMax15":12,"loadMax5":12,"sysContact":"Customer Name <admin@customer.com>","sysLocation":"Network Closet 1","sysServices":78,"trapCommunity":"public","trapSource":"none","communitiesReference":{"link":"https://localhost/mgmt/tm/sys/snmp/communities?ver=11.6.0","isSubcollection":true},"diskMonitors":[{"name":"root","partition":"Common","minspace":2000,"minspaceType":"size","path":"/"},{"name":"var","partition":"Common","minspace":10000,"minspaceType":"size","path":"/var"}],"processMonitors":[{"name":"bigd","partition":"Common","maxProcesses":"1","minProcesses":1,"process":"bigd"},{"name":"chmand","partition":"Common","maxProcesses":"1","minProcesses":1,"process":"chmand"},{"name":"httpd","partition":"Common","maxProcesses":"infinity","minProcesses":1,"process":"httpd"},{"name":"mcpd","partition":"Common","maxProcesses":"1","minProcesses":1,"process":"mcpd"},{"name":"sod","partition":"Common","maxProcesses":"1","minProcesses":1,"process":"sod"},{"name":"tmm","partition":"Common","maxProcesses":"infinity","minProcesses":1,"process":"tmm"}],"trapsReference":{"link":"https://localhost/mgmt/tm/sys/snmp/traps?ver=11.6.0","isSubcollection":true},"usersReference":{"link":"https://localhost/mgmt/tm/sys/snmp/users?ver=11.6.0","isSubcollection":true}} 2015-11-12 09:46:36 : Configuring FastL4 profiles ... fastL4-route-friendly GET mgmt/tm/ltm/profile/fastl4 "" Method GET mgmt/tm/ltm/profile/fastl4 returned: {"kind":"tm:ltm:profile:fastl4:fastl4collectionstate","selfLink":"https://localhost/mgmt/tm/ltm/profile/fastl4?ver=11.6.0","items":[{"kind":"tm:ltm:profile:fastl4:fastl4state","name":"fastL4","partition":"Common","fullPath":"/Common/fastL4","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/profile/fastl4/~Common~fastL4?ver=11.6.0","clientTimeout":30,"explicitFlowMigration":"disabled","hardwareSynCookie":"enabled","idleTimeout":"300","ipTosToClient":"pass-through","ipTosToServer":"pass-through","keepAliveInterval":"disabled","lateBinding":"disabled","linkQosToClient":"pass-through","linkQosToServer":"pass-through","looseClose":"disabled","looseInitialization":"disabled","mssOverride":0,"priorityToClient":"pass-through","priorityToServer":"pass-through","pvaAcceleration":"full","pvaDynamicClientPackets":1,"pvaDynamicServerPackets":0,"pvaFlowAging":"enabled","pvaFlowEvict":"enabled","pvaOffloadDynamic":"enabled","pvaOffloadState":"embryonic","reassembleFragments":"disabled","receiveWindowSize":0,"resetOnTimeout":"enabled","rttFromClient":"disabled","rttFromServer":"disabled","serverSack":"disabled","serverTimestamp":"disabled","softwareSynCookie":"disabled","synCookieWhitelist":"disabled","tcpCloseTimeout":"5","tcpGenerateIsn":"disabled","tcpHandshakeTimeout":"5","tcpStripSack":"disabled","tcpTimestampMode":"preserve","tcpWscaleMode":"preserve","timeoutRecovery":"disconnect"}]} 2015-11-12 09:46:37 : Configuring FastL4 profiles ... fastL4-route-friendly POST mgmt/tm/ltm/profile/fastl4 {"looseClose": "enabled", "resetOnTimeout": "disabled", "name": "fastL4-route-friendly", "looseInitialization": "enabled"} Method POST mgmt/tm/ltm/profile/fastl4 returned: {"kind":"tm:ltm:profile:fastl4:fastl4state","name":"fastL4-route-friendly","fullPath":"fastL4-route-friendly","generation":68,"selfLink":"https://localhost/mgmt/tm/ltm/profile/fastl4/fastL4-route-friendly?ver=11.6.0","clientTimeout":30,"defaultsFrom":"/Common/fastL4","explicitFlowMigration":"disabled","hardwareSynCookie":"enabled","idleTimeout":"300","ipTosToClient":"pass-through","ipTosToServer":"pass-through","keepAliveInterval":"disabled","lateBinding":"disabled","linkQosToClient":"pass-through","linkQosToServer":"pass-through","looseClose":"enabled","looseInitialization":"enabled","mssOverride":0,"priorityToClient":"pass-through","priorityToServer":"pass-through","pvaAcceleration":"full","pvaDynamicClientPackets":1,"pvaDynamicServerPackets":0,"pvaFlowAging":"enabled","pvaFlowEvict":"enabled","pvaOffloadDynamic":"enabled","pvaOffloadState":"embryonic","reassembleFragments":"disabled","receiveWindowSize":0,"resetOnTimeout":"disabled","rttFromClient":"disabled","rttFromServer":"disabled","serverSack":"disabled","serverTimestamp":"disabled","softwareSynCookie":"disabled","synCookieWhitelist":"disabled","tcpCloseTimeout":"5","tcpGenerateIsn":"disabled","tcpHandshakeTimeout":"5","tcpStripSack":"disabled","tcpTimestampMode":"preserve","tcpWscaleMode":"preserve","timeoutRecovery":"disconnect"} ... 2015-11-12 09:46:44 : PATCH mgmt/tm/sys/provision/asm {"level": "nominal"} Method PATCH mgmt/tm/sys/provision/asm returned: {"kind":"tm:sys:provision:provisionstate","name":"asm","fullPath":"asm","generation":71,"selfLink":"https://localhost/mgmt/tm/sys/provision/asm?ver=11.6.0","cpuRatio":0,"diskRatio":0,"level":"nominal","memoryRatio":0} 2015-11-12 09:47:18 : PATCH mgmt/tm/sys/provision/avr {"level": "nominal"} Method PATCH mgmt/tm/sys/provision/avr returned: {"kind":"tm:sys:provision:provisionstate","name":"avr","fullPath":"avr","generation":114,"selfLink":"https://localhost/mgmt/tm/sys/provision/avr?ver=11.6.0","cpuRatio":0,"diskRatio":0,"level":"nominal","memoryRatio":0} AWS-specific System Configuration This next workflow is very small and simple. We are adding some variables to global-settings which are only necessary because BIG-IP is running in AWS. We've obfuscated the AWS Access Key and Secret Key in the output. 2015-11-12 09:48:12 : Adding/updating AWS access and secret keys PATCH mgmt/tm/sys/global-settings {"awsAccessKey": "...<my access key>...", "awsSecretKey": "...<my secret key>..."} Method PATCH mgmt/tm/sys/global-settings returned: {"kind":"tm:sys:global-settings:global-settingsstate","selfLink":"https://localhost/mgmt/tm/sys/global-settings?ver=11.6.0","awsAccessKey":"...<my access key>...","awsApiMaxConcurrency":1,"awsSecretKey":"...<my secret key>...","consoleInactivityTimeout":0,"customAddr":"none","failsafeAction":"go-offline-restart-tm","fileLocalPathPrefix":"{/shared/} {/tmp/}","guiSecurityBanner":"enabled","guiSecurityBannerText":"Welcome to the BIG-IP Configuration Utility.\n\nLog in with your username and password using the fields on the left.","guiSetup":"disabled","hostAddrMode":"management","hostname":"ip-172-16-11-77.ec2.internal","lcdDisplay":"enabled","mgmtDhcp":"enabled","netReboot":"disabled","passwordPrompt":"Password","quietBoot":"enabled","usernamePrompt":"Username"} Network Attachment Setup of self-IPs, VLANs, and other network specific configuration is relatively straight forward. 2015-11-12 09:48:15 : Disabling dhcp PATCH mgmt/tm/sys/db/dhclient.mgmt {"value": "disable"} Method PATCH mgmt/tm/sys/db/dhclient.mgmt returned: {"kind":"tm:sys:db:dbstate","name":"dhclient.mgmt","fullPath":"dhclient.mgmt","generation":154,"selfLink":"https://localhost/mgmt/tm/sys/db/dhclient.mgmt?ver=11.6.0","defaultValue":"disable","scfConfig":"true","value":"disable","valueRange":"disable enable"} 2015-11-12 09:48:18 : Adding/updating internal vlan GET mgmt/tm/net/vlan "" Method GET mgmt/tm/net/vlan returned: {"kind":"tm:net:vlan:vlancollectionstate","selfLink":"https://localhost/mgmt/tm/net/vlan?ver=11.6.0"} 2015-11-12 09:48:19 : Adding/updating internal vlan POST mgmt/tm/net/vlan {"interfaces": "1.2", "name": "private"} Method POST mgmt/tm/net/vlan returned: {"kind":"tm:net:vlan:vlanstate","name":"private","fullPath":"private","generation":167,"selfLink":"https://localhost/mgmt/tm/net/vlan/private?ver=11.6.0","autoLasthop":"default","cmpHash":"default","dagRoundRobin":"disabled","dagTunnel":"outer","failsafe":"disabled","failsafeAction":"failover-restart-tm","failsafeTimeout":90,"ifIndex":80,"learning":"enable-forward","mtu":1500,"sflow":{"pollInterval":0,"pollIntervalGlobal":"yes","samplingRate":0,"samplingRateGlobal":"yes"},"sourceChecking":"disabled","tag":4094,"interfacesReference":{"link":"https://localhost/mgmt/tm/net/vlan/~Common~private/interfaces?ver=11.6.0","isSubcollection":true}} 2015-11-12 09:48:21 : Adding/updating external vlan GET mgmt/tm/net/vlan "" Method GET mgmt/tm/net/vlan returned: {"kind":"tm:net:vlan:vlancollectionstate","selfLink":"https://localhost/mgmt/tm/net/vlan?ver=11.6.0","items":[{"kind":"tm:net:vlan:vlanstate","name":"private","partition":"Common","fullPath":"/Common/private","generation":167,"selfLink":"https://localhost/mgmt/tm/net/vlan/~Common~private?ver=11.6.0","autoLasthop":"default","cmpHash":"default","dagRoundRobin":"disabled","dagTunnel":"outer","failsafe":"disabled","failsafeAction":"failover-restart-tm","failsafeTimeout":90,"ifIndex":80,"learning":"enable-forward","mtu":1500,"sflow":{"pollInterval":0,"pollIntervalGlobal":"yes","samplingRate":0,"samplingRateGlobal":"yes"},"sourceChecking":"disabled","tag":4094,"interfacesReference":{"link":"https://localhost/mgmt/tm/net/vlan/~Common~private/interfaces?ver=11.6.0","isSubcollection":true}}]} ... 2015-11-12 09:48:24 : Adding/updating internal selfip GET mgmt/tm/net/self "" Method GET mgmt/tm/net/self returned: {"kind":"tm:net:self:selfcollectionstate","selfLink":"https://localhost/mgmt/tm/net/self?ver=11.6.0"} 2015-11-12 09:48:24 : Adding/updating internal selfip POST mgmt/tm/net/self {"allowService": "default", "vlan": "private", "trafficGroup": "traffic-group-local-only", "name": "private", "address": "172.16.12.44/24"} Method POST mgmt/tm/net/self returned: {"kind":"tm:net:self:selfstate","name":"private","fullPath":"private","generation":177,"selfLink":"https://localhost/mgmt/tm/net/self/private?ver=11.6.0","address":"172.16.12.44/24","floating":"disabled","inheritedTrafficGroup":"false","trafficGroup":"/Common/traffic-group-local-only","unit":0,"vlan":"/Common/private","allowService":["default"]} 2015-11-12 09:48:26 : Adding/updating external selfip GET mgmt/tm/net/self "" Method GET mgmt/tm/net/self returned: {"kind":"tm:net:self:selfcollectionstate","selfLink":"https://localhost/mgmt/tm/net/self?ver=11.6.0","items":[{"kind":"tm:net:self:selfstate","name":"private","partition":"Common","fullPath":"/Common/private","generation":177,"selfLink":"https://localhost/mgmt/tm/net/self/~Common~private?ver=11.6.0","address":"172.16.12.44/24","floating":"disabled","inheritedTrafficGroup":"false","trafficGroup":"/Common/traffic-group-local-only","unit":0,"vlan":"/Common/private","allowService":["default"]}]} 2015-11-12 09:48:27 : Adding/updating external selfip POST mgmt/tm/net/self {"allowService": ["tcp:4353"], "vlan": "public", "trafficGroup": "traffic-group-local-only", "name": "public", "address": "172.16.13.83/24"} Method POST mgmt/tm/net/self returned: {"kind":"tm:net:self:selfstate","name":"public","fullPath":"public","generation":178,"selfLink":"https://localhost/mgmt/tm/net/self/public?ver=11.6.0","address":"172.16.13.83/24","floating":"disabled","inheritedTrafficGroup":"false","trafficGroup":"/Common/traffic-group-local-only","unit":0,"vlan":"/Common/public","allowService":["tcp:4353"]} 2015-11-12 09:48:29 : Setting default route using default_gateway or gateway_pool GET /mgmt/tm/net/route "" Method GET /mgmt/tm/net/route returned: {"kind":"tm:net:route:routecollectionstate","selfLink":"https://localhost/mgmt/tm/net/route?ver=11.6.0"} 2015-11-12 09:48:29 : Setting default route using default_gateway or gateway_pool POST /mgmt/tm/net/route {"gw": "172.16.13.1", "name": "default_route", "network": "default"} Method POST /mgmt/tm/net/route returned: {"kind":"tm:net:route:routestate","name":"default_route","fullPath":"default_route","generation":0,"selfLink":"https://localhost/mgmt/tm/net/route/default_route?ver=11.6.0"} Application Service Provisioning This workflow is where things really get interesting. Let's break it down. We are deploying two sets of virtual servers (the pool members are the same, but the VIP is different). For virtual 1 (VIP =172.16.13.128), we use an iApp to deploy a HTTPS virtual with an ASM policy. To do so, we: Deploy all resources that are needed to support the iApp deployment: A high-speed logging pool An LTM logging profile (which will send logs to Splunk on port 514) An ASM logging profile(which will send logs to Splunk on port 515) An analytics profile, in case we want to inspect traffic with AVR on-box Base64 encoded images to an iRule data-group iRules to support a sorry page and the analytics profile An ASM policy (we've encoded the XML policy file into base64). Deploying the ASM policies requires first making a new policy via a POST command, then importing the policy over the defaults for the one we have just created. Note that we check the status of the asynchronous REST tasks which are started during the policy 'create' and 'apply' steps. An LTM policy which attaches the ASM policy above using a ruleset. Deploy the iApp template (look here to understand how we built the JSON payload for the iApp template). Finally, deploy the iApp service, an instantiation of the template that references all the above content (look here to understand how we built theJSON payload for the iApp service). For virtual 2 (VIP =172.16.13.124), just deploy the web server pool, iRule, and virtual server directly (without an iApp). 2015-11-12 09:49:13 : Deploying/updating Webserver Pool GET mgmt/tm/ltm/pool "" Method GET mgmt/tm/ltm/pool returned: {"kind":"tm:ltm:pool:poolcollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/pool?ver=11.6.0"} 2015-11-12 09:49:14 : Deploying/updating Webserver Pool POST mgmt/tm/ltm/pool {"name": "Vip1_pool", "members": [{"description": "Name=/boring_lovelace,ContainerHostname=a0085832ad28,Image=mutzel/all-in-one-hackazon:postinstall", "name": "172.16.14.87:80", "address": "172.16.14.87"}], "monitor": "http"} Method POST mgmt/tm/ltm/pool returned: {"kind":"tm:ltm:pool:poolstate","name":"Vip1_pool","fullPath":"Vip1_pool","generation":236,"selfLink":"https://localhost/mgmt/tm/ltm/pool/Vip1_pool?ver=11.6.0","allowNat":"yes","allowSnat":"yes","ignorePersistedWeight":"disabled","ipTosToClient":"pass-through","ipTosToServer":"pass-through","linkQosToClient":"pass-through","linkQosToServer":"pass-through","loadBalancingMode":"round-robin","minActiveMembers":0,"minUpMembers":0,"minUpMembersAction":"failover","minUpMembersChecking":"disabled","monitor":"/Common/http ","queueDepthLimit":0,"queueOnConnectionLimit":"disabled","queueTimeLimit":0,"reselectTries":0,"serviceDownAction":"none","slowRampTime":10,"membersReference":{"link":"https://localhost/mgmt/tm/ltm/pool/~Common~Vip1_pool/members?ver=11.6.0","isSubcollection":true}} 2015-11-12 09:49:18 : Deploying/updating High Speed Logging pool to send to Analytics Server GET mgmt/tm/ltm/pool "" Method GET mgmt/tm/ltm/pool returned: {"kind":"tm:ltm:pool:poolcollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/pool?ver=11.6.0","items":[{"kind":"tm:ltm:pool:poolstate","name":"Vip1_pool","partition":"Common","fullPath":"/Common/Vip1_pool","generation":236,"selfLink":"https://localhost/mgmt/tm/ltm/pool/~Common~Vip1_pool?ver=11.6.0","allowNat":"yes","allowSnat":"yes","ignorePersistedWeight":"disabled","ipTosToClient":"pass-through","ipTosToServer":"pass-through","linkQosToClient":"pass-through","linkQosToServer":"pass-through","loadBalancingMode":"round-robin","minActiveMembers":0,"minUpMembers":0,"minUpMembersAction":"failover","minUpMembersChecking":"disabled","monitor":"/Common/http ","queueDepthLimit":0,"queueOnConnectionLimit":"disabled","queueTimeLimit":0,"reselectTries":0,"serviceDownAction":"none","slowRampTime":10,"membersReference":{"link":"https://localhost/mgmt/tm/ltm/pool/~Common~Vip1_pool/members?ver=11.6.0","isSubcollection":true}}]} 2015-11-12 09:49:19 : Deploying/updating High Speed Logging pool to send to Analytics Server POST mgmt/tm/ltm/pool {"name": "syslog_pool", "members": [{"name": "172.16.14.180:514", "address": "172.16.14.180"}], "monitor": "tcp"} Method POST mgmt/tm/ltm/pool returned: {"kind":"tm:ltm:pool:poolstate","name":"syslog_pool","fullPath":"syslog_pool","generation":239,"selfLink":"https://localhost/mgmt/tm/ltm/pool/syslog_pool?ver=11.6.0","allowNat":"yes","allowSnat":"yes","ignorePersistedWeight":"disabled","ipTosToClient":"pass-through","ipTosToServer":"pass-through","linkQosToClient":"pass-through","linkQosToServer":"pass-through","loadBalancingMode":"round-robin","minActiveMembers":0,"minUpMembers":0,"minUpMembersAction":"failover","minUpMembersChecking":"disabled","monitor":"/Common/tcp ","queueDepthLimit":0,"queueOnConnectionLimit":"disabled","queueTimeLimit":0,"reselectTries":0,"serviceDownAction":"none","slowRampTime":10,"membersReference":{"link":"https://localhost/mgmt/tm/ltm/pool/~Common~syslog_pool/members?ver=11.6.0","isSubcollection":true}} ... 2015-11-12 09:49:21 : Deploying/updating ASM Logging Profile to send to Remote Analytics Server POST mgmt/tm/security/log/profile {"application": [{"guaranteeLogging": "enabled", "guaranteeResponseLogging": "disabled", "logicOperation": "or", "protocol": "tcp", "name": "asm_log_to_splunk", "format": {"fieldDelimiter": ",", "type": "predefined"}, "reportAnomalies": "disabled", "facility": "local0", "partition": "Common", "filter": [{"values": ["all"], "name": "protocol"}, {"values": ["all"], "name": "request-type"}, {"name": "search-all"}], "maximumHeaderSize": "any", "localStorage": "enabled", "maximumQuerySize": "any", "maximumEntryLength": "2k", "servers": [{"name": "172.16.14.180:515"}], "remoteStorage": "splunk", "maximumRequestSize": "any", "responseLogging": "none"}], "name": "asm_log_to_splunk"} Method POST mgmt/tm/security/log/profile returned: {"kind":"tm:security:log:profile:profilestate","name":"asm_log_to_splunk","fullPath":"asm_log_to_splunk","generation":240,"selfLink":"https://localhost/mgmt/tm/security/log/profile/asm_log_to_splunk?ver=11.6.0","applicationReference":{"link":"https://localhost/mgmt/tm/security/log/profile/~Common~asm_log_to_splunk/application?ver=11.6.0","isSubcollection":true}} 2015-11-12 09:49:23 : Deploying/updating Analytics Profile GET mgmt/tm/ltm/profile/analytics "" Method GET mgmt/tm/ltm/profile/analytics returned: {"kind":"tm:ltm:profile:analytics:analyticscollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/profile/analytics?ver=11.6.0","items":[{"kind":"tm:ltm:profile:analytics:analyticsstate","name":"analytics","partition":"Common","fullPath":"/Common/analytics","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/profile/analytics/~Common~analytics?ver=11.6.0","capturedTrafficExternalLogging":"disabled","capturedTrafficInternalLogging":"disabled","collectGeo":"disabled","collectIp":"disabled","collectMaxTpsAndThroughput":"disabled","collectMethods":"enabled","collectPageLoadTime":"disabled","collectResponseCodes":"enabled","collectSubnets":"disabled","collectUrl":"disabled","collectUserAgent":"disabled","collectUserSessions":"disabled","collectedStatsExternalLogging":"disabled","collectedStatsInternalLogging":"enabled","notificationByEmail":"disabled","notificationBySnmp":"disabled","notificationBySyslog":"disabled","publishIruleStatistics":"disabled","sampling":"enabled","sessionCookieSecurity":"ssl-only","sessionTimeoutMinutes":"5","alertsReference":{"link":"https://localhost/mgmt/tm/ltm/profile/analytics/~Common~analytics/alerts?ver=11.6.0","isSubcollection":true},"trafficCaptureReference":{"link":"https://localhost/mgmt/tm/ltm/profile/analytics/~Common~analytics/traffic-capture?ver=11.6.0","isSubcollection":true}}]} ... 2015-11-12 09:49:26 : Uploading Datagroup ... background for sorry page GET mgmt/tm/ltm/data-group/internal "" Method GET mgmt/tm/ltm/data-group/internal returned: {"kind":"tm:ltm:data-group:internal:internalcollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal?ver=11.6.0","items":[{"kind":"tm:ltm:data-group:internal:internalstate","name":"aol","partition":"Common","fullPath":"/Common/aol","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/~Common~aol?ver=11.6.0","type":"ip","records":[{"name":"64.12.96.0/19"},{"name":"195.93.16.0/20"},{"name":"195.93.48.0/22"},{"name":"195.93.64.0/19"},{"name":"195.93.96.0/19"},{"name":"198.81.0.0/22"},{"name":"198.81.8.0/23"},{"name":"198.81.16.0/20"},{"name":"202.67.65.128/25"},{"name":"205.188.112.0/20"},{"name":"205.188.146.144/30"},{"name":"205.188.192.0/20"},{"name":"205.188.208.0/23"},{"name":"207.200.112.0/21"}]},{"kind":"tm:ltm:data-group:internal:internalstate","name":"images","partition":"Common","fullPath":"/Common/images","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/~Common~images?ver=11.6.0","type":"string","records":[{"name":".bmp"},{"name":".gif"},{"name":".jpg"}]},{"kind":"tm:ltm:data-group:internal:internalstate","name":"private_net","partition":"Common","fullPath":"/Common/private_net","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/~Common~private_net?ver=11.6.0","type":"ip","records":[{"name":"10.0.0.0/8"},{"name":"172.16.0.0/12"},{"name":"192.168.0.0/16"}]}]} 2015-11-12 09:49:26 : Uploading Datagroup ... background for sorry page POST mgmt/tm/ltm/data-group/internal {"records": [{"name": "...<base64 image>..."}], "type": "string", "name": "background_images"} Method POST mgmt/tm/ltm/data-group/internal returned: {"kind":"tm:ltm:data-group:internal:internalstate","name":"background_images","fullPath":"background_images","generation":244,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/background_images?ver=11.6.0","type":"string","records":[{"name":"....large base64 image...."}]} 2015-11-12 09:49:29 : Uploading Datagroup ... image for sorry page GET mgmt/tm/ltm/data-group/internal "" Method GET mgmt/tm/ltm/data-group/internal returned: {"kind":"tm:ltm:data-group:internal:internalcollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal?ver=11.6.0","items":[{"kind":"tm:ltm:data-group:internal:internalstate","name":"aol","partition":"Common","fullPath":"/Common/aol","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/~Common~aol?ver=11.6.0","type":"ip","records":[{"name":"64.12.96.0/19"},{"name":"195.93.16.0/20"},{"name":"195.93.48.0/22"},{"name":"195.93.64.0/19"},{"name":"195.93.96.0/19"},{"name":"198.81.0.0/22"},{"name":"198.81.8.0/23"},{"name":"198.81.16.0/20"},{"name":"202.67.65.128/25"},{"name":"205.188.112.0/20"},{"name":"205.188.146.144/30"},{"name":"205.188.192.0/20"},{"name":"205.188.208.0/23"},{"name":"207.200.112.0/21"}]},{"kind":"tm:ltm:data-group:internal:internalstate","name":"background_images","partition":"Common","fullPath":"/Common/background_images","generation":244,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/~Common~background_images?ver=11.6.0","type":"string","records":[{"name":"...<base64 image>..."}]},{"kind":"tm:ltm:data-group:internal:internalstate","name":"images","partition":"Common","fullPath":"/Common/images","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/~Common~images?ver=11.6.0","type":"string","records":[{"name":".bmp"},{"name":".gif"},{"name":".jpg"}]},{"kind":"tm:ltm:data-group:internal:internalstate","name":"private_net","partition":"Common","fullPath":"/Common/private_net","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/~Common~private_net?ver=11.6.0","type":"ip","records":[{"name":"10.0.0.0/8"},{"name":"172.16.0.0/12"},{"name":"192.168.0.0/16"}]}]} 2015-11-12 09:49:30 : Uploading Datagroup ... image for sorry page POST mgmt/tm/ltm/data-group/internal {"records": [{"name": "...<base64 image>..."}], "type": "string", "name": "sorry_images"} Method POST mgmt/tm/ltm/data-group/internal returned: {"kind":"tm:ltm:data-group:internal:internalstate","name":"sorry_images","fullPath":"sorry_images","generation":245,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/sorry_images?ver=11.6.0","type":"string","records":[{"name":"....base 64 image...."}]} 2015-11-12 09:49:32 : Uploading iRules ... sorry_page_rule GET mgmt/tm/ltm/rule "" Method GET mgmt/tm/ltm/rule returned: {"kind":"tm:ltm:rule:rulecollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/rule?ver=11.6.0","items":[<a whole bunch of irules>]} 2015-11-12 09:49:35 : Uploading iRules ... demo_analytics_rule GET mgmt/tm/ltm/rule "" Method GET mgmt/tm/ltm/rule returned: {"kind":"tm:ltm:rule:rulecollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/rule?ver=11.6.0","items":[<a whole bunnch of irules>]} 2015-11-12 09:49:38 : Create the ASM policy GET mgmt/tm/asm/policies "" Method GET mgmt/tm/asm/policies returned: {"selfLink":"https://localhost/mgmt/tm/asm/policies","kind":"tm:asm:policies:policycollectionstate","totalItems":0,"items":[]} 2015-11-12 09:49:39 : Create the ASM policy POST mgmt/tm/asm/policies {"caseInsensitive": true, "name": "linux_high-Vip1", "applicationLanguage": "utf-8"} ... 2015-11-12 09:49:53 : Import our policy over the one existing above POST mgmt/tm/asm/tasks/import-policy {"policyReference": {"link": "https://localhost/mgmt/tm/asm/policies/qnU5A8PUMuPurLRLUt8VHg"}, "isBase64": true, "file": "...<base64 policy file>...","lastUpdateMicros":1.447350597e+15,"selfLink":"https://localhost/mgmt/tm/asm/tasks/import-policy/hP37L9EM650WeWKkgX7law","kind":"tm:asm:tasks:import-policy:import-policy-taskstate","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/qnU5A8PUMuPurLRLUt8VHg"},"startTime":"2015-11-12T17:49:57Z","id":"hP37L9EM650WeWKkgX7law"} 2015-11-12 09:50:00 : Determine whether the asm policy import task is complete GET mgmt/tm/asm/tasks/import-policy/hP37L9EM650WeWKkgX7law "" Method GET mgmt/tm/asm/tasks/import-policy/hP37L9EM650WeWKkgX7law returned: {"isBase64":true,"status":"STARTED","file":"...<base64 policy file>...","lastUpdateMicros":1.447350597e+15,"selfLink":"https://localhost/mgmt/tm/asm/tasks/import-policy/hP37L9EM650WeWKkgX7law","kind":"tm:asm:tasks:import-policy:import-policy-taskstate","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/qnU5A8PUMuPurLRLUt8VHg"},"startTime":"2015-11-12T17:49:57Z","id":"hP37L9EM650WeWKkgX7law"} 2015-11-12 09:50:05 : Determine whether the asm policy import task is complete GET mgmt/tm/asm/tasks/import-policy/hP37L9EM650WeWKkgX7law "" Method GET mgmt/tm/asm/tasks/import-policy/hP37L9EM650WeWKkgX7law returned: {"isBase64":true,"status":"COMPLETED","file":"...<base64 policy file>...","lastUpdateMicros":1.447350605e+15,"selfLink":"https://localhost/mgmt/tm/asm/tasks/import-policy/hP37L9EM650WeWKkgX7law","kind":"tm:asm:tasks:import-policy:import-policy-taskstate","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/qnU5A8PUMuPurLRLUt8VHg"},"endTime":"2015-11-12T17:50:05Z","startTime":"2015-11-12T17:49:57Z","id":"hP37L9EM650WeWKkgX7law","result":{"policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/qnU5A8PUMuPurLRLUt8VHg"},"message":"Security policy version information will be ignored, since the file has been modified since it was exported.\nSignature Set linux-high (previously used in this security policy) was added to this system.\nThe operation was completed successfully. The security policy name is '/Common/linux_high-Vip1'."}} 2015-11-12 09:50:08 : Apply the ASM policy POST mgmt/tm/asm/tasks/apply-policy {"policyReference": {"link": "https://localhost/mgmt/tm/asm/policies/qnU5A8PUMuPurLRLUt8VHg"}} Method POST mgmt/tm/asm/tasks/apply-policy returned: {"selfLink":"https://localhost/mgmt/tm/asm/tasks/apply-policy/38B8slfPm1_lBBRG1STNeg","kind":"tm:asm:tasks:apply-policy:apply-policy-taskstate","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/qnU5A8PUMuPurLRLUt8VHg"},"status":"NEW","lastUpdateMicros":1.447350608e+15,"startTime":"2015-11-12T17:50:08Z","id":"38B8slfPm1_lBBRG1STNeg"} 2015-11-12 09:50:10 : Determine whether the asm policy apply task is complete GET mgmt/tm/asm/tasks/apply-policy/38B8slfPm1_lBBRG1STNeg "" Method GET mgmt/tm/asm/tasks/apply-policy/38B8slfPm1_lBBRG1STNeg returned: {"selfLink":"https://localhost/mgmt/tm/asm/tasks/apply-policy/38B8slfPm1_lBBRG1STNeg","kind":"tm:asm:tasks:apply-policy:apply-policy-taskstate","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/qnU5A8PUMuPurLRLUt8VHg"},"status":"STARTED","lastUpdateMicros":1.447350608e+15,"startTime":"2015-11-12T17:50:08Z","id":"38B8slfPm1_lBBRG1STNeg"} 2015-11-12 09:50:14 : Determine whether the asm policy apply task is complete GET mgmt/tm/asm/tasks/apply-policy/38B8slfPm1_lBBRG1STNeg "" Method GET mgmt/tm/asm/tasks/apply-policy/38B8slfPm1_lBBRG1STNeg returned: {"selfLink":"https://localhost/mgmt/tm/asm/tasks/apply-policy/38B8slfPm1_lBBRG1STNeg","kind":"tm:asm:tasks:apply-policy:apply-policy-taskstate","policyReference":{"link":"https://localhost/mgmt/tm/asm/policies/qnU5A8PUMuPurLRLUt8VHg"},"status":"STARTED","lastUpdateMicros":1.447350608e+15,"startTime":"2015-11-12T17:50:08Z","id":"38B8slfPm1_lBBRG1STNeg"} 2015-11-12 09:50:20 : Create an LTM policy for use with by iApp which associates the ASM policy GET mgmt/tm/ltm/policy "" Method GET mgmt/tm/ltm/policy returned: {"kind":"tm:ltm:policy:policycollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/policy?ver=11.6.0","items":[{"kind":"tm:ltm:policy:policystate","name":"_sys_CEC_SSL_client_policy","partition":"Common","fullPath":"/Common/_sys_CEC_SSL_client_policy","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/policy/~Common~_sys_CEC_SSL_client_policy?ver=11.6.0","controls":["classification"],"hints":["no-write","no-delete","no-exclusion"],"requires":["ssl-persistence"],"strategy":"/Common/first-match","rulesReference":{"link":"https://localhost/mgmt/tm/ltm/policy/~Common~_sys_CEC_SSL_client_policy/rules?ver=11.6.0","isSubcollection":true}},{"kind":"tm:ltm:policy:policystate","name":"_sys_CEC_SSL_server_policy","partition":"Common","fullPath":"/Common/_sys_CEC_SSL_server_policy","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/policy/~Common~_sys_CEC_SSL_server_policy?ver=11.6.0","controls":["classification"],"hints":["no-write","no-delete","no-exclusion"],"requires":["ssl-persistence"],"strategy":"/Common/first-match","rulesReference":{"link":"https://localhost/mgmt/tm/ltm/policy/~Common~_sys_CEC_SSL_server_policy/rules?ver=11.6.0","isSubcollection":true}},{"kind":"tm:ltm:policy:policystate","name":"_sys_CEC_video_policy","partition":"Common","fullPath":"/Common/_sys_CEC_video_policy","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/policy/~Common~_sys_CEC_video_policy?ver=11.6.0","controls":["classification"],"hints":["no-write","no-delete","no-exclusion"],"requires":["http"],"strategy":"/Common/first-match","rulesReference":{"link":"https://localhost/mgmt/tm/ltm/policy/~Common~_sys_CEC_video_policy/rules?ver=11.6.0","isSubcollection":true}}]} 2015-11-12 09:50:23 : Create an LTM policy for use with by iApp which associates the ASM policy POST mgmt/tm/ltm/policy {"name": "ltm_policy_w_asm_linux_high-Vip1", "rules": [{"ordinal": 1, "conditions": [], "name": "rule-1", "actions": [{"status": 0, "enable": true, "name": "0", "request": true, "vlanId": 0, "code": 0, "policy": "/Common/linux_high-Vip1", "port": 0, "asm": true}]}], "partition": "Common", "controls": ["asm"], "strategy": "/Common/first-match", "requires": ["http"]} Method POST mgmt/tm/ltm/policy returned: {"kind":"tm:ltm:policy:policystate","name":"ltm_policy_w_asm_linux_high-Vip1","partition":"Common","fullPath":"/Common/ltm_policy_w_asm_linux_high-Vip1","generation":318,"selfLink":"https://localhost/mgmt/tm/ltm/policy/~Common~ltm_policy_w_asm_linux_high-Vip1?ver=11.6.0","controls":["asm"],"requires":["http"],"strategy":"/Common/first-match","rulesReference":{"link":"https://localhost/mgmt/tm/ltm/policy/~Common~ltm_policy_w_asm_linux_high-Vip1/rules?ver=11.6.0","isSubcollection":true}} 2015-11-12 09:50:25 : Deploy the iApp template, since we are not using a default iApp on box GET mgmt/tm/sys/application/template "" Method GET mgmt/tm/sys/application/template returned: {"kind":"tm:sys:application:template:templatecollectionstate","selfLink":"https://localhost/mgmt/tm/sys/application/template?ver=11.6.0","items":[...<list of iApp templates on the box>...]} 2015-11-12 09:50:30 : Deploy the iApp service from the f5 http backport template GET mgmt/tm/sys/application/service "" Method GET mgmt/tm/sys/application/service returned: {"kind":"tm:sys:application:service:servicecollectionstate","selfLink":"https://localhost/mgmt/tm/sys/application/service?ver=11.6.0"} 2015-11-12 09:51:37 : Deploying/updating webserver pool GET mgmt/tm/ltm/pool "" Method GET mgmt/tm/ltm/pool returned: {"kind":"tm:ltm:pool:poolcollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/pool?ver=11.6.0","items":[{"kind":"tm:ltm:pool:poolstate","name":"Vip1_pool","partition":"Common","fullPath":"/Common/Vip1_pool","generation":236,"selfLink":"https://localhost/mgmt/tm/ltm/pool/~Common~Vip1_pool?ver=11.6.0","allowNat":"yes","allowSnat":"yes","ignorePersistedWeight":"disabled","ipTosToClient":"pass-through","ipTosToServer":"pass-through","linkQosToClient":"pass-through","linkQosToServer":"pass-through","loadBalancingMode":"round-robin","minActiveMembers":0,"minUpMembers":0,"minUpMembersAction":"failover","minUpMembersChecking":"disabled","monitor":"/Common/http ","queueDepthLimit":0,"queueOnConnectionLimit":"disabled","queueTimeLimit":0,"reselectTries":0,"serviceDownAction":"none","slowRampTime":10,"membersReference":{"link":"https://localhost/mgmt/tm/ltm/pool/~Common~Vip1_pool/members?ver=11.6.0","isSubcollection":true}},{"kind":"tm:ltm:pool:poolstate","name":"syslog_pool","partition":"Common","fullPath":"/Common/syslog_pool","generation":239,"selfLink":"https://localhost/mgmt/tm/ltm/pool/~Common~syslog_pool?ver=11.6.0","allowNat":"yes","allowSnat":"yes","ignorePersistedWeight":"disabled","ipTosToClient":"pass-through","ipTosToServer":"pass-through","linkQosToClient":"pass-through","linkQosToServer":"pass-through","loadBalancingMode":"round-robin","minActiveMembers":0,"minUpMembers":0,"minUpMembersAction":"failover","minUpMembersChecking":"disabled","monitor":"/Common/tcp ","queueDepthLimit":0,"queueOnConnectionLimit":"disabled","queueTimeLimit":0,"reselectTries":0,"serviceDownAction":"none","slowRampTime":10,"membersReference":{"link":"https://localhost/mgmt/tm/ltm/pool/~Common~syslog_pool/members?ver=11.6.0","isSubcollection":true}}]} 2015-11-12 09:51:37 : Deploying/updating webserver pool POST mgmt/tm/ltm/pool {"name": "Vip2_pool", "members": [{"description": "Name=/boring_lovelace,ContainerHostname=a0085832ad28,Image=mutzel/all-in-one-hackazon:postinstall", "name": "172.16.14.87:80", "address": "172.16.14.87"}], "monitor": "http"} Method POST mgmt/tm/ltm/pool returned: {"kind":"tm:ltm:pool:poolstate","name":"Vip2_pool","fullPath":"Vip2_pool","generation":352,"selfLink":"https://localhost/mgmt/tm/ltm/pool/Vip2_pool?ver=11.6.0","allowNat":"yes","allowSnat":"yes","ignorePersistedWeight":"disabled","ipTosToClient":"pass-through","ipTosToServer":"pass-through","linkQosToClient":"pass-through","linkQosToServer":"pass-through","loadBalancingMode":"round-robin","minActiveMembers":0,"minUpMembers":0,"minUpMembersAction":"failover","minUpMembersChecking":"disabled","monitor":"/Common/http ","queueDepthLimit":0,"queueOnConnectionLimit":"disabled","queueTimeLimit":0,"reselectTries":0,"serviceDownAction":"none","slowRampTime":10,"membersReference":{"link":"https://localhost/mgmt/tm/ltm/pool/~Common~Vip2_pool/members?ver=11.6.0","isSubcollection":true}} 2015-11-12 09:51:39 : Uploading iRules ... irule_random_snat GET mgmt/tm/ltm/rule "" Method GET mgmt/tm/ltm/rule returned: {"kind":"tm:ltm:rule:rulecollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/rule?ver=11.6.0","items":[...<list of irules on the box>...]} ... 2015-11-12 09:51:43 : Setup the HTTP virtual server POST mgmt/tm/ltm/virtual {"name": "Vip2_http", "rules": ["/Common/irule_random_snat"], "translateAddress": "enabled", "destination": "/Common/172.16.13.145:80", "mask": "255.255.255.255", "sourceAddressTranslation": {"type": "automap"}, "profiles": [{"name": "http"}, {"name": "tcp-wan-optimized", "context": "clientside"}, {"name": "tcp-lan-optimized", "context": "serverside"}], "translatePort": "enabled", "ipProtocol": "tcp", "pool": "/Common/Vip2_pool"} Method POST mgmt/tm/ltm/virtual returned: {"kind":"tm:ltm:virtual:virtualstate","name":"Vip2_http","fullPath":"Vip2_http","generation":355,"selfLink":"https://localhost/mgmt/tm/ltm/virtual/Vip2_http?ver=11.6.0","addressStatus":"yes","autoLasthop":"default","cmpEnabled":"yes","connectionLimit":0,"destination":"/Common/172.16.13.145:80","enabled":true,"gtmScore":0,"ipProtocol":"tcp","mask":"255.255.255.255","mirror":"disabled","mobileAppTunnel":"disabled","nat64":"disabled","pool":"/Common/Vip2_pool","rateLimit":"disabled","rateLimitDstMask":0,"rateLimitMode":"object","rateLimitSrcMask":0,"source":"0.0.0.0/0","sourceAddressTranslation":{"type":"automap"},"sourcePort":"preserve","synCookieStatus":"not-activated","translateAddress":"enabled","translatePort":"enabled","vlansDisabled":true,"vsIndex":4,"rules":["/Common/irule_random_snat"],"policiesReference":{"link":"https://localhost/mgmt/tm/ltm/virtual/~Common~Vip2_http/policies?ver=11.6.0","isSubcollection":true},"profilesReference":{"link":"https://localhost/mgmt/tm/ltm/virtual/~Common~Vip2_http/profiles?ver=11.6.0","isSubcollection":true}} 2015-11-12 09:51:45 : Setup the HTTPS virtual server POST mgmt/tm/ltm/virtual {"name": "Vip2_https", "rules": ["/Common/irule_random_snat"], "translateAddress": "enabled", "destination": "/Common/172.16.13.145:443", "mask": "255.255.255.255", "sourceAddressTranslation": {"type": "automap"}, "profiles": [{"name": "tcp-ssl-wan-optimized", "context": "clientside"}, {"name": "tcp-ssl-lan-optimized", "context": "serverside"}], "translatePort": "enabled", "ipProtocol": "tcp", "pool": "/Common/Vip2_pool"} Method POST mgmt/tm/ltm/virtual returned: {"kind":"tm:ltm:virtual:virtualstate","name":"Vip2_https","fullPath":"Vip2_https","generation":356,"selfLink":"https://localhost/mgmt/tm/ltm/virtual/Vip2_https?ver=11.6.0","addressStatus":"yes","autoLasthop":"default","cmpEnabled":"yes","connectionLimit":0,"destination":"/Common/172.16.13.145:443","enabled":true,"gtmScore":0,"ipProtocol":"tcp","mask":"255.255.255.255","mirror":"disabled","mobileAppTunnel":"disabled","nat64":"disabled","pool":"/Common/Vip2_pool","rateLimit":"disabled","rateLimitDstMask":0,"rateLimitMode":"object","rateLimitSrcMask":0,"source":"0.0.0.0/0","sourceAddressTranslation":{"type":"automap"},"sourcePort":"preserve","synCookieStatus":"not-activated","translateAddress":"enabled","translatePort":"enabled","vlansDisabled":true,"vsIndex":5,"rules":["/Common/irule_random_snat"],"policiesReference":{"link":"https://localhost/mgmt/tm/ltm/virtual/~Common~Vip2_https/policies?ver=11.6.0","isSubcollection":true},"profilesReference":{"link":"https://localhost/mgmt/tm/ltm/virtual/~Common~Vip2_https/profiles?ver=11.6.0","isSubcollection":true}} Deploying an iApp template using iControlREST Because we recognize that it may be not obvious how we are deploying iApp templates using iControlREST, we break it down into more detail here. First, note that there is no 'import' action we can invoke via REST to import the iApp template which mirrors the action in the Configuration Utility (GUI). This means that we need to create the JSON payload containing the iApp and POST it. Given an iApp template, like those found on DevCentral, here are the steps to create the JSON body. On a pre-existing BIG-IP install (or one have created in your build process for your code) Import the iApp template in the Configuration Utility in the 'Common' partition Do an HTTP GET to retrieve the iApp template payload. Make sure that you use the expandSubcollections=True as a query parameter, as we want to include the stuff in the 'actionsReference' sub-collection. curl -sku <user>:<password> -X GET https://<management ip>/mgmt/tm/sys/application/template/~Common~<name of your iApp>?expandSubcollections=true You should get something back that looks like the following (which is the payload for the f5.http backport iApp). I have truncated the 'implementation', 'presentation' and 'htmlHelp' actions: { "actionsReference": { "isSubcollection": true, "items": [ { "fullPath": "definition", "generation": 4672, "htmlHelp": "....", "implementation": "...", "kind": "tm:sys:application:template:actions:actionsstate", "name": "definition", "presentation": "...", "roleAcl": [ "admin", "manager", "resource-admin" ], "selfLink": "https://localhost/mgmt/tm/sys/application/template/~Common~f5.http.backport.1.1.2/actions/definition?ver=11.6.0" } ], "link": "https://localhost/mgmt/tm/sys/application/template/~Common~f5.http.backport.1.1.2/actions?ver=11.6.0" }, "fullPath": "/Common/f5.http.backport.1.1.2", "generation": 4672, "ignoreVerification": "false", "kind": "tm:sys:application:template:templatestate", "name": "f5.http.backport.1.1.2", "partition": "Common", "requiresBigipVersionMin": "11.6.0", "selfLink": "https://localhost/mgmt/tm/sys/application/template/~Common~f5.http.backport.1.1.2?expandSubcollections=true&ver=11.6.0", "totalSigningStatus": "not-all-signed", "verificationStatus": "none" } Before we can POST this payload back to any BIG-IP, we need to cleanup a few things: Remove any of the extraneousfields including 'verificationStatus', 'totalSigningStatus', 'selfLink', 'partition', 'kind', 'generation', 'fullPath'. Make a new top-level key in the payload called 'actions'. The value for this key should everything in the 'items' array under the top-level key 'actionsReference'. Finally, delete the 'actionsReference' key/value pair from the JSON body. The final JSON payload should look like: { "actions": [ { "htmlHelp": "....", "implementation": "...", "name": "definition", "presentation": "...", "roleAcl": [ "admin", "manager", "resource-admin" ] } ], "ignoreVerification": "false", "name": "f5.http.backport.1.1.2", "requiresBigipVersionMin": "11.6.0", "totalSigningStatus": "not-all-signed" } Finally, we can use this to deploy an iApp template on BIG-IP. In the example below, the iApp_template.json file is formatted like the above. I have also attached it to this page for inspection. curl -sku rest_admin:<obfuscated> -H "Content-type: application/json" -X POST -d@./iApp_template.json https://52.23.149.16//mgmt/tm/sys/application/template Before you go POSTing iApps to any old version of TMOS, be aware that there are still some remaining issues you might have to solve. Some of the official iApps found on DevCentral are prepended with aTCL library that defines functions used within the iApp. The iApp solutions team made this design decision so that newer iApps will work against older versions of TMOS. For example, see the 'F5 HTTP', which starts with the library definition on line 0: "cli script f5.iapp.1.3.0.cli {....". When you export the iApp template to JSON using REST as we have documented above, this library will not be included in the payload. Because newer versions of BIG-IP (11.6) might already include a version of this 'iApp' library, you can work around this issue by updating the function references to use the existing library on-box. Here are the high-level steps: Downloading the iApp from DevCentral Change the function references to leverage the library that is installed on your BIG-IP. See an example of this by comparing the F5 HTTP template on the codeshare with the one attached to this page. You'll probably have to do some "find and replace" like the following: f5.iapp.1.3.0.cli:iapp_get_provisioned -> iapp::get_provisioned There may be some references to functions that do not exist yet. These will have to be dealt with on a case-by-case basis. Uploading the iApp to BIG-IP and exporting we documented above. Deploying an iApp service using iControlREST Fortunately, using iControlREST to manage instances of iApps (also known as iApp services) is much easier than managing templates. The high-level steps are similar: Deploy an iApp service via the Configuration Utility. Do an HTTP GET to acquire the JSON representation (notice the URL formatting!). curl -sku <user>:<password> -X GET https://<management ip>/mgmt/tm/sys/application/service/~Common~<your iapp name>.app~<your iapp name> Depending on the variables presented by the iApp template, the JSON payload for the iApp service might look something like: { "deviceGroup": "none", "fullPath": "/Common/Vip1_iApp.app/Vip1_iApp", "generation": 4674, "inheritedDevicegroup": "true", "inheritedTrafficGroup": "true", "kind": "tm:sys:application:service:servicestate", "lists": [ { "encrypted": "no", "name": "irules__irules", "value": [ "/Common/irule_demo_analytics", "/Common/irule_sorry_page" ] } ], "name": "Vip1_iApp", "partition": "Common", "selfLink": "https://localhost/mgmt/tm/sys/application/service/~Common~Vip1_iApp.app~Vip1_iApp?ver=11.6.0", "strictUpdates": "enabled", "subPath": "Vip1_iApp.app", "tables": [ { "name": "basic__snatpool_members" }, { "name": "net__snatpool_members" }, { "name": "optimizations__hosts" }, { "columnNames": [ "name" ], "name": "pool__hosts", "rows": [ { "row": [ "demo.example.com" ] } ] }, { "name": "pool__members" }, { "name": "server_pools__servers" } ], "template": "/Common/f5.http.backport.1.1.2", "templateModified": "yes", "trafficGroup": "/Common/traffic-group-1", "variables": [ { "encrypted": "no", "name": "asm__security_logging", "value": "asm_log_to_splunk" }, { "encrypted": "no", "name": "asm__use_asm", "value": "/Common/ltm_policy_w_asm_linux_high-Vip1" }, { "encrypted": "no", "name": "client__http_compression", "value": "/#do_not_use#" }, { "encrypted": "no", "name": "client__standard_caching_without_wa", "value": "/#do_not_use#" }, { "encrypted": "no", "name": "client__tcp_wan_opt", "value": "/Common/tcp-ssl-wan-optimized" }, { "encrypted": "no", "name": "net__client_mode", "value": "wan" }, { "encrypted": "no", "name": "net__route_to_bigip", "value": "no" }, { "encrypted": "no", "name": "net__same_subnet", "value": "no" }, { "encrypted": "no", "name": "net__server_mode", "value": "lan" }, { "encrypted": "no", "name": "net__snat_type", "value": "automap" }, { "encrypted": "no", "name": "net__vlan_mode", "value": "all" }, { "encrypted": "no", "name": "pool__addr", "value": "172.16.13.128" }, { "encrypted": "no", "name": "pool__http", "value": "/#create_new#" }, { "encrypted": "no", "name": "pool__mask", "value": "none" }, { "encrypted": "no", "name": "pool__persist", "value": "/#cookie#" }, { "encrypted": "no", "name": "pool__pool_to_use", "value": "/Common/Vip1_pool" }, { "encrypted": "no", "name": "pool__port_secure", "value": "443" }, { "encrypted": "no", "name": "pool__redirect_port", "value": "80" }, { "encrypted": "no", "name": "pool__redirect_to_https", "value": "yes" }, { "encrypted": "no", "name": "pool__xff", "value": "yes" }, { "encrypted": "no", "name": "server__oneconnect", "value": "/#do_not_use#" }, { "encrypted": "no", "name": "server__tcp_lan_opt", "value": "/Common/tcp-wan-optimized" }, { "encrypted": "no", "name": "server__tcp_req_queueing", "value": "no" }, { "encrypted": "no", "name": "ssl__cert", "value": "/Common/default.crt" }, { "encrypted": "no", "name": "ssl__client_ssl_profile", "value": "/#create_new#" }, { "encrypted": "no", "name": "ssl__key", "value": "/Common/default.key" }, { "encrypted": "no", "name": "ssl__mode", "value": "client_ssl" }, { "encrypted": "no", "name": "ssl__use_chain_cert", "value": "/#do_not_use#" }, { "encrypted": "no", "name": "ssl_encryption_questions__advanced", "value": "yes" }, { "encrypted": "no", "name": "ssl_encryption_questions__help", "value": "hide" }, { "encrypted": "no", "name": "stats__analytics", "value": "/Common/Vip1-demo_analytics" }, { "encrypted": "no", "name": "stats__request_logging", "value": "/#do_not_use#" } ] } As ealier, remove some of the fields that don't make sense to re-post. This includes 'deviceGroup', 'fullPath', 'generation', 'kind', 'partition', 'selfLink', and 'subPath'. You can now use this JSON body with updates to the variable values as needed. Example python code for deploying iApp Templates In addition to the above procedures, we'd like you point you to some python examples which show how to push iApp templates using REST. Hitesh Patel, another monster F5er, has put together the following code: https://github.com/0xHiteshPatel/appsvcs_integration_iapp/tree/80cc40dcf85e352a25c7ec44d9e4dcc253e51e69/scripts In his words: "that's 152 lines of awesome right there". His examples run against 11.5.x, 11.6.x and 12.0. Debugging When trying to create or update an instance of an iApp via REST, you will get error messages in the HTTP response if your POST is unsuccessful. In addition to the HTTP payload in the response, the following debug steps can be helpful: 1) Set the scriptd log level to debug: modify sys scriptd log-level debug 2) Look at the TMSH output from the iApp printed to /var/log/scriptd.out. Typically the last line will show the error that has occured. In closing The above examples should bring you one step closer to automating the delivery of advanced network services for your applications. We're looking forward to doing future posts on how to automate your deployment. Finally, if you haven't checked out the Application Services Integration iApp, also by Hitesh, you should probably do so now:https://github.com/0xHiteshPatel/appsvcs_integration_iapp. Cheers!Getting Started with iApps: Modifying an Existing iApp
Now that the esteemed Chase Abbott has blazed the trail with all the iApps foundational goodness surrounding concepts and componentsin the first two articles in this Getting Started with iApps series, we can move into the practical! In this article, we'll start small, copying an existing iApp template and modifying it slightly. The IP Forwarding iApp Before we rip open the hood and dismount the engine, let's first take a walk through to evaluate the existing options. As indicated in the video above, we're going to make a couple modifications to this iApp. I don't want the end user of the iApp to have ability to forward all traffic as a default option, regardless of IP version. I want to make sure the port is all ports; no port specification supported. I want to disable the advanced features altogether, removing the ability to specify the fastL4 profile and iRules. Now that we've established what we're going to do, let's get it done! Existing Template The existing template is shown by section below: Implementation Presentation package require iapp 1.0.0 iapp::template start set app $tmsh::app_name set advanced [iapp::is ::options__advanced yes] # destination & mask array # array keys: $::basic__forward_all array set destination { IPv4 { destination any:any mask 0.0.0.0 } IPv6 { destination 0::0.any mask 0::0 } * { destination [iapp::destination $::basic__addr $::basic__port] \ mask $::basic__mask \ [expr { [string match "*:*:*" $::basic__addr] ? "source ::/0" : "" }]} } # PROFILES set use_default_profile [iapp::is ::basic__l4_profile_to_use "/#default#"] # array keys: $advanced,$use_default_profile array set profiles { 1,0 { profiles replace-all-with \{ $::basic__l4_profile_to_use \} } * { profiles replace-all-with \{ \ [iapp::conf create \ ltm profile fastl4 ${app}_fastL4 \ defaults-from fastL4 \ reset-on-timeout disabled \ loose-close enabled \ loose-initialization enabled] \} } } # VLANS # array keys: $::basic__vlan_listening array set vlans { enabled { vlans-enabled vlans replace-all-with \{ $::basic__vlan_selections \} } disabled { vlans-disabled vlans replace-all-with \{ $::basic__vlan_selections \} } default { vlans-disabled vlans none } } # RULES set have_irules [expr { ![iapp::is ::irules__irules ""] }] # array keys $advanced,$have_irules array set rules { 1,1 { rules \{ $::irules__irules \} } * { rules none } } # VIRTUALS set virtual { [iapp::conf create ltm virtual ${app}_VER_Forwarding \ ip-forward \ [iapp::substa destination(VER)] \ [iapp::substa profiles($advanced,$use_default_profile)] \ [iapp::substa vlans($::basic__vlan_listening)] \ [iapp::substa rules($advanced,$have_irules)]] } switch $::basic__forward_all { Yes { subst [string map "VER IPv4" $virtual] subst [string map "VER IPv6" $virtual] } No { subst [string map "VER IP" $virtual] } default { subst [string map "VER $::basic__forward_all" $virtual] } } iapp::template stop section intro { message hello message check_for_updates } section options { choice display_help display "xxlarge" default "hide" optional ( options.display_help == "max" ) { message display_help_max } choice advanced display "xxlarge" default "no" optional ( options.display_help == "max" ) { message conf_mode_max } } section basic { choice forward_all display "xxlarge" default "No" optional ( options.display_help == "max" ) { message forward_all_max } optional ( forward_all == "No" ) { string addr default "0.0.0.0" required validator "IpAddress" optional ( options.display_help == "max" ) { message addr_max } string mask default "0.0.0.0" required validator "IpAddress" optional ( options.display_help == "max" ) { message mask_max } string port default "0" required validator "PortNumber" optional ( options.display_help == "max" ) { message port_max } } choice vlan_listening default "default" display "xxlarge" optional ( options.display_help == "max" ) { message vlan_listening_max } optional ( vlan_listening != "default" ) { multichoice vlan_selections display "xlarge" tcl { package require iapp 1.0.0 set ::choices [iapp::get_items net vlan] return [iapp::safe_display ::choices] } optional ( vlan_listening == "enabled" ) { optional ( options.display_help == "max" ) { message vlan_selections_enabled_max } } optional ( vlan_listening == "disabled" ) { optional ( options.display_help == "max" ) { message vlan_selections_disabled_max } } } optional ( options.advanced == "yes" ) { choice l4_profile_to_use default "/#default#" display "xxlarge" tcl { package require iapp 1.0.0 set ::choices "/#default#\n[iapp::get_items ltm profile fastl4]" return [iapp::safe_display ::choices] } optional ( options.display_help == "max" ) { message l4_profile_to_use_max } } } optional ( options.advanced == "yes" ) { section irules { message irule_1_max optional ( options.display_help == "max" ) { message irule_2_max message irule_3_max } multichoice irules display "xlarge" tcl { package require iapp 1.0.0 set ::choices [iapp::get_items -filter NAME !~ "^_sys_" ltm rule] return [iapp::safe_display ::choices] } } } text { intro "Welcome to the IP Forwarding template" intro.hello "Introduction" "This template supports configuring the BIG-IP to perform IP traffic forwarding." intro.check_for_updates "Check for Updates" "Check for new versions of this template on the AskF5 Knowledge Base website (http://support.f5.com/kb/en-us/solutions/public/13000/400/sol13422.html)." options "Template Options" options.display_help "Do you want to see inline help?" { "Yes, show inline help" => "max", "No, do not show inline help" => "hide" } options.display_help_max "" "Inline help is available to provide contextual descriptions to aid in the completion of this configuration. Select to show or hide the inline help in this template. Important notes and warnings are always visible, no matter which selection you make here." options.advanced "Which configuration mode do you want to use?" { "Basic - Use F5's recommended settings" => "no", "Advanced - Expose advanced options" => "yes" } options.conf_mode_max "" "This template supports twp configurations modes. Basic mode exposes the most commonly used settings, and automatically configures the rest of the options. Advanced mode allows you to review and change all settings." basic "Virtual Server Questions" basic.forward_all "Should this forward all traffic not destined for virtual servers?" { "Forward all IPv4 traffic" => "IPv4", "Forward all IPv6 traffic" => "IPv6", "Forward all IPv4 and IPv6 traffic" => "Yes", "Forward specific traffic" => "No" } basic.forward_all_max "" "Choose whether to have the BIG-IP system forward all traffic that is not destined for a virtual server, or only forward traffic on a specific network, port and/or VLAN that you define. If you select Yes, the system forwards all traffic. If you select No, you must enter the address, mask, and port that define the traffic you want to forward." basic.addr "What network address do you want to use for this virtual server?" basic.addr_max "" "Specifies the address of the destination network, for example, 10.0.0.0. The default 0.0.0.0 is a wildcard for all addresses, however you can still limit what is forwarded by netmask, port, or VLAN." basic.mask "What is the netmask for that address?" basic.mask_max "" "Specifies the netmask, for example 255.255.255.0. The default 0.0.0.0 is a wildcard for any netmask." basic.port "What port do you want this virtual server to use?" basic.port_max "" "Specifies a specific port for forwarding traffic, for example 22. The default 0 is a wildcard for any port." basic.vlan_listening "What VLANs should this IP Forwarding virtual server listen on?" { "All VLANs" => "default", "Enabled on ..." => "enabled", "Disabled on ..." => "disabled" } basic.vlan_listening_max "" "You can choose to forward traffic from all VLANs or just from some of them. If you don't want to enable all VLANs, you can choose to include or exclude named VLANs and forward traffic from all of the rest." basic.vlan_selections "On which VLAN(s) should this IP Forwarding virtual server listen?" basic.vlan_selections_enabled_max "" "Select the VLANs from which traffic will be forwarded." basic.vlan_selections_disabled_max "" "Select the VLANs from which traffic NOT will be forwarded." basic.l4_profile_to_use "What Fast L4 profile do you want to use?" {"Use F5's recommended Fast L4 profile" => "/#default#"} basic.l4_profile_to_use_max "" "If you created a custom Fast L4 profile, you can select it from the list. This is not required for most purposes, and only necessary if you need to replace the default Fast L4 profile functionality." irules "iRules" irules.irules "Do you want to add any custom iRules to this configuration?" irules.irule_1_max "WARNING" "Improper use or misconfigurations of an iRule can result in unwanted application behavior and poor performance of your BIG-IP system. For this reason we recommended you verify the impact of an iRule prior to deployment in a production environment." irules.irule_2_max "" "The BIG-IP system supports a scripting language to allow an administrator to instruct the system to intercept, inspect, transform, direct and track inbound or outbound application traffic. An iRule contains the set of instructions the system uses to process data flowing through it, either in the header or payload of a packet." irules.irule_3_max "" "Correct event priority is critical when assigning multiple iRules. For more information about iRule event priority, see https://devcentral.f5.com/s/wiki/iRules.priority.ashx" } To get started with modifying this template, we want to create a copy. In the GUI, click iApps->Templates->f5.ip_forwarding, and then at the very bottom, click copy. At the iApps >> Clone Template screen, change the Template Name(s) field to the name you want your template to have. I called mine f5_stripped.ip_forwarding. You can edit from here, but at this step I just clicked Finish before re-entering to do the editing work. Back at the iApps template listing, you should see your new iApp, which other than title, should be the same as the original, with the exception of validation (which should now read None.) Click into your new template and now we'll get to work editing. Presentation In the presentation section, we can weed out the basic/advanced section since we aren't allowing profile or iRule manipulation (lines 13-19.) Next we'll eliminate the forward_all option since we're not giving them the choice of of IPv4/IPv6 defaults or all networks (lines 23-28; 41.) Moving on, we'll trim the port section (lines 37-40.) Finally, we'll pull the advanced section altogether, which deals with the profiles and iRules (lines 69-96.) In the text area of the presentation, we pull out all the unnecessary text now that those GUI objects will not be present (lines 110-14; 117-123; 130-131; 137; 142-153.) This is best shown in the following images. Implementation The changes in the implementation section follow suit. We remove the arrays for destination (lines 7-15) and profiles (lines 17-29,) as well as the army for rules (lines 38-46,) and the switch for the type of forwarding virtual this will be since we're forcing end users to specify the networks (lines 56-61.) The only thing we need to slightly modify is the virtuals section (lines 48-54.) Here, we are removing the need for the variable and just creating the virtual server with the iapp::conf command. We are not passing a port from the iApp, so we use :any here (line 15) and then use the array as is for vlans to configure those. Screenshot to show these updates is below. Easy enough! Updated iApp code is below. Implementation Presentation package require iapp 1.0.0 iapp::template start set app $tmsh::app_name # VLANS # array keys: $::basic__vlan_listening array set vlans { enabled { vlans-enabled vlans replace-all-with \{ $::basic__vlan_selections \} } disabled { vlans-disabled vlans replace-all-with \{ $::basic__vlan_selections \} } default { vlans-disabled vlans none } } iapp::conf create ltm virtual ${app}_IPv4_Forwarding \ ip-forward \ destination $::basic__addr:any mask $::basic__mask \ [iapp::substa vlans($::basic__vlan_listening)] iapp::template stop section intro { message hello message check_for_updates } section options { choice display_help display "xxlarge" default "hide" optional ( options.display_help == "max" ) { message display_help_max } } section basic { string addr default "0.0.0.0" required validator "IpAddress" optional ( options.display_help == "max" ) { message addr_max } string mask default "0.0.0.0" required validator "IpAddress" optional ( options.display_help == "max" ) { message mask_max } choice vlan_listening default "default" display "xxlarge" optional ( options.display_help == "max" ) { message vlan_listening_max } optional ( vlan_listening != "default" ) { multichoice vlan_selections display "xlarge" tcl { package require iapp 1.0.0 set ::choices [iapp::get_items net vlan] return [iapp::safe_display ::choices] } optional ( vlan_listening == "enabled" ) { optional ( options.display_help == "max" ) { message vlan_selections_enabled_max } } optional ( vlan_listening == "disabled" ) { optional ( options.display_help == "max" ) { message vlan_selections_disabled_max } } } } text { intro "Welcome to the IP Forwarding template" intro.hello "Introduction" "This template supports configuring the BIG-IP to perform IP traffic forwarding." intro.check_for_updates "Check for Updates" "Check for new versions of this template on the AskF5 Knowledge Base website (http://support.f5.com/kb/en-us/solutions/public/13000/400/sol13422.html)." options "Template Options" options.display_help "Do you want to see inline help?" { "Yes, show inline help" => "max", "No, do not show inline help" => "hide" } options.display_help_max "" "Inline help is available to provide contextual descriptions to aid in the completion of this configuration. Select to show or hide the inline help in this template. Important notes and warnings are always visible, no matter which selection you make here." basic "Virtual Server Questions" basic.addr "What network address do you want to use for this virtual server?" basic.addr_max "" "Specifies the address of the destination network, for example, 10.0.0.0. The default 0.0.0.0 is a wildcard for all addresses, however you can still limit what is forwarded by netmask, port, or VLAN." basic.mask "What is the netmask for that address?" basic.mask_max "" "Specifies the netmask, for example 255.255.255.0. The default 0.0.0.0 is a wildcard for any netmask." basic.vlan_listening "What VLANs should this IP Forwarding virtual server listen on?" { "All VLANs" => "default", "Enabled on ..." => "enabled", "Disabled on ..." => "disabled" } basic.vlan_listening_max "" "You can choose to forward traffic from all VLANs or just from some of them. If you don't want to enable all VLANs, you can choose to include or exclude named VLANs and forward traffic from all of the rest." basic.vlan_selections "On which VLAN(s) should this IP Forwarding virtual server listen?" basic.vlan_selections_enabled_max "" "Select the VLANs from which traffic will be forwarded." basic.vlan_selections_disabled_max "" "Select the VLANs from which traffic NOT will be forwarded." } With the finished stripped version of the ip forwarding iApp, we can now deploy the template in a new application service. Join us for the next article in this series, where we'll tackle writing an iApp from scratch!Getting Started with iApps: A Conceptual Overview
tl;dr - iApps provide admins and service desks a template solution for application deployment and management services. Deploying and managing applications require a lot of information across several disciplines. Architects have their holistic view of the application ecosystem and relevant lifecycles. Developers have their granular relationship with each application under their umbrella. Networks admins make sure applications are behaving appropriately on the network instead of hijacking QoS classes or hijacking DNS. Then there are those missing details that no one wants to own until something breaks (looking at you Java CA store). Originally F5 introduced deployment guides to help administrators understand the requirements and configurations needed to deploy popular applications behind BIG-IP. However, after the deployment was complete, those configurations were still managed through object types alone (e.g. virtual servers, pools, profiles, iRules, monitors). That can get quite tedious when you have hundreds of applications on a single BIG-IP stack. Someone somewhere said “Wouldn’t it be nice if we could have an application-based view of all the different objects that help us deploy, manage, and secure each application”? Enter iApps Introduced in BIG-IP 11.0, iApps are a customizable framework for deploying and managing application as a service. Using out-of-the-box templates, administrators can deploy commonly-used applications such as Oracle, SAP, or Exchange by completing a series of questions that relate to their management and infrastructure needs. Rather than create a bunch of virtual servers, followed by a handful of monitors, then a plethora of whatever, the responses to iApps questions create all of the BIG-IP objects needed to properly run your application. The iApps application service becomes the responsible manager of all virtual servers, monitors, policies, profiles, and iRules required to run. Consolidating these into a single view makes management and troubleshooting much easier to handle. iApps Framework iApps consist of two main elements, the template and application services created by publishing a template. We’ll dive into this in our next article, Getting Started With iApps: Components. Templates: The base configuration object which contains the layout and scripting used to configure and publish application instances. Some templates are prebuilt and included in BIG-IP, while others can be download from DevCentral (are not officially supported) or F5 support (certified and supported). Developer-oriented teams can also build custom templates for frequently used configurations or services. Application Service: An application service is the result of using an iApps template to drive the configuration process. The Administrator uses the configuration utility to create a new application service from the selected iApps template. Created objects are grouped into components of the application service and are managed accordingly. The iApps Advantage iApps are not for everyone. If you like keeping tribal control over your BIG-IP ecosystem or if you like naming virtual servers after your pets, iApps may not be for you. iApps do have an advantage if you want to templatize your deployment scenarios or wish to allow other administrators access to the services they manage. iApps reduce a lot of the mystique and intimidation a lengthy set of profiles, policies, and pools can sometimes cause to the new or intermediate administrator. Above we show an example of building a highly available LDAP namespace for internal applications with the default built-in LDAP iApps template. By providing a certificate and and answering a few questions, an LDAP environment is created for all of your internal directory authentication or lookup requirements. From there, modifying the configuration is easy as selecting the Reconfigure tab in the existing application service. Changing settings within iApps Sometimes you just want a template to assist with application deployment and from there you’re perfectly fine managing the individual object types. The Component view will show you all objects affected by the application service but if you try to apply a change, you’ll receive an error similar to: This is by design because the iApps application service is the rightful owner of the system object and shouldn’t be edited directly. However in certain cases you don’t the iApp anymore or want more granular control of some features the iApps my not have, there is an option. Each application service published via iApps have a Properties tab which allow you to disable the Strict Updates method of management. Unchecked, each object is configurable on it’s own but will deviate from the templates last known state. Some administrators prefer to operate this way, only using the iApp as a deployment method, and that’s perfectly fine. We’re leaving your application management style and method up to you. As BIG-IP expands to cover more of the application landscape, people are increasingly taking advantage of more programmatic features and iApps is no exception. Allowing our administrators to improve their ease of deployment and use is why iApps exist and we’ll continue to develop and improve these features. Our next article Getting Started with iApps: Components will dive into more detail on the properties required to create and manage iApps. Take the time to get to know iApps, they’re your ally for keeping your applications in order.Enabling SharePoint 2013 Hybrid Search with the BIG-IP
Over my past several blog posts, I’ve talked about federating our on-premise environments with Office 365. Now, that we have that handled, (it’s handled right?) let’s talk about another lesser known piece of the puzzle; hybrid SharePoint environments. With federation and SSO between on-premise applications and Office 365 provided by the BIG-IP and APM, (Access Policy Manager) users can move seamlessly between their on-premise SharePoint and SharePoint online as if they were one integrated application. Cool right? But what about when I need to search for content across environments? Funny I should ask. That’s what this post is all about. Microsoft actually has this figured out. The only piece missing is a device that can act as the reverse proxy for the on-premise environment and facilitate the secure connection between the two environments. Hmmm… What could we use? Configuring Hybrid Search for SharePoint 2013 This post covers the pieces required to use the BIG-IP as a reverse-proxy for a hybrid SharePoint search deployment. For guidance on configuring both your on-premise SharePoint 2013 and SharePoint Online environments refer to the Hybrid for SharePoint 2013 guidance provided by Microsoft. It’s quite thorough and, to be honest a little daunting. Currently, Microsoft’s SharePoint 2013 hybrid capabilities are intended to let users in Office 365 access and search across certain content from an on premises SharePoint farm. For detailed information on architectural recommendations and limitations take a look at the blog post from Microsoft architect, Steve Peschka. Client Certificate authentication is utilized to allow secure access for hybrid search, (SharePoint content located both on-premise and in the cloud) from Office 365 environment. To accomplish this a target application is configured in the SharePoint Online Secure Store where a client certificate, (issued by a public root certificate authority). Figure 1 – SharePoint Online Secure Store & Target application Configuring the BIG-IP for Client Certificate Authentication The following configuration was tested by F5 in collaboration with the Office365 product team. The testing was conducted at the Microsoft Technology Center in Irvine where an environment was provided for hybrid enterprise search & LOB scenarios between O365 and on-premise SharePoint farms. We collaborated with the MTC’s expert staff and took advantage of the alliance partnership between Microsoft and F5, whose solutions were made available as part of their engagement. For the purposes of hybrid search, the BIG-IP and LTM, (Local Traffic Manager) acts as a reverse proxy providing a secure Internet facing endpoint for connections from the SharePoint Online environment. The deployment of SharePoint behind the BIG-IP can either be configured manually or via the BIG-IP iApp configuration. Once SharePoint 2013 has been successfully deployed it is simple matter of modifying the virtual server’s Client SSL Profile. Figure 2 – SharePoint 2013 On-Premise Virtual Server The SSL Profile is configured to require a client certificate for successful access. Additionally, the certificate must be issued from the specified certificate authority whose CA certificate is installed on the BIG-IP. In the example below, (Figure 3) the client certificate presented must be issued by StartCom, Ltd. Figure 3 – Client SSL Profile with Client Authentication Required As long as the certificate installed and configured in the SharePoint Online Secure Store is valid and issued by the appropriate certificate authority, users in O365 will be able to search for and receive results from the on-premise SharePoint environment. Enhancing Security with iRules Ok, the above configuration provides security to the hybrid connection via client certificates. However, it’s still vulnerable. As previously noted, the certificate used must be issued by a public root certificate authority. While this is good, this does not guarantee that the certificate presented is the actual certificate configured in the SharePoint Online environment. Merely, it just guarantees that the certificate was issued from the same place, (StartCom, Ltd. in our example). Fortunately, there’s an iRule for that! The iRule provided below examines the client certificate provided and ensures that the serial number of the certificate matches the SharePoint Online configured certificate. This ensures that not only is the presented certificate issued from a specific and valid certificate authority, but also ensures that the certificate itself is an exact match. Figure 4 – Granular_Cert_Verification iRule validating the Client Certificate Serial Number 1: when CLIENTSSL_CLIENTCERT { 2: 3: # Check if client provided a cert 4: if {[SSL::cert 0] eq ""}{ 5: 6: # Reset the connection 7: reject 8: 9: } else { 10: 11: #Example Subject SN: 01 ED 51 12: set subject_sn [X509::serial_number [SSL::cert 0]] 13: log "Client Certificate Received: $subject_sn" 14: #Check if the client certificate contains the correct serial number 15: if {$subject_sn contains "0c e4 11"} { 16: #Accept the client cert 17: log "Client Certificate Accepted: $subject_sn" 18: } else { 19: log "No Matching Client Certificate Was Found Using: $subject_sn" 20: reject 21: } 22: } 23: } Figure 5 – Client Certificate Subject Name Figure 6 – iRule Assigned to Virtual Server That’s It! Not too bad huh? Of course this is the easy part. We still need to ensure that we have properly configured both environments, (on-premise and Office 365) to enable the hybrid search functionality. Accomplishing that is a little more tricky, but definitely doable. Be sure to refer to the links below for information from Microsoft on how to set this up. In addition, check out Steve Peschka’s blog posts for information. Additional Links: Hybrid for SharePoint Server 2013 Configure hybrid Search for SharePoint Server 2013 F5 and SharePoint 2013 – Deployment Guidance Architecture Design Recommendation for SharePoint 2013 Hybrid Search Features – Steve Peschka Blog Big-IP and ADFS Part 1 – “Load balancing the ADFS Farm”
