Controlling a Pool Members Ratio and Priority Group with iControl
A Little Background A question came in through the iControl forums about controlling a pool members ratio and priority programmatically. The issue really involves how the API’s use multi-dimensional arrays but I thought it would be a good opportunity to talk about ratio and priority groups for those that don’t understand how they work. In the first part of this article, I’ll talk a little about what pool members are and how their ratio and priorities apply to how traffic is assigned to them in a load balancing setup. The details in this article were based on BIG-IP version 11.1, but the concepts can apply to other previous versions as well. Load Balancing In it’s very basic form, a load balancing setup involves a virtual ip address (referred to as a VIP) that virtualized a set of backend servers. The idea is that if your application gets very popular, you don’t want to have to rely on a single server to handle the traffic. A VIP contains an object called a “pool” which is essentially a collection of servers that it can distribute traffic to. The method of distributing traffic is referred to as a “Load Balancing Method”. You may have heard the term “Round Robin” before. In this method, connections are passed one at a time from server to server. In most cases though, this is not the best method due to characteristics of the application you are serving. Here are a list of the available load balancing methods in BIG-IP version 11.1. Load Balancing Methods in BIG-IP version 11.1 Round Robin: Specifies that the system passes each new connection request to the next server in line, eventually distributing connections evenly across the array of machines being load balanced. This method works well in most configurations, especially if the equipment that you are load balancing is roughly equal in processing speed and memory. Ratio (member): Specifies that the number of connections that each machine receives over time is proportionate to a ratio weight you define for each machine within the pool. Least Connections (member): Specifies that the system passes a new connection to the node that has the least number of current connections in the pool. This method works best in environments where the servers or other equipment you are load balancing have similar capabilities. This is a dynamic load balancing method, distributing connections based on various aspects of real-time server performance analysis, such as the current number of connections per node or the fastest node response time. Observed (member): Specifies that the system ranks nodes based on the number of connections. Nodes that have a better balance of fewest connections receive a greater proportion of the connections. This method differs from Least Connections (member), in that the Least Connections method measures connections only at the moment of load balancing, while the Observed method tracks the number of Layer 4 connections to each node over time and creates a ratio for load balancing. This dynamic load balancing method works well in any environment, but may be particularly useful in environments where node performance varies significantly. Predictive (member): Uses the ranking method used by the Observed (member) methods, except that the system analyzes the trend of the ranking over time, determining whether a node's performance is improving or declining. The nodes in the pool with better performance rankings that are currently improving, rather than declining, receive a higher proportion of the connections. This dynamic load balancing method works well in any environment. Ratio (node): Specifies that the number of connections that each machine receives over time is proportionate to a ratio weight you define for each machine across all pools of which the server is a member. Least Connections (node): Specifies that the system passes a new connection to the node that has the least number of current connections out of all pools of which a node is a member. This method works best in environments where the servers or other equipment you are load balancing have similar capabilities. This is a dynamic load balancing method, distributing connections based on various aspects of real-time server performance analysis, such as the number of current connections per node, or the fastest node response time. Fastest (node): Specifies that the system passes a new connection based on the fastest response of all pools of which a server is a member. This method might be particularly useful in environments where nodes are distributed across different logical networks. Observed (node): Specifies that the system ranks nodes based on the number of connections. Nodes that have a better balance of fewest connections receive a greater proportion of the connections. This method differs from Least Connections (node), in that the Least Connections method measures connections only at the moment of load balancing, while the Observed method tracks the number of Layer 4 connections to each node over time and creates a ratio for load balancing. This dynamic load balancing method works well in any environment, but may be particularly useful in environments where node performance varies significantly. Predictive (node): Uses the ranking method used by the Observed (member) methods, except that the system analyzes the trend of the ranking over time, determining whether a node's performance is improving or declining. The nodes in the pool with better performance rankings that are currently improving, rather than declining, receive a higher proportion of the connections. This dynamic load balancing method works well in any environment. Dynamic Ratio (node) : This method is similar to Ratio (node) mode, except that weights are based on continuous monitoring of the servers and are therefore continually changing. This is a dynamic load balancing method, distributing connections based on various aspects of real-time server performance analysis, such as the number of current connections per node or the fastest node response time. Fastest (application): Passes a new connection based on the fastest response of all currently active nodes in a pool. This method might be particularly useful in environments where nodes are distributed across different logical networks. Least Sessions: Specifies that the system passes a new connection to the node that has the least number of current sessions. This method works best in environments where the servers or other equipment you are load balancing have similar capabilities. This is a dynamic load balancing method, distributing connections based on various aspects of real-time server performance analysis, such as the number of current sessions. Dynamic Ratio (member): This method is similar to Ratio (node) mode, except that weights are based on continuous monitoring of the servers and are therefore continually changing. This is a dynamic load balancing method, distributing connections based on various aspects of real-time server performance analysis, such as the number of current connections per node or the fastest node response time. L3 Address: This method functions in the same way as the Least Connections methods. We are deprecating it, so you should not use it. Weighted Least Connections (member): Specifies that the system uses the value you specify in Connection Limit to establish a proportional algorithm for each pool member. The system bases the load balancing decision on that proportion and the number of current connections to that pool member. For example,member_a has 20 connections and its connection limit is 100, so it is at 20% of capacity. Similarly, member_b has 20 connections and its connection limit is 200, so it is at 10% of capacity. In this case, the system select selects member_b. This algorithm requires all pool members to have a non-zero connection limit specified. Weighted Least Connections (node): Specifies that the system uses the value you specify in the node's Connection Limitand the number of current connections to a node to establish a proportional algorithm. This algorithm requires all nodes used by pool members to have a non-zero connection limit specified. Ratios The ratio is used by the ratio-related load balancing methods to load balance connections. The ratio specifies the ratio weight to assign to the pool member. Valid values range from 1 through 100. The default is 1, which means that each pool member has an equal ratio proportion. So, if you have server1 a with a ratio value of “10” and server2 with a ratio value of “1”, server1 will get served 10 connections for every one that server2 receives. This can be useful when you have different classes of servers with different performance capabilities. Priority Group The priority group is a number that groups pool members together. The default is 0, meaning that the member has no priority. To specify a priority, you must activate priority group usage when you create a new pool or when adding or removing pool members. When activated, the system load balances traffic according to the priority group number assigned to the pool member. The higher the number, the higher the priority, so a member with a priority of 3 has higher priority than a member with a priority of 1. The easiest way to think of priority groups is as if you are creating mini-pools of servers within a single pool. You put members A, B, and C in to priority group 5 and members D, E, and F in priority group 1. Members A, B, and C will be served traffic according to their ratios (assuming you have ratio loadbalancing configured). If all those servers have reached their thresholds, then traffic will be distributed to servers D, E, and F in priority group 1. he default setting for priority group activation is Disabled. Once you enable this setting, you can specify pool member priority when you create a new pool or on a pool member's properties screen. The system treats same-priority pool members as a group. To enable priority group activation in the admin GUI, select Less than from the list, and in the Available Member(s) box, type a number from 0 to 65535 that represents the minimum number of members that must be available in one priority group before the system directs traffic to members in a lower priority group. When a sufficient number of members become available in the higher priority group, the system again directs traffic to the higher priority group. Implementing in Code The two methods to retrieve the priority and ratio values are very similar. They both take two parameters: a list of pools to query, and a 2-D array of members (a list for each pool member passed in). long [] [] get_member_priority( in String [] pool_names, in Common__AddressPort [] [] members ); long [] [] get_member_ratio( in String [] pool_names, in Common__AddressPort [] [] members ); The following PowerShell function (utilizing the iControl PowerShell Library), takes as input a pool and a single member. It then make a call to query the ratio and priority for the specific member and writes it to the console. function Get-PoolMemberDetails() { param( $Pool = $null, $Member = $null ); $AddrPort = Parse-AddressPort $Member; $RatioAofA = (Get-F5.iControl).LocalLBPool.get_member_ratio( @($Pool), @( @($AddrPort) ) ); $PriorityAofA = (Get-F5.iControl).LocalLBPool.get_member_priority( @($Pool), @( @($AddrPort) ) ); $ratio = $RatioAofA[0][0]; $priority = $PriorityAofA[0][0]; "Pool '$Pool' member '$Member' ratio '$ratio' priority '$priority'"; } Setting the values with the set_member_priority and set_member_ratio methods take the same first two parameters as their associated get_* methods, but add a third parameter for the priorities and ratios for the pool members. set_member_priority( in String [] pool_names, in Common::AddressPort [] [] members, in long [] [] priorities ); set_member_ratio( in String [] pool_names, in Common::AddressPort [] [] members, in long [] [] ratios ); The following Powershell function takes as input the Pool and Member with optional values for the Ratio and Priority. If either of those are set, the function will call the appropriate iControl methods to set their values. function Set-PoolMemberDetails() { param( $Pool = $null, $Member = $null, $Ratio = $null, $Priority = $null ); $AddrPort = Parse-AddressPort $Member; if ( $null -ne $Ratio ) { (Get-F5.iControl).LocalLBPool.set_member_ratio( @($Pool), @( @($AddrPort) ), @($Ratio) ); } if ( $null -ne $Priority ) { (Get-F5.iControl).LocalLBPool.set_member_priority( @($Pool), @( @($AddrPort) ), @($Priority) ); } } In case you were wondering how to create the Common::AddressPort structure for the $AddrPort variables in the above examples, here’s a helper function I wrote to allocate the object and fill in it’s properties. function Parse-AddressPort() { param($Value); $tokens = $Value.Split(":"); $r = New-Object iControl.CommonAddressPort; $r.address = $tokens[0]; $r.port = $tokens[1]; $r; } Download The Source The full source for this example can be found in the iControl CodeShare under PowerShell PoolMember Ratio and Priority.Microsoft Powershell with iControl
This component is not supported and we recommend reviewing Joel Newton's Powershell Module for iControlREST Code submission first. From the desk of Joe Pruitt (July 29, 2013) When we shipped DC4, we started looking at Windows PowerShell and how we could build some integration points with our products. The first pass was a set of PowerShell script files that we introduced in the PowerShell Labs section of DevCentral. No soon after we posted them, the requests started pouring in on when we would provide some native PowerShell CmdLets in addition to the function scripts. Well, I spent a little bit of time working some out and whipped out a good first rough draft. I've been holding on to these for a while now but figured they would do better out in the wild then trapped in a folder on my laptop. So, last night I posted an installer for the first release of the iControl CmdLets for PowerShell. Here's a step by step on getting up and running with the new bits. Download and install PowerShell from Microsoft Go to the PowerShell Labs page on DevCentral and select the "Download Now" link. This will download the Cmdlet installer. Run the iControlSnapInSetup.msi installer. This will install the SnapIn into the c:\program files\F5 Networks\iControlSnapIn directory. Start PowerShell from the Windows Start menu. Cd to c:\program files\F5 Networks\iControlSnapIn directory Dot Source the setup script (only once after the install) PS > . .\setupSnapIn.ps1) Load the SnapIn into the Runtime PS > Add-PSSnapIn iControlSnapIn Initialize the iControl connection with the Initialize-F5.iControl CmdLet PS > Initialize-F5.iControl -Hostname bigip_address -Credentials (Get-Credential) Run the Get-F5.iControlCommands CmdLet to list out all the available Cmdlets. PS > Get-F5.iControlCommands Try out some of the CmdLets PS > Get-F5.LTMPool Notes From the Legacy Download: Comment made 08-Jun-2016 by Patrik Jonsson Needed to add .Net 2.0 in add/remove windows features. Then it worked in Windows 10. The installation script should be changed to throw and error if the installutil file does not exist instead of quitting silently. Ken B Comment made 22-Jun-2016 by Ken B The problem I had getting this working was that I had to right-click the downloaded .zip file, properties, and click the "unblock" button on the General tab. Then I had to copy the files from the .zip to a folder under c:\Program Files\f5\icontrol. Then I ran PS As Administrator, then ran .\setupSnapIn.ps1, then I was *finally* able to run the "Add-PSSnapIn iControlSnapIn" command to get things going. Comment made 17-Jan-2017 by Joel Newton You can update the InstallPSSnapin.ps1 script to reference the .NET v4 install utility. Just replace the reference in setupSnapin.ps1 from $env:windir\Microsoft.Net\Framework${platform}\v2.0.50727\installUtil.exe to $env:windir\Microsoft.Net\Framework${platform}\v4.0.30319\installUtil.exe I don't believe there are any plans to replace the snapin with a module. My recommendation would be to use the REST API if possible. Comment made 05-Jul-2017 by Patrik Jonsson You can also use Joel's module: Powershell Module for the F5 LTM REST API Downloads: v11.00.00 Released August 10, 2013 v11.04.01 Released December 02, 2013 v11.05.00 Released February 18, 2014 v11.06.00 Released August 28, 2014 v12.01.00 Released May 09, 2016 v13.00.00 Released March 21, 2017 v13.01.00 Released November 11, 2017Getting Started with iControl: Working with Statistics
In the previous article in the Getting Started with iControl series, we threw the lion's share of languages at you for both iControl portals, exposing you to many of the possible attack angles available for programmatically interacting with the BIG-IP. In this article, we're going to scale back a bit, focusing a few rest-based examples on how to gather statistics. Note: This article published by Jason Rahm but co-authored with Joe Pruitt. Statistics on BIG-IP As much as you can do with iControl, it isn’t the only methodology for gathering statistics. In fact, I’d argue that there are at least two better ways to get at them, besides the on-box rrd graphs. SNMP - Yes, it’s old school. But it has persisted as long as it has for a reason. It’s broad and lightweight. It does, however, require client-side software to do something with all that data. AVR - By provisioning the AVR module on BIG-IP, you can configure an expansive amount of metrics to measure. Combined with syslog to send the data off-box, this push approach is also pretty lightweight, though like with snmp, having a tool to consume those stats is helpful. Ken Bocchinoreleased an analytics iApp that combines AVR and its data output with Splunk and its indexing ability to deliver quite a solution. But maybe you don’t have those configured, or available, and you just need to do some lab testing and would like to consume those stats locally as you generate your test traffic. In these code samples, we’ll take a look at virtual server and pool stats. Pool Stats At the pool level, several stats are available for consumption, primarily around bandwidth utilization and connection counts. You can also get state information as well, though I'd argue that's not really a statistic, unless I suppose you are counting how often the state is available. With snmp, the stats come in high/low form, so you have to do some bit shifting in order to arrive at the actual value. Not so with with iControl. The value comes as is. In each of these samples below, you'll see two functions. The first is the list function, where when you run the script you will get a list of pools so you can then specify which pool you want stats for. The second function will then take that pool as an argument and return the statistics. Node.js Powershell Python Node.Js Pool Stats //-------------------------------------------------------- function getPoolList(name) { //-------------------------------------------------------- var uri = "/mgmt/tm/ltm/pool"; handleVERB("GET", uri, null, function(json) { //console.log(json); var obj = JSON.parse(json); var items = obj.items; console.log("POOLS"); console.log("-----------"); for(var i=0; i<items.length; i++) { var fullPath = items[i].fullPath; console.log(" " + fullPath); } }); } //-------------------------------------------------------- function getPoolStats(name) { //-------------------------------------------------------- var uri = "/mgmt/tm/ltm/pool/" + name + "/stats"; handleVERB("GET", uri, null, function(json) { //console.log(json); var obj = JSON.parse(json); var selfLink = obj.selfLink; var entries = obj.entries; for(var n in entries) { console.log("--------------------------------------"); console.log("NAME : " + getStatsDescription(entries[n].nestedStats, "tmName")); console.log("--------------------------------------"); console.log("AVAILABILITY STATE : " + getStatsDescription(entries[n].nestedStats, "status.availabilityState")); console.log("ENABLED STATE : " + getStatsDescription(entries[n].nestedStats, "status.enabledState")); console.log("REASON : " + getStatsDescription(entries[n].nestedStats, "status.statusReason")); console.log("SERVER BITS IN : " + getStatsValue(entries[n].nestedStats, "serverside.bitsIn")); console.log("SERVER BITS OUT : " + getStatsValue(entries[n].nestedStats, "serverside.bitsOut")); console.log("SERVER PACKETS IN : " + getStatsValue(entries[n].nestedStats, "serverside.pktsIn")); console.log("SERVER PACKETS OUT : " + getStatsValue(entries[n].nestedStats, "serverside.pktsOut")); console.log("CURRENT CONNECTIONS : " + getStatsValue(entries[n].nestedStats, "serverside.curConns")); console.log("MAXIMUM CONNECTIONS : " + getStatsValue(entries[n].nestedStats, "serverside.maxConns")); console.log("TOTAL CONNECTIONS : " + getStatsValue(entries[n].nestedStats, "serverside.totConns")); console.log("TOTAL REQUESTS : " + getStatsValue(entries[n].nestedStats, "totRequests")); } }); } Powershell Pool Stats #---------------------------------------------------------------------------- function Get-PoolList() # # Description: # This function returns the list of pools # # Parameters: # None #---------------------------------------------------------------------------- { $uri = "/mgmt/tm/ltm/pool"; $link = "https://$Bigip$uri"; $headers = @{}; $headers.Add("ServerHost", $Bigip); $secpasswd = ConvertTo-SecureString $Pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($User, $secpasswd) $obj = Invoke-RestMethod -Method GET -Headers $headers -Uri $link -Credential $mycreds $items = $obj.items; Write-Host "POOL NAMES"; Write-Host "----------"; for($i=0; $i -lt $items.length; $i++) { $name = $items[$i].fullPath; Write-Host " $name"; } } #---------------------------------------------------------------------------- function Get-PoolStats() # # Description: # This function returns the statistics for a pool # # Parameters: # Name - The name of the pool #---------------------------------------------------------------------------- { $uri = "/mgmt/tm/ltm/pool/${Name}/stats"; $link = "https://$Bigip$uri"; $headers = @{}; $headers.Add("ServerHost", $Bigip); $secpasswd = ConvertTo-SecureString $Pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($User, $secpasswd) $obj = Invoke-RestMethod -Method GET -Headers $headers -Uri $link -Credential $mycreds $entries = $obj.entries; $names = $entries | get-member -MemberType NoteProperty | select -ExpandProperty Name; $desc = $entries | Select -ExpandProperty $names $nestedStats = $desc.nestedStats; Write-Host ("--------------------------------------"); Write-Host ("NAME : $(Get-StatsDescription $nestedStats 'tmName')"); Write-Host ("--------------------------------------"); Write-Host ("AVAILABILITY STATE : $(Get-StatsDescription $nestedStats 'status.availabilityState')"); Write-Host ("ENABLED STATE : $(Get-StatsDescription $nestedStats 'status.enabledState')"); Write-Host ("REASON : $(Get-StatsDescription $nestedStats 'status.statusReason')"); Write-Host ("SERVER BITS IN : $(Get-StatsValue $nestedStats 'serverside.bitsIn')"); Write-Host ("SERVER BITS OUT : $(Get-StatsValue $nestedStats 'serverside.bitsOut')"); Write-Host ("SERVER PACKETS IN : $(Get-StatsValue $nestedStats 'serverside.pktsIn')"); Write-Host ("SERVER PACKETS OUT : $(Get-StatsValue $nestedStats 'serverside.pktsOut')"); Write-Host ("CURRENT CONNECTIONS : $(Get-StatsValue $nestedStats 'serverside.curConns')"); Write-Host ("MAXIMUM CONNECTIONS : $(Get-StatsValue $nestedStats 'serverside.maxConns')"); Write-Host ("TOTAL CONNECTIONS : $(Get-StatsValue $nestedStats 'serverside.totConns')"); Write-Host ("TOTAL REQUESTS : $(Get-StatsValue $nestedStats 'totRequests')"); } Python Pool Stats def get_pool_list(bigip, url): try: pools = bigip.get("%s/ltm/pool" % url).json() print "POOL LIST" print " ---------" for pool in pools['items']: print " /%s/%s" % (pool['partition'], pool['name']) except Exception, e: print e def get_pool_stats(bigip, url, pool): try: pool_stats = bigip.get("%s/ltm/pool/%s/stats" % (url, pool)).json() selflink = "https://localhost/mgmt/tm/ltm/pool/%s/~Common~%s/stats" % (pool, pool) nested_stats = pool_stats['entries'][selflink]['nestedStats']['entries'] print '' print ' --------------------------------------' print ' NAME : %s' % nested_stats['tmName']['description'] print ' --------------------------------------' print ' AVAILABILITY STATE : %s' % nested_stats['status.availabilityState']['description'] print ' ENABLED STATE : %s' % nested_stats['status.enabledState']['description'] print ' REASON : %s' % nested_stats['status.statusReason']['description'] print ' SERVER BITS IN : %s' % nested_stats['serverside.bitsIn']['value'] print ' SERVER BITS OUT : %s' % nested_stats['serverside.bitsOut']['value'] print ' SERVER PACKETS IN : %s' % nested_stats['serverside.pktsIn']['value'] print ' SERVER PACKETS OUT : %s' % nested_stats['serverside.pktsOut']['value'] print ' CURRENT CONNECTIONS : %s' % nested_stats['serverside.curConns']['value'] print ' MAXIMUM CONNECTIONS : %s' % nested_stats['serverside.maxConns']['value'] print ' TOTAL CONNECTIONS : %s' % nested_stats['serverside.totConns']['value'] print ' TOTAL REQUESTS : %s' % nested_stats['totRequests']['value'] Virtual Server Stats The virtual server stats are very similar in nature to the pool stats, though you are getting an aggregate view of traffic to the virtual server, whereas there might be several pools of traffic being serviced by that single virtual server. In any event, if you compare the code below to the code from the pool functions, they are nearly functionally identical. The only differences should be discernable: the method calls themselves and then the object properties that are different (clientside/serverside for example.) Node.js Powershell Python Node.Js Virtual Server Stats //-------------------------------------------------------- function getVirtualList(name) { //-------------------------------------------------------- var uri = "/mgmt/tm/ltm/virtual"; handleVERB("GET", uri, null, function(json) { //console.log(json); var obj = JSON.parse(json); var items = obj.items; console.log("VIRTUALS"); console.log("-----------"); for(var i=0; i<items.length; i++) { var fullPath = items[i].fullPath; console.log(" " + fullPath); } }); } //-------------------------------------------------------- function getVirtualStats(name) { //-------------------------------------------------------- var uri = "/mgmt/tm/ltm/virtual/" + name + "/stats"; handleVERB("GET", uri, null, function(json) { //console.log(json); var obj = JSON.parse(json); var selfLink = obj.selfLink; var entries = obj.entries; for(var n in entries) { console.log("--------------------------------------"); console.log("NAME : " + getStatsDescription(entries[n].nestedStats, "tmName")); console.log("--------------------------------------"); console.log("DESTINATION : " + getStatsDescription(entries[n].nestedStats, "destination")); console.log("AVAILABILITY STATE : " + getStatsDescription(entries[n].nestedStats, "status.availabilityState")); console.log("ENABLED STATE : " + getStatsDescription(entries[n].nestedStats, "status.enabledState")); console.log("REASON : " + getStatsDescription(entries[n].nestedStats, "status.statusReason")); console.log("CLIENT BITS IN : " + getStatsValue(entries[n].nestedStats, "clientside.bitsIn")); console.log("CLIENT BITS OUT : " + getStatsValue(entries[n].nestedStats, "clientside.bitsOut")); console.log("CLIENT PACKETS IN : " + getStatsValue(entries[n].nestedStats, "clientside.pktsIn")); console.log("CLIENT PACKETS OUT : " + getStatsValue(entries[n].nestedStats, "clientside.pktsOut")); console.log("CURRENT CONNECTIONS : " + getStatsValue(entries[n].nestedStats, "clientside.curConns")); console.log("MAXIMUM CONNECTIONS : " + getStatsValue(entries[n].nestedStats, "clientside.maxConns")); console.log("TOTAL CONNECTIONS : " + getStatsValue(entries[n].nestedStats, "clientside.totConns")); console.log("TOTAL REQUESTS : " + getStatsValue(entries[n].nestedStats, "totRequests")); } }); } Powershell Virtual Server Stats #---------------------------------------------------------------------------- function Get-VirtualList() # # Description: # This function lists all virtual servers. # # Parameters: # None #---------------------------------------------------------------------------- { $uri = "/mgmt/tm/ltm/virtual"; $link = "https://$Bigip$uri"; $headers = @{}; $headers.Add("ServerHost", $Bigip); $secpasswd = ConvertTo-SecureString $Pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($User, $secpasswd) $obj = Invoke-RestMethod -Method GET -Headers $headers -Uri $link -Credential $mycreds $items = $obj.items; Write-Host "POOL NAMES"; Write-Host "----------"; for($i=0; $i -lt $items.length; $i++) { $name = $items[$i].fullPath; Write-Host " $name"; } } #---------------------------------------------------------------------------- function Get-VirtualStats() # # Description: # This function returns the statistics for a virtual server # # Parameters: # Name - The name of the virtual server #---------------------------------------------------------------------------- { param( [string]$Name ); $uri = "/mgmt/tm/ltm/virtual/${Name}/stats"; $link = "https://$Bigip$uri"; $headers = @{}; $headers.Add("ServerHost", $Bigip); $secpasswd = ConvertTo-SecureString $Pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($User, $secpasswd) $obj = Invoke-RestMethod -Method GET -Headers $headers -Uri $link -Credential $mycreds $entries = $obj.entries; $names = $entries | get-member -MemberType NoteProperty | select -ExpandProperty Name; $desc = $entries | Select -ExpandProperty $names $nestedStats = $desc.nestedStats; Write-Host ("--------------------------------------"); Write-Host ("NAME : $(Get-StatsDescription $nestedStats 'tmName')"); Write-Host ("--------------------------------------"); Write-Host ("DESTINATION : $(Get-StatsDescription $nestedStats 'destination')"); Write-Host ("AVAILABILITY STATE : $(Get-StatsDescription $nestedStats 'status.availabilityState')"); Write-Host ("ENABLED STATE : $(Get-StatsDescription $nestedStats 'status.enabledState')"); Write-Host ("REASON : $(Get-StatsDescription $nestedStats 'status.statusReason')"); Write-Host ("CLIENT BITS IN : $(Get-StatsValue $nestedStats 'clientside.bitsIn')"); Write-Host ("CLIENT BITS OUT : $(Get-StatsValue $nestedStats 'clientside.bitsOut')"); Write-Host ("CLIENT PACKETS IN : $(Get-StatsValue $nestedStats 'clientside.pktsIn')"); Write-Host ("CLIENT PACKETS OUT : $(Get-StatsValue $nestedStats 'clientside.pktsOut')"); Write-Host ("CURRENT CONNECTIONS : $(Get-StatsValue $nestedStats 'clientside.curConns')"); Write-Host ("MAXIMUM CONNECTIONS : $(Get-StatsValue $nestedStats 'clientside.maxConns')"); Write-Host ("TOTAL CONNECTIONS : $(Get-StatsValue $nestedStats 'clientside.totConns')"); Write-Host ("TOTAL REQUESTS : $(Get-StatsValue $nestedStats 'totRequests')"); } Python Virtual Server Stats def get_virtual_list(bigip, url): try: vips = bigip.get("%s/ltm/virtual" % url).json() print "VIRTUAL SERVER LIST" print " -------------------" for vip in vips['items']: print " /%s/%s" % (vip['partition'], vip['name']) except Exception, e: print e def get_virtual_stats(bigip, url, vip): try: vip_stats = bigip.get("%s/ltm/virtual/%s/stats" % (url, vip)).json() selflink = "https://localhost/mgmt/tm/ltm/virtual/%s/~Common~%s/stats" % (vip, vip) nested_stats = vip_stats['entries'][selflink]['nestedStats']['entries'] print '' print ' --------------------------------------' print ' NAME : %s' % nested_stats['tmName']['description'] print ' --------------------------------------' print ' AVAILABILITY STATE : %s' % nested_stats['status.availabilityState']['description'] print ' ENABLED STATE : %s' % nested_stats['status.enabledState']['description'] print ' REASON : %s' % nested_stats['status.statusReason']['description'] print ' CLIENT BITS IN : %s' % nested_stats['clientside.bitsIn']['value'] print ' CLIENT BITS OUT : %s' % nested_stats['clientside.bitsOut']['value'] print ' CLIENT PACKETS IN : %s' % nested_stats['clientside.pktsIn']['value'] print ' CLIENT PACKETS OUT : %s' % nested_stats['clientside.pktsOut']['value'] print ' CURRENT CONNECTIONS : %s' % nested_stats['clientside.curConns']['value'] print ' MAXIMUM CONNECTIONS : %s' % nested_stats['clientside.maxConns']['value'] print ' TOTAL CONNECTIONS : %s' % nested_stats['clientside.totConns']['value'] print ' TOTAL REQUESTS : %s' % nested_stats['totRequests']['value'] Going Further So now that you have scripts that will pull stats from the pools and virtual servers, what might you do to extend these? As an exercise in skill development, I’d suggest trying these tasks below. Note that there are dozens of existing articles on iControl that you can glean details from that will help you in your quest. Given that you can pull back a list of pools or virtual servers by not providing one specifically, iterate through the appropriate list and print stats for all of them instead of just one. Extend that even further, building a table of virtual servers and their respective stats These stats are a point in time, and not terribly useful as a single entity. Create a timer where the stats are pulled back every 10s or so and take a delta to display. Extend that even further by narrowing down to maybe the bits in and out, and graph them over time. Resources All the scripts for this article are available in the codeshare under "Getting Started with iControl Code Samples."Getting Started with iControl: Working with Configuration Objects
In the previous articles in the Getting Started with iControl series, we covered the history of iControl, taxonomy, and introduced the languages most common to our community. In this article, we'll start looking at some code samples. With each task we cover, we'll show an example for a few languages for both the soap and rest interface. The sheer amount of space in this article required to show full samples of each language for both soap and rest would be overwhelming in line, so for the sample code, we have pulled out the appropriate sections of the code for you to study. Understand that based on the libraries used, if at all, some of the work may be abstracted. Full scripts for each language and portal preference are linked at the bottom of this article. If you click each of the iControl REST and iControl SOAP tabs, you'll see a language listing for the sample code provided. Joe Pruitt is the super hero providing everything not python here, so give him a shout out next time you see him in these parts. Note: This article published by Jason Rahm but co-authored with Joe Pruitt. With all the disclaimers out of the way, let's get down to business! The first thing you have to do to talk to the BIG-IP is know where to connect. Some prerequisites: BIG-IP Hostname Username Password Portal URL rest: https://host/mgmt/<tm> (most calls are for /tm, though there are others) soap: https://host/iControl/iControlPortal.cgi There are dozens of articles and codeshare samples on DevCentral on creating/modifying objects both in rest and soap, so we're not going to cover all the details ad nauseam here. But we do want to introduce the basic concepts of retrieving, creating, and modifying a couple simple objects in this article. The first one we'll tackle is one of the most important building blocks of the BIG-IP configuration: the pool. Working with the Pool Object The pool has many attributes that can be set, but not all are required. In this exercise however, we're going to simply demonstrate the creation of a pool and a pool listing. The pool object can be created via rest with a simple name, and with soap with the name, the lb method, and the members attribute (it can be empty, but the attribute needs to be present.) iControl SOAP Java Perl Powershell Python iControl REST Node.js Perl Powershell Python Java SOAP Example public void displayAllPools() throws Exception { String [] pool_list = m_interfaces.getLocalLBPool().get_list(); System.out.println("Pools\n"); System.out.println("-----------\n"); for(int i=0; i<pool_list.length; i++) { System.out.println(pool_list[i]); } } public void createPool(String poolname, String partition) throws Exception { String [] pool_list = new String[] {"/" + partition + "/" + poolname}; iControl.LocalLBLBMethod [] lb_methods = new iControl.LocalLBLBMethod[] { iControl.LocalLBLBMethod.LB_METHOD_ROUND_ROBIN }; iControl.CommonIPPortDefinition[][] membersAofA = new iControl.CommonIPPortDefinition[1][]; membersAofA[0] = new iControl.CommonIPPortDefinition[1]; membersAofA[0][0] = new iControl.CommonIPPortDefinition(); membersAofA[0][0].setAddress("10.10.10.10"); membersAofA[0][0].setPort(80); m_interfaces.getLocalLBPool().create( pool_list, lb_methods, membersAofA ); System.out.println("Pool " + poolname + " created in partition " + partition); } Perl SOAP Example #---------------------------------------------------------------------------- # getPoolList #---------------------------------------------------------------------------- sub getPoolList() { $soapResponse = $Pool->get_list(); &checkResponse($soapResponse); my @pool_list = @{$soapResponse->result}; print "POOL LIST\n"; print "---------\n"; foreach $pool (@pool_list) { print " ${pool}\n"; } } #---------------------------------------------------------------------------- # createPool() #---------------------------------------------------------------------------- sub createPool() { my ($partition, $pool) = @_; print "CREATING POOL $pool\n"; my @pool_names = [$pool]; my @lb_methods = ["LB_METHOD_ROUND_ROBIN"]; $member = { address => "10.10.10.10", port => 80 }; # memberA is the 1st dimension of the array, we need one for each pool push @memberA, $member; # memberAofA is the 2nd dimension. push pool members for each pool here. push @memberAofA, [@memberA]; $soapResponse = $Pool->create( SOAP::Data->name( pool_names => ["/$partition/$pool"]), SOAP::Data->name( lb_methods => ["LB_METHOD_ROUND_ROBIN"]), SOAP::Data->name(members => [@memberAofA]) ); &checkResponse($soapResponse); print "POOL ${pool} created in partition ${partition}...\n"; } Powershell SOAP Example #---------------------------------------------------------------------------- function Get-PoolList() #---------------------------------------------------------------------------- { $pool_list = $(Get-F5.iControl).LocalLBPool.get_list(); Write-Host "POOL LIST"; Write-Host "---------"; foreach($pool in $pool_list) { Write-Host " $pool"; } } #---------------------------------------------------------------------------- function Create-Pool() # # Description: # This function creates a new pool if the given pool name doesn't # already exist. # # Parameters: # Name - The Name of the pool you wish to create. # MemberList - A string list of pool member addresses. # MemberPort - The port the pool members will be configured with. #---------------------------------------------------------------------------- { param( [string]$Name, [string[]]$MemberList, [int]$MemberPort ); $IPPortDefList = New-Object -TypeName iControl.CommonIPPortDefinition[] $MemberList.Length; for($i=0; $i-lt$MemberList.Length; $i++) { $IPPortDefList[$i] = New-Object -TypeName iControl.CommonIPPortDefinition; $IPPortDefList[$i].address = $MemberList[$i]; $IPPortDefList[$i].port = $MemberPort; } Write-Host "Creating Pool $Name"; $(Get-F5.iControl).LocalLBPool.create( (,$Name), (,"LB_METHOD_ROUND_ROBIN"), (,$IPPortDefList) ); Write-Host "Pool '$Name' Successfully Created"; } Python SOAP Example def get_pool_list(bigip): try: poollist = bigip.LocalLB.Pool.get_list() print "POOL LIST" print " ---------" for pool in poollist: print " %s" % pool except Exception, e: print e def create_pool(bigip, pool): try: bigip.LocalLB.Pool.create_v2([pool], ['LB_METHOD_ROUND_ROBIN'], [[]]) except Exception, e: print e Node.js REST Example //-------------------------------------------------------- function handleVERB(verb, resource, body, callback) { //-------------------------------------------------------- getAuthToken( function(token) { httpRequest(verb, resource, body, null, null, token, function(json) { callback(json); }); }); } //-------------------------------------------------------- function handleGetPoolList(callback) { //-------------------------------------------------------- var uri = "/mgmt/tm/ltm/pool"; handleVERB("GET", uri, null, function(json) { callback(json); }); } //-------------------------------------------------------- function handleCreatePool(pool, partition, callback) { //-------------------------------------------------------- var uri = "/mgmt/tm/ltm/pool"; var body = '{"name":"' + pool + '", "partition":"' + partition + '"}'; console.log("BODY: " + body); handleVERB("POST", uri, body, function(json) { callback(json); }); } Perl REST Example #--------------------------------------------------- sub getPoolList() { #--------------------------------------------------- $resp_json = &handleGET("/mgmt/tm/ltm/pool"); $resp = decode_json($resp_json); @items = @{$resp->{"items"}}; print "POOL NAMES\n"; print "----------\n"; foreach $item (@items) { $fullPath = $item->{"fullPath"}; print " $fullPath\n"; print Dumper($item); } } #--------------------------------------------------- sub handleGET() { #--------------------------------------------------- my ($resource) = @_; $url = &buildURL($resource); $resp = &getHttpRequest($url); return $resp; } #--------------------------------------------------- sub createPool() { #--------------------------------------------------- my ($pool, $partition) = @_; my $poolObj; $poolObj->{"name"} = $pool; $poolObj->{"partition"} = $partition; my $json = encode_json($poolObj); print "JSON: $json\n"; exit(); $resp = &handlePOST("/mgmt/tm/ltm/pool", $json); print Dumper($resp); } #--------------------------------------------------- sub handlePOST() { #--------------------------------------------------- my ($resource, $body) = @_; if ( $body eq "" ) { &usage(); } $url = &buildURL($resource); $resp = &postHttpRequest($url, $body); return $resp; } Powershell REST Example #---------------------------------------------------------------------------- function Get-PoolList() #---------------------------------------------------------------------------- { $uri = "/mgmt/tm/ltm/pool"; $link = "https://$Bigip$uri"; $headers = @{}; $headers.Add("ServerHost", $Bigip); $secpasswd = ConvertTo-SecureString $Pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($User, $secpasswd) $obj = Invoke-RestMethod -Method GET -Headers $headers -Uri $link -Credential $mycreds $items = $obj.items; Write-Host "POOL NAMES"; Write-Host "----------"; for($i=0; $i -lt $items.length; $i++) { $name = $items[$i].fullPath; Write-Host " $name"; } } #---------------------------------------------------------------------------- function Create-Pool() # # Description: # This function creates a new pool if the given pool name doesn't # already exist. # # Parameters: # Name - The Name of the pool you wish to create. # Partition - The name of the partition to place the pool in. #---------------------------------------------------------------------------- { param( [string]$Name, [string]$Partition ); $uri = "/mgmt/tm/ltm/pool"; $link = "https://$Bigip$uri"; $headers = @{}; $headers.Add("ServerHost", $Bigip); $headers.Add("Content-Type", "application/json"); $obj = @{ name=$Name partition=$Partition }; $body = $obj | ConvertTo-Json $secpasswd = ConvertTo-SecureString $Pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($User, $secpasswd) $obj = Invoke-RestMethod -Method POST -Uri $link -Headers $headers -Credential $mycreds -Body $body; Write-Host "Pool ${Name} created in partition ${Partition}" } Python REST Example def get_pool_list(bigip, url): try: pools = bigip.get("%s/ltm/pool" % url).json() print "POOL LIST" print " ---------" for pool in pools['items']: print " /%s/%s" % (pool['partition'], pool['name']) except Exception, e: print e def create_pool(bigip, url, pool, part=None): try: payload = {} payload['name'] = pool if part is not None: payload['partition'] = part pool_config = bigip.post("%s/ltm/pool" % url, json.dumps(payload)).json() print pool_config except Exception, e: print e Working with the Data-Group Object Data-groups are interesting objects with which to work, primarily because you can have internal data-groups, which are wholly contained within bigip.conf, and external data-groups, which has it's definition in bigip.conf but the records are kept in an external file. Also interesting is that this particular object has different attributes via soap than it does in rest. In soap, all the records can be individually managed, so you can add, find, and delete on a record by record basis. Not so with rest. In rest we have the concepts of collections and a subcollections. A collection is like a list of pools. A sub-collection would be the members of a pool. Ideally, a data-group itself would be part of a collection, and it's records would be a subcollection, but that is not the case today. This has major implications if you want to update a data-group. Say you have a data-group with 10,000 records. You want add a record, so you issue an HTTP PUT method with your payload of one record. Instead of the desired outcome of a modified data-group with 10,001 records, you now have a much smaller data-group of exactly one record. Not good! So make sure if you are using the rest interface with data-groups, you store all the existing records, make all your adds/changes/deletes, then PUT the entire record collection. But where the soap interface has the upper hand with being able to update individual records, rest punches back with the ability to list out all data-group at once, whereas with soap, you have to query each type of data-group among string, integer, and address. So there are some pros and cons to both approaches to weigh as you dig in. iControl SOAP Java Perl Powershell Python iControl REST Node.js Perl Powershell Python Java SOAP Example public void createDataGroup(String datagroup) throws Exception { // Create String Class iControl.LocalLBClassStringClass [] StringClassA = new iControl.LocalLBClassStringClass[1]; StringClassA[0] = new iControl.LocalLBClassStringClass(); StringClassA[0].setName(datagroup); StringClassA[0].setMembers(new String [] { "a", "b", "c" }); m_interfaces.getLocalLBClass().create_string_class(StringClassA); // Set Values String [][] valuesAofA = new String[1][]; valuesAofA[0] = new String[] { "data 1", "data 2", "data 3" }; m_interfaces.getLocalLBClass().set_string_class_member_data_value( StringClassA, valuesAofA ); getDataGroup(datagroup); } public void removeFromDataGroup(String datagroup) throws Exception { String [] names = new String[] {"c"}; iControl.LocalLBClassStringClass [] StringClassA = new iControl.LocalLBClassStringClass[1]; StringClassA[0] = new iControl.LocalLBClassStringClass(); StringClassA[0].setName(datagroup); StringClassA[0].setMembers(new String [] { "c" }); m_interfaces.getLocalLBClass().delete_string_class_member(StringClassA); getDataGroup(datagroup); } //-------------------------------------------------------------------------- // //-------------------------------------------------------------------------- public void addToDataGroup(String datagroup) throws Exception { // Create String Class iControl.LocalLBClassStringClass [] StringClassA = new iControl.LocalLBClassStringClass[1]; StringClassA[0] = new iControl.LocalLBClassStringClass(); StringClassA[0].setName(datagroup); StringClassA[0].setMembers(new String [] { "d", "e" }); m_interfaces.getLocalLBClass().add_string_class_member(StringClassA); // Set Values String [][] valuesAofA = new String[1][]; valuesAofA[0] = new String[] { "data 4", "data 5" }; m_interfaces.getLocalLBClass().set_string_class_member_data_value( StringClassA, valuesAofA ); getDataGroup(datagroup); } Perl SOAP Example #---------------------------------------------------------------------------- sub createDataGroup() #---------------------------------------------------------------------------- { my ($datagroup) = @_; my @names = ("a", "b", "c"); my $StringClass = { name => $datagroup, members => [@names] }; # Create Data group with names $soapResponse = $Class->create_string_class( SOAP::Data->name(classes => [$StringClass]) ); &checkResponse($soapResponse); # Set values # Build Values 2-D Array for values parameter my @valuesA = ("data 1", "data 2", "data 3"); my @valuesAofA; push @valuesAofA, [@valuesA]; $soapResponse = $Class->set_string_class_member_data_value ( SOAP::Data->name(class_members => [$StringClass]), SOAP::Data->name(values => [@valuesAofA]) ); &checkResponse($soapResponse); &getDataGroup($datagroup); } #---------------------------------------------------------------------------- sub removeFromDataGroup() #---------------------------------------------------------------------------- { my ($datagroup) = @_; my @names = ("c"); my $StringClass = { name => $datagroup, members => [@names] }; # Create Data group with names $soapResponse = $Class->delete_string_class_member( SOAP::Data->name(class_members => [$StringClass]) ); &checkResponse($soapResponse); &getDataGroup($datagroup); } #---------------------------------------------------------------------------- sub addToDataGroup() #---------------------------------------------------------------------------- { my ($datagroup) = @_; my @names = ("d", "e"); my $StringClass = { name => $datagroup, members => [@names] }; # Create Data group with names $soapResponse = $Class->add_string_class_member( SOAP::Data->name(class_members => [$StringClass]) ); &checkResponse($soapResponse); # Set values # Build Values 2-D Array for values parameter my @valuesA = ("data 4", "data 5"); my @valuesAofA; push @valuesAofA, [@valuesA]; $soapResponse = $Class->set_string_class_member_data_value ( SOAP::Data->name(class_members => [$StringClass]), SOAP::Data->name(values => [@valuesAofA]) ); &checkResponse($soapResponse); &getDataGroup($datagroup); } Powershell SOAP Example #------------------------------------------------------------------------- function Create-DataGroup() #------------------------------------------------------------------------- { param( [string]$Name ); $StringClassA = New-Object -TypeName iControl.LocalLBClassStringClass[] 1; $StringClassA[0] = New-Object -TypeName iControl.LocalLBClassStringClass; $StringClassA[0].name = $Name; $StringClassA[0].members = ("a", "b", "c"); $(Get-F5.iControl).LocalLBClass.create_string_class( $StringClassA ); $DataValueA = ("data 1", "data 2", "data 3"); $DataValuesAofA = $(Get-F5.iControl).LocalLBClass.set_string_class_member_data_value( $StringClassA, (, $DataValueA) ) Get-DataGroup -Name $Name; } #------------------------------------------------------------------------- function RemoveFrom-DataGroup() #------------------------------------------------------------------------- { param( [string]$Name ); $StringClassA = New-Object -TypeName iControl.LocalLBClassStringClass[] 1; $StringClassA[0] = New-Object -TypeName iControl.LocalLBClassStringClass; $StringClassA[0].name = $Name; $StringClassA[0].members = ("c"); $(Get-F5.iControl).LocalLBClass.delete_string_class_member( $StringClassA ); Get-DataGroup -Name $Name; } #------------------------------------------------------------------------- function AddTo-DataGroup() #------------------------------------------------------------------------- { param( [string]$Name ); $StringClassA = New-Object -TypeName iControl.LocalLBClassStringClass[] 1; $StringClassA[0] = New-Object -TypeName iControl.LocalLBClassStringClass; $StringClassA[0].name = $Name; $StringClassA[0].members = ("d", "e"); $(Get-F5.iControl).LocalLBClass.add_string_class_member( $StringClassA ); $DataValueA = ("data 4", "data 5"); $DataValuesAofA = $(Get-F5.iControl).LocalLBClass.set_string_class_member_data_value( $StringClassA, (, $DataValueA) ) Get-DataGroup -Name $Name; } Python SOAP Example def get_dg_list(bigip): try: dg_str_list = bigip.LocalLB.Class.get_string_class_list() dg_str_names = bigip.LocalLB.Class.get_string_class(dg_str_list) for dg in dg_str_names: print dg print ' Data Group: %s' % dg['name'] for x in dg['members']: print ' %s' % x except Exception, e: print e def extend_dg(bigip, dgname, keys, values): try: bigip.LocalLB.Class.add_string_class_member([{'name': dgname, 'members': keys}]) bigip.LocalLB.Class.set_string_class_member_data_value([{'name': dgname, 'members': keys}], [[values]]) except Exception, e: print e def contract_dg(bigip, dgname, keys): try: bigip.LocalLB.Class.delete_string_class_member([{'name': dgname, 'members': keys}]) except Exception, e: print e def create_dg(bigip, dgname, keys, values): try: bigip.LocalLB.Class.create_string_class([{'name': dgname, 'members': keys}]) bigip.LocalLB.Class.set_string_class_member_data_value([{'name': dgname, 'members': keys}], [[values]]) except Exception, e: print e Node.js REST Example //-------------------------------------------------------- function createDataGroup(datagroup) { //-------------------------------------------------------- datagroup = datagroup.replace(/\//g, "~"); var uri = "/mgmt/tm/ltm/data-group/internal"; var dgObj = {}; dgObj.name = datagroup; dgObj.type = "string"; dgObj.records = [ {name: "a", data: "data 1"}, {name: "b", data: "data 2"}, {name: "c", data: "data 3"}, ]; var body = JSON.stringify(dgObj); handleVERB("POST", uri, body, function(json) { console.log(json); }); } //-------------------------------------------------------- function removeFromDataGroup(datagroup) { //-------------------------------------------------------- dg_uri = datagroup.replace(/\//g, "~"); var uri = "/mgmt/tm/ltm/data-group/internal/" + dg_uri; var dgObj = {}; dgObj.name = datagroup; dgObj.records = [ {name: "a", data: "data 1"}, {name: "b", data: "data 2"} ]; var body = JSON.stringify(dgObj); handleVERB("PATCH", uri, body, function(json) { console.log(json); }); } //-------------------------------------------------------- function addToDataGroup(datagroup) { //-------------------------------------------------------- dg_uri = datagroup.replace(/\//g, "~"); var uri = "/mgmt/tm/ltm/data-group/internal/" + dg_uri; var dgObj = {}; dgObj.name = datagroup; dgObj.records = [ {name: "a", data: "data 1"}, {name: "b", data: "data 2"}, {name: "d", data: "data 4"}, {name: "e", data: "data 5"} ]; var body = JSON.stringify(dgObj); handleVERB("PATCH", uri, body, function(json) { console.log(json); }); } Perl REST Example #---------------------------------------------------------------------------- sub createDataGroup() #---------------------------------------------------------------------------- { my ($datagroup) = @_; my @names = ("a", "b", "c"); my $StringClass = { name => $datagroup, members => [@names] }; # Create Data group with names $soapResponse = $Class->create_string_class( SOAP::Data->name(classes => [$StringClass]) ); &checkResponse($soapResponse); # Set values # Build Values 2-D Array for values parameter my @valuesA = ("data 1", "data 2", "data 3"); my @valuesAofA; push @valuesAofA, [@valuesA]; $soapResponse = $Class->set_string_class_member_data_value ( SOAP::Data->name(class_members => [$StringClass]), SOAP::Data->name(values => [@valuesAofA]) ); &checkResponse($soapResponse); &getDataGroup($datagroup); } #---------------------------------------------------------------------------- sub removeFromDataGroup() #---------------------------------------------------------------------------- { my ($datagroup) = @_; my @names = ("c"); my $StringClass = { name => $datagroup, members => [@names] }; # Create Data group with names $soapResponse = $Class->delete_string_class_member( SOAP::Data->name(class_members => [$StringClass]) ); &checkResponse($soapResponse); &getDataGroup($datagroup); } #---------------------------------------------------------------------------- sub addToDataGroup() #---------------------------------------------------------------------------- { my ($datagroup) = @_; my @names = ("d", "e"); my $StringClass = { name => $datagroup, members => [@names] }; # Create Data group with names $soapResponse = $Class->add_string_class_member( SOAP::Data->name(class_members => [$StringClass]) ); &checkResponse($soapResponse); # Set values # Build Values 2-D Array for values parameter my @valuesA = ("data 4", "data 5"); my @valuesAofA; push @valuesAofA, [@valuesA]; $soapResponse = $Class->set_string_class_member_data_value ( SOAP::Data->name(class_members => [$StringClass]), SOAP::Data->name(values => [@valuesAofA]) ); &checkResponse($soapResponse); &getDataGroup($datagroup); } Powershell REST Example #---------------------------------------------------------------------------- function Create-DataGroup() # # Description: # This function creates a new internal data group # # Parameters: # Name - The Name of the data group #---------------------------------------------------------------------------- { param( [string]$Name ); $uri = "/mgmt/tm/ltm/data-group/internal"; $link = "https://$Bigip$uri"; $headers = @{}; $headers.Add("ServerHost", $Bigip); $headers.Add("Content-Type", "application/json"); $obj = @{ name=$Name type="string" records= ( @{ name="a" data="data 1" }, @{ name="b" data="data 2" }, @{ name="c" data="data 3" } ) }; $body = $obj | ConvertTo-Json $secpasswd = ConvertTo-SecureString $Pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($User, $secpasswd) $obj = Invoke-RestMethod -Method POST -Uri $link -Headers $headers -Credential $mycreds -Body $body; Write-Host "Pool ${Name} created in partition ${Partition}" } #---------------------------------------------------------------------------- function RemoveFrom-DataGroup() # # Description: # This function removes an entry from a data group # # Parameters: # Name - The Name of the data group #---------------------------------------------------------------------------- { param( [string]$Name ); $uri = "/mgmt/tm/ltm/data-group/internal/${Name}"; $link = "https://$Bigip$uri"; $headers = @{}; $headers.Add("ServerHost", $Bigip); $headers.Add("Content-Type", "application/json"); $obj = @{ name=$Name records= ( @{ name="a" data="data 1" }, @{ name="b" data="data 2" } ) }; $body = $obj | ConvertTo-Json $secpasswd = ConvertTo-SecureString $Pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($User, $secpasswd) $obj = Invoke-RestMethod -Method PATCH -Uri $link -Headers $headers -Credential $mycreds -Body $body; $obj | Format-List } #---------------------------------------------------------------------------- function AddTo-DataGroup() # # Description: # This function adds records to an existing data group # # Parameters: # Name - The Name of the data group #---------------------------------------------------------------------------- { param( [string]$Name ); $uri = "/mgmt/tm/ltm/data-group/internal/${Name}"; $link = "https://$Bigip$uri"; $headers = @{}; $headers.Add("ServerHost", $Bigip); $headers.Add("Content-Type", "application/json"); $obj = @{ name=$Name records= ( @{ name="a" data="data 1" }, @{ name="b" data="data 2" }, @{ name="d" data="data 4" }, @{ name="e" data="data 5" } ) }; $body = $obj | ConvertTo-Json $secpasswd = ConvertTo-SecureString $Pass -AsPlainText -Force $mycreds = New-Object System.Management.Automation.PSCredential ($User, $secpasswd) $obj = Invoke-RestMethod -Method PATCH -Uri $link -Headers $headers -Credential $mycreds -Body $body; $obj | Format-List } Python REST Example def get_dg_list(rq, url): try: dg_list = rq.get('%s/ltm/data-group/internal' % url).json() for dg in dg_list['items']: print dg print ' Data Group: %s' % dg['name'] print ' --------------' if 'records' in dg: for record in dg['records']: if 'data' in record: print ' %s: %s' % (record['name'], record['data']) else: print ' %s' % record['name'] except Exception, e: print e def extend_dg(rq, url, dgname, additional_records): dg = rq.get('%s/ltm/data-group/internal/%s' % (url, dgname)).json() current_records = dg['records'] new_records = [] for record in current_records: if 'data' in record: nr = [{'name': record['name'], 'data': record['data']}] else: nr = [{'name': record['name']}] new_records.extend(nr) for record in additional_records: if 'data' in record: nr = [{'name': record['name'], 'data': record['data']}] else: nr = [{'name': record['name']}] new_records.extend(nr) payload = {} payload['records'] = new_records rq.put('%s/ltm/data-group/internal/%s' % (url, dgname), json.dumps(payload)) def contract_dg(rq, url, dgname, removal_records): dg = rq.get('%s/ltm/data-group/internal/%s' % (url, dgname)).json() current_records = dg['records'] new_records = [] for record in removal_records: if 'data' in record: nr = [{'name': record['name'], 'data': record['data']}] else: nr = [{'name': record['name']}] new_records.extend(nr) new_records = [x for x in current_records if x not in new_records] payload = {} payload['records'] = new_records rq.put('%s/ltm/data-group/internal/%s' % (url, dgname), json.dumps(payload)) def create_dg(rq, url, dgname, records): new_records = [] for record in records: if 'data' in record: nr = [{'name': record['name'], 'data': record['data']}] else: nr = [{'name': record['name']}] new_records.extend(nr) payload = {} payload['type'] = 'string' payload['name'] = dgname payload['records'] = new_records try: rq.post('%s/ltm/data-group/internal' % url, json.dumps(payload)) except Exception, e: print e Resources All the scripts for this article are available in the codeshare under "Getting Started with iControl Code Samples."iControl Library For Java With Source
Included are the binary distribution of the iControl library for Java and Apache Axis. These releases coincides through version BIG-IP, version 13.0.0. iControl Assembly for Java 11.3.0 iControl Assembly for Java 11.4.0 iControl Assembly for Java 11.4.1 iControl Assembly for Java 11.5.0 iControl Assembly for Java 11.6.0 iControl Assembly for Java 12.0.0 iControl Assembly for Java 12.1.0 iControl Assembly for Java 13.0.0 iControl Assembly for Java 13.1.0 The source distribution of the iControl library for Java with Apache Axis. These releases coincide through version BIG-IP, version 12.1.0. The source code for this project is no longer maintained but available at our at f5-icontrol-library-java repository on GitHub. iControl Assembly Java Source 11.4.0 (older release not available on Github) iControl Assembly Java Source 11.4.1 iControl Assembly Java Source 11.5.0 iControl Assembly Java Source 11.6.0 iControl Assembly Java Source 12.1.0
Dig deeper into Ansible and F5 integration
Basics of Ansible and F5 integration were covered in a joint webinar held earlier in March 2017. To learn more about the integration and current F5 module support along with some use cases view the webinar . We had another joint webinar in June 2017, which went into details on the integration. We spoke about how to F5 ansible modules (that will be part of upcoming Ansible 2.4 release) can be used to perfrom administrative tasks on the BIG-IP, make the BIG-IP ready for application deployment in weeks rather than in days. We also touched upon usage of F5 ansible iApps module to deploy applications on the BIG-IP. The webinar's was very well received which ended with a great Q and A session. Some of the questions that came up were how do we create playbooks for different workflows, what are the best practices that F5 recommends, can we get a sample playbook etc. We will use this forum to answer some of the questions and dig deeper into the F5 and Ansible integration. Now what really is a playbook, it is nothing but a collection of tasks that are to be performed sequentially on a system. Let us consider a use case where a customer has just purchased 20 BIG-IP’s and needs to get all of them networked and to a state where the BIG-IPs are ready to deploy applications. We can define a playbook which consists of tasks required to perform Day0 and Day1 configuration on the BIG-IPs. Lets start with Day0, now networking the system consists of assigning it a NTP and DNS server address, assigning a hostname, making some ssh customizations etc., some of these settings are common to all the BIG-IPs. The common set of configurations can be defined using the concept of a ‘role’ in ansible. Let’s define a ‘onboarding’ role which will configure the common settings like NTP, DNS and SSHD settings. PLAYBOOK FOR ONBOARDING - name: Onboarding BIG-IP hosts: bigip gather_facts: false roles: - onboarding //playbook runs tasks defined in the ‘onboarding’ role This play book will run against all the BIG-IP’s defined in the inventory host file Example of inventory host file [bigip] 10.192.73.218 10.192.73.219 10.192.73.220 10.192.73.221 The above playbook will run tasks specified in the 'onboarding' role in file main.yaml (playbooks/roles/onboarding/tasks/main.yaml) - name: Configure NTP server on BIG-IP bigip_device_ntp: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" ntp_servers: "{{ ntp_servers }}" validate_certs: False delegate_to: localhost - name: Manage SSHD setting on BIG-IP bigip_device_sshd: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" banner: "enabled" banner_text: " {{ banner_text }}" validate_certs: False delegate_to: localhost - name: Manage BIG-IP DNS settings bigip_device_dns: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" name_servers: "{{ dns_servers }}" search: "{{ dns_search_domains }}" ip_version: "{{ ip_version }}" validate_certs: False delegate_to: localhost Variables will be referenced from the main.yaml file under default directory for the ‘onboarding’ role (playbooks/roles/onboarding/default/main.yaml) username: admin password: admin banner_text: "--------Welcome to Onboarding BIGIP----------" ntp_servers: - '172.27.1.1' - '172.27.1.2' dns_servers: - '8.8.8.8' - '4.4.4.4' dns_search_domains: - 'local' - 'localhost' ip_version: 4 The BIG-IP is now ready to deploy applications. One application is configuring the BIG-IP to securely load balance applications. This requires configuring the following on the BIG-IP Vlans Self-IPs Nodes/members (2) Pool (1) Assigning the nodes to the Pool Creating a HTTPS Virtual server Creating a redirect Virtual server, which will redirect all HTTP requests to the HTTPS virtual server (iRule is assigned to the virtual server to achieve this) This playbook will be run individually for each BIG-IP since each will use different values for VLANS/Self IP’s/Virtual server address etc. The variables values for this playbook is defined inline and not in a separate file. PLAYBOOK FOR APPLICATION DEPLOYMENT - name: creating HTTPS application hosts: bigip tasks: - name: Configure VLANs on the BIG-IP bigip_vlan: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" validate_certs: False name: "{{ item.name }}" tag: "{{ item.tag }}" tagged_interface: "{{ item.interface }}" with_items: - name: 'External' tag: '10' interface: '1.1' - name: 'Internal tag: '11’ interface: '1.2' delegate_to: localhost - name: Configure SELF-IPs on the BIG-IP bigip_selfip: server: "{{ inventory_hostname }}" user: "{{ username }}" password: "{{ password }}" validate_certs: False name: "{{ item.name }}" address: "{{ item.address }}" netmask: "{{ item.netmask }}" vlan: "{{ item.vlan }}" allow_service: "{{item.allow_service}}" with_items: - name: 'External-SelfIP' address: '10.10.10.10' netmask: '255.255.255.0' vlan: 'External' allow_service: 'default' - name: 'Internal-SelfIP' address: '192.10.10.10' netmask: '255.255.255.0' vlan: 'Internal' allow_service: 'default' delegate_to: localhost - name: Create a web01.internal node //Creating Node1 bigip_node: server: "{{ inventory_hostname }}" user: "admin" password: "admin" host: "192.168.68.140" name: "web01.internal" validate_certs: False delegate_to: localhost - name: Create a web02.internal node //Creating Node2 bigip_node: server: "{{ inventory_hostname }}" user: "admin" password: "admin" host: "192.168.68.141" name: "web02.internal" validate_certs: False delegate_to: localhost - name: Create a web-pool //Creating a pool bigip_pool: server: "{{ inventory_hostname }}" user: "admin" password: "admin" lb_method: "ratio_member" monitors: http name: "web-pool" validate_certs: False delegate_to: localhost - name: Add http node to web-pool //Assigning members to a pool bigip_pool_member: description: "HTTP Webserver-1" host: "{{ item.host }}" name: "{{ item.name }}" user: "admin" password: "admin" pool: "web-pool" port: "80" server: "{{ inventory_hostname }}" validate_certs: False with_items: - host: "192.168.168.140" name: "web01.internal" - host: "192.168.68.141" name: "web02.internal" delegate_to: localhost - name: Create a virtual server //Create a HTTPS Virtual Server bigip_virtual_server: description: "Secure web application" server: "{{ inventory_hostname }}" user: "admin" password: "admin" name: "https_vs" destination: "10.10.20.120" port: 443 snat: "Automap" all_profiles: - http - clientssl pool: "web-pool" validate_certs: False delegate_to: localhost - name: Create a redirect virtual server //Create a redirect virtual server bigip_virtual_server: description: "Redirect Virtual server" server: "{{ inventory_hostname }}" user: "admin" password: "admin" name: "http_redirect" destination: "10.10.20.120" validate_certs: False port: 80 all_profiles: - http all_rules: //Attach an iRule to the Virtual server - _sys_https_redirect delegate_to: localhost Bookmark this page if you are interested in learning more. We will be updating this blog with new F5 modules that are going to be supported with Ansible 2.4 release
F5 Friday: Python SDK for BIG-IP
We know programmability is important. Whether we’re talking about networking and SDN, or DevOps and APIs and templates, the most impactful technologies and trends today are those involving programmability. F5 is, as you’re no doubt aware, no stranger to programmability. Since 2001 when we introduced iControl (API) and iRules (data path programmability) we’ve continued to improve, enhance, and expand the ability of partners, customers, and our own engineers and architects to programmatically control and redefine the application delivery experience. With the emphasis today on automation and orchestration as a means for ops (and through it, the business) to scale more quickly and efficiently, programmability has never before been so critical to both operational and business success. Which means we can’t stop improving and expanding the ways in which you (and us, too) can manage, extend, and deliver the app services everyone needs to keep their apps secure, fast, and available. Now, iControl and iControl REST are both APIs built on open standards like SOAP, JSON, and HTTP. That means anyone who knows how to use an API can sit down and start coding up scripts that automate provisioning, configuration, and general management of not just BIG-IP (the platform) but the app services that get deployed on that platform. And we’re not against that at all. But we also recognize that not everyone has the time to get intimately familiar with iControl in either of its forms. So we’re pretty much always actively developing new (and improving existing) software development kits (SDKs) that enable folks to start doing more faster. But so are you. We’ve got a metric ton of code samples, libraries, and solutions here on DevCentral that have been developed by customers and partners alike. They’re freely available and are being updated, optimized, extended and re-used every single day. We think that’s a big part of what an open community is – it’s about developing and sharing solutions to some of the industry’s greatest challenges. And that’s what brings us to today’s exciting news. Well, exciting if you’re a Python user, at least, because we’re happy to point out the availability of the F5 BIG-IP Python SDK. And not just available to download and use, but available as an open source project that you can actively add, enhance, fork, and improve. Because open source and open communities produce some amazing things. This project implements an SDK for the iControl REST interface for BIG-IP, which lets you create, edit, update, and delete (CRUD) configuration objects on a BIG-IP. Documentation is up to date and available here. The BIG-IP Python SDK layers an object model over the API and makes it simpler to develop scripts or integrate with other Python-based frameworks. The abstraction is nice (and I say that with my developer hat on) and certainly makes the code more readable (and maintainable, one would assume) which should help eliminate some of the technical debt that’s incurred whenever you write software, including operational scripts and software. Seriously, here’s a basic sample from the documentation: from f5.bigip import BigIP # Connect to the BigIP bigip = BigIP("bigip.example.com", "admin", "somepassword") # Get a list of all pools on the BigIP and print their name and their # members' name pools = bigip.ltm.pools.get_collection() for pool in pools: print pool.name for member in pool.members: print member.name # Create a new pool on the BigIP mypool = bigip.ltm.pools.pool.create(name='mypool', partition='Common') # Load an existing pool and update its description pool_a = bigip.ltm.pools.pool.load(name='mypool', partition='Common') pool_a.description = "New description" pool_a.update() # Delete a pool if it exists if bigip.ltm.pools.pool.exists(name='mypool', partition='Common'): pool_b = bigip.ltm.pools.pool.load(name='oldpool', partition='Common') pool_b.delete() Isn’t that nice? Neat, understandable, readable. That’s some nice code right there (and I’m not even a Python fan, so that’s saying something). Don’t let the OpenStack reference fool you. While the first “user” of the SDK is OpenStack, it is stand-alone and can be used on its own or incorporated into other Python-based frameworks. So if you’re using Python (or were thinking about) to manage, manipulate, or monitor your BIG-IPs, check this one out. Use it, extend it, improve it, and share it. Happy scripting!Getting Started with iControl: History
tl;dr - iControl provides access to BIG-IP management plane services through SOAP and REST API interfaces. The Early Days iControl started back in early 2000. F5 had 2 main products: BIG-IP and 3-DNS (later GTM, now BIG-IP DNS). BIG-IP managed the local datacenter's traffic, while 3-DNS was the DNS orchestrator for all the BIG-IP's in numerous data centers. The two products needed a way to communicate with each other to ensure they were making the right traffic management decisions respective to all of the products in the system. At the time, the development team was focused on developing the fastest running code possible and that idea found it's way into the cross product communication feature that was developed. The technology the team chose to use was the Common Object Request Broker Architecture (CORBA) as standardized by the Object Management Group (OMG). Coming hot off the heels of F5's first management product SEE-IT (which was another one of my babies), the dev team coined this internal feature as "LINK-IT" since it "linked" the two products together. With the development of our management, monitoring, and visualization product SEE-IT, we needed a way to get the data off of the BIG-IP. SEE-IT was written for Windows Server and we really didn't want to go down the route of integrating into the CORBA interface due to several factors. So, we wrote a custom XML provider on BIG-IP and 3-DNS to allow for configuration and statistic data to be retrieved and consumed by SEE-IT. It was becoming clear to me that automation and customization of our products would be beneficial to our customers who had been previously relying on our SNMP offerings. We now had 2 interfaces for managing and monitoring our devices: one purely internal (LINK-IT) and the other partially (XML provider). The XML provider was very specific to our SEE-IT products use case and we didn't see a benefit of trying to morph that so we looked back at LINK-IT to see what we could to do make that a publicly supported interface. We began work on documenting and packaging it into F5's first public SDK. About that time, a new standard was emerging for exchanging structured information. The Simple Object Access Protocol (SOAP), which allows for structured information exchange, was being developed but not fully ratified until version 1.2 in 2003. I had to choose to roll our own XML implementation or make use of this new proposed specification. There was risk as the specification was not a standard yet but I made the choice to go the SOAP route as I felt that picking a standard format would give us the best 3rd party software compatibility down the road. Our CORBA interface was built on a nice class model which I used as a basis for an SOAP/XML wrapper on top of that code. I even had a great code name for the interface: X-LINK-IT! For those who were around when I gave my "XML at F5" presentation to F5 Product Development, you may remember the snide comments going around afterwards about how XML was not a great technology and a big mistake supporting. Good thing I didn't listen to them... At this point in mid-2001, the LINK-IT SDK was ready to go and development of X-LINK-IT was well underway. Well, let's just say that Marketing didn't agree with our ingenious product naming and jumped in to VETO our internal code names for our public release. I'll give our Chief Marketer Jeff Pancottine credit for coining the term "iControl" which was explained to me as "Internet Control". This was the start of F5's whole Internet Controlled Architecture messaging by the way. So, LINK-IT and X-LINK-IT were dead and iControl CORBA and iControl SOAP were born. The Death of CORBA, All Hail SOAP The first version of the iControl SDK for CORBA was released on 5/25/2001 with the SOAP version trailing it by a few months. This was around the BIG-IP version 3 time frame. We chugged along for a few years through the BIG-IP version 4 life and then a big event occurred that was the demise for CORBA - well, it's actually 2 events. The first event was the full rewrite of the BIG-IP data plane when TMOS was introduced in BIG-IP, version 9 (we skipped from version 4 to version 9 for some reason that slips my mind). Since virtually the entire product was rewritten, the interfaces that were tied to the product features, would have to change drastically. We used this as an opportunity to look at the next evolution of iControl. Until this point, iControl SOAP was just a shim on top of CORBA and it had some performance issues so we worked at splitting them apart and having SOAP talk directly to our configuration engine. Now we had 2 interface stacks side by side. The second event was learning we only had 1 confirmed customer using the CORBA interface compared to the 100's using SOAP. Given that knowledge and now that BIG-IP and 3-DNS no longer used iControl CORBA to talk to each other, the decision was made to End of Life iControl CORBA with Version 9.0. But, iControl SOAP still used the CORBA IDL files for it's API definitions and documentation so fun trivia note: the same CORBA tools are still in place in today's iControl SOAP build tools that were there in version 3 of BIG-IP. I'm fairly sure that is the longest running component in our build system. The Birth of DevCentral I can't speak about iControl without mentioning DevCentral. We had our iControl SDK out but no where to directly support the developers using it. At that time, F5 was a "hardware" company and product support wasn't ready to support application developers. Not many know that DevCentral was created due to the popularity of iControl with our customer base and was born on a PC under my desk in 2003. I continued to help with DevCentral part time for a few years but in 2007 I decided to work full time on building our community and focusing 100% on DevCentral. It was about this time that we were pushing the idea of merging the application and infrastructure teams together - or at least getting them to talk more frequently. This was a precursor to the whole DevOps mentality so I'd like to think we were a bit ahead of the curve on that. Enter iControl REST In 2013, iControl was reaching it's teenage years and starting to show it's age a bit. While SOAP is still supported by all the major tool vendors, application development was shifting to richer browser-based apps. And with that, Representational State Transfer (REST) was gaining steam. REST defined a usage pattern for using browser based mechanisms with HTTP to access objects across the network with a JavaScript Object Notation (JSON) format for the content. To keep up with current technologies, the PD team at F5 developed the first REST/JSON interface in BIG-IP version 11.5 as Early Access and was made Generally Available in version 11.6. With the REST interface, more modern web paradigms could be supported and you could actually code to iControl directly from a browser! There were also additional interface based tools for developing and debugging built directly into the service to give service listings and schema definitions. At the time of this writing, F5 supports both of the main iControl interfaces (SOAP and REST) but are focusing all new energy on our REST platform for the future. For those who have developed SOAP integrations, have no fear as that interface is not going away. It will just likely not get all the new feature support that will be added into the REST interfaces over time. SDKs, Toolkits, and Libraries Through the years, I've developed several client libraries (.Net, PowerShell, Java) for iControl SOAP to assist with ramp-up time for initial development. There have also been numerous other language based libraries for languages like Ruby, PHP, and Python developed by the community and other development teams. Most recently, F5 has published the iControl library for Python which is available as part of our OpenStack integration. DevCentral is your place to for our API documentation where we update our wiki with API changes on each major release. And as time rolls on, we are adding REST support for new and existing products such as iWorkflow, BIG-IQ, and other products yet to be released that will include SDKs and other reference material. F5 has a strong commitment to our end users and their automation and integration projects with F5 technologies. Coming full circle, my current role is overseeing APIs and SDKs for standards, consistency, and completeness in our Programmability and Orchestration (P&O) team so keep a look out for future articles from me on our efforts on that front. And REST assured, we will continue to do all we can to help our customers move to new architectures and deployment models with programmability and automation with F5 technologies.Introducing a RESTful interface for iControl
iControl isn’t just SOAP anymore… No, iControl isn’t getting lazy. While taking it easy is an important part of life, I’m talking about the other kind of REST. REST, or “REpresentational State Transfer” for you technically inclined, is a style of architectural principals with which you can design web services that focus on a system’s resources. It also defines how resource states are addressed and transferred over the network. REST is really a “style” of getting and setting resources and doesn’t define the underlying communications. Most implementations out there make use of HTTP and JSON as a content format, which is what we’ve chosen to do as well. There are plenty of articles on the web that compare and contrast SOAP and REST, so I won’t get into those here. I’m also not going to go really deep into the principals of REST as you can find dozens of those easily with a web search. In this article, I’ll discuss how we’ve chosen to implement our REST interface for iControl and give you some examples on how to use it. Oh, and don’t get any ideas that our SOAP based interface is going anywhere. We are creating our REST interfaces as an alternate method for performing automation and monitoring. We don’t currently have plans on ceasing development on our SOAP interface. iControl-REST The REST interface for iControl was introduced in BIG-IP version 11.4. For this release, we are considering the feature as “Early Access”. Call it beta or whatever you want. But what that really means is that it will change in our next release. We are using this release for feedback from the users out there to find out what works and what doesn’t. Several key features are not implemented yet (for example versioning) which we have targeted for an upcoming product release when we finalize the implementation. Ok, with that said, now we can get into the details. iControl-REST, like it’s SOAP counterpart, is implemented on HTTPS and uses the same authentication and authorization roles for user access. We support the following HTTP commands: GET - for retrieving (ie. Querying the status of a Pool) POST - for creating (ie. Creating a Pool Member) PUT - for updating (ie. Changing the load balancing method on a Pool) DELETE - for deleting (ie. Deleting a Virtual Server) And, the format of the requests and responses is JSON. Starting the iControl REST Service (icrd) Run the “modify sys service icrd” TMSH command to add and start the iControl REST service. Notice the nice “EA” warning. root@(BIG-IP1)(…)(tmos) # modify sys service ircd add WARNING: This early-access feature comes with minimal documentation and testing. Version control is not implemented; therefore any scripts that you write using this API version may not work with subsequent releases. root@(BIG-IP1)(…)(tmos) # Once the service is running, you can use the “show sys service”, “stop sys service” and “start sys service” TMSH commands to monitor and control the status of the iControl REST service. Writing Your First Script Since iControl-REST is just using HTTPS, you can use any scripting technology you heart desires. For this article, we’ll use the command line tool cURL. It’s cross platform, so you should be able to wrap a bash script for Unix, Mac, etc or PowerShell for Windows around it very easily. Use the “-u” parameter to pass in the user credentials, the “-X” parameter to specify the HTTP method, and the uri for the resource you wish to access. curl -k -u user:pass -X http-method uri The URI format is as follows Module URI https://management-ip/mgmt/tm/module This access all of the sub-modules and/or components under the given module (ltm, gtm, etc). Sub-module URI https://management-ip/mgmt/tm/module/sub-module This access all of the sub-modules and/or components under the given sub-module. Component URI https://management-ip/mgmt/tm/module[/sub-module]/component This accesses the details about the given component. The tmsh Traffic Management Shell Reference documents the hierarchy of modules and components, and identifies the details of each component. Example This curl command will query all of the information about the ltm module. $ curl -k -u user:pass -H “Content-Type: application/json” -X GET https://bigip_ip/mgmt/tm/ltm { "currentItemCount": 22, "items": [ { "reference": { "link": "https://bigip_ip/mgmt/tm/ltm/auth" } }, { "reference": { "link": "https://bigip_ip/mgmt/tm/ltm/data-group" } }, { "reference": { "link": "https://bigip_ip/mgmt/tm/ltm/dns" } } ... ], "kind": "tm:ltm:ltmstate", "pageIndex": 1, "partition": "/Common/", "selfLink": "https://bigip-ip/mgmt/tm/ltm", "startIndex": 1, "totalItems": 22, "totalPages": 1 } User Guide For this release, we have provided a User Guide to assist in getting started with using the new REST interface. It can be downloaded from this link: iControlRest-UserGuide.pdf. We would love to hear your feedback on using iControl-REST. Any and all comments and questions should be posted to the iControl group. -Joe
F5 Security on Owasp Top 10
Everyone is familiar with the Owasp Top 10. Below, you will find some notes on the Top 10, as well as ways to mitigate these potential threats to your environment. You can also download the PDF format by clicking the blankie ––> This is the first in a series that will cover the attack vectors and how to apply the protection methods. OWASP Attack OWASP DEFINITION F5 PROTECTION A1 Injection Injection flaws, such as SQL, OS, and LDAP injection, occur when untrusted data is sent to an interpreter as part of a command or query. The attacker’s hostile data can trick the interpreter into executing unintended commands or accessing unauthorized data. BIG-IP ASM inspects application traffic and blocks the insertion of malicious scripts. It does so by enforcing injection attack patterns, enforcing an accurate usage of metacharacters within the URI and parameter names. ASM also looks at parameter values and can enforce pre-defined allowed values, length and accurate usage of metacharacters. A2 Cross-Site Scripting (XSS) XSS flaws occur whenever an application takes untrusted data and sends it to a web browser without proper validation and escaping. XSS allows attackers to execute scripts in the victim’s browser which can hijack user sessions, deface web sites, or redirect the user to malicious sites. BIG-IP ASM protects against Cross-Site Scripting attacks by enforcing XSS attack patterns, enforcing an accurate usage of metacharacters within the URI and parameter names. ASM also looks at parameter values and can enforce pre-defined allowed values, length and accurate usage of metacharacters. A3 Broken Authentication and Session Management Application functions related to authentication and session management are often not implemented correctly, allowing attackers to compromise passwords, keys, session tokens, or exploit other implementation flaws to assume other users’ identities. BIG-IP ASM enables protection by: • Using ASM’s unique login page enforcement configuration • Enforcing login page timeouts • Enabling application flow enforcement and dynamic parameter protection • Using SSL on the login page • Monitoring request attack patterns • Using ASM signed cookies so none are being manipulated A4 Insecure Direct Object References A direct object reference occurs when a developer exposes a reference to an internal implementation object, such as a file, directory,or database key. Without an access control check or other protection, attackers can manipulate these references to access unauthorized data. If a hacker changes his account number to another random number hoping to access a different user’s account they can manipulate those references to access other objects without authorization. These can include: • Fraud (price changes, user ID changes) • Session highjacking • Enforcing parameter values with high parameters BIG-IP ASM mitigates this vulnerability by enforcing dynamic parameters (making sure values that were set by the server will not be changed on the client side). Also the admin. can whitelist the allowed URLs for the specific application and scan the requests with attack patterns. A5 Cross-Site Request Forgery (CSRF) A CSRF attack forces a logged-on victim’s browser to send a forged HTTP request, including the victim’s session cookie and any other automatically included authentication information, to a vulnerable web application. This allows the attacker to force the victim’s browser to generate requests the vulnerable application thinks are legitimate requests from the victim. BIG-IP ASM mitigates CSRF attacks by adding a random nonce to every URL. This nonce cannot be guessed in advance by an attacker and therefore makes the attack almost impossible. In addition, ASM is preventing XSS within an application and enforcing the application flow and dynamic parameter values. With flow access, a session timeout can be combined with an F5 iRule™ designed to note referrer header check to minimize CSRF. For instance, flow enforcement mitigates CSRF by limiting the entry points or web pages of attacks along with session timeouts being short. If referring to say www.food.com, ASM checks the referrer header in the URL to make sure it’s food.com. A6 Security Misconfiguration Good security requires having a secure configuration defined and deployed for the application, frameworks, application server, web server, database server, and platform. All these settings should be defined, implemented, and maintained as many are not shipped with secure defaults. This includes keeping all software up to date, including all code libraries used by the application. BIG-IP ASM can mitigate attacks that are related to misconfiguration by using a broad range of controls starting with: • RFC enforcement • Enforcing various limits on the requests • Whitelisting the URLs and parameters names and values • Enforcing a login page • Being a native full reverse proxy A7 Insecure Cryptographic Storage Many web applications do not properly protect sensitive data, such as credit cards, SSNs, and authentication credentials, with appropriate encryption or hashing. Attackers may steal or modify such weakly protected data to conduct identity theft, credit card fraud, or other crimes. While this isn’t directly related to BIG-IP ASM or WAF, OWASP is mostly concerned with what type of encryption is used and how it is used. These are both outside of the enforcement purview of ASM; however, ASM delivers the following: • Data Guard - if someone managed to cause an information leakage, Data Guard can block it • BIG-IP certificate management allows the user to store private keys in a central and secure place. A8 Failure to Restrict URL Access Many web applications check URL access rights before rendering protected links and buttons. However, applications need to perform similar access control checks each time these pages are accessed, or attackers will be able to forge URLs to access these hidden pages anyway. There are multiple ways that BIG-IP ASM can mitigate this issue. , ASM enforces allowed file types and URLs, and accurate parameter values and login pages. BIG-IP ASM’s “flow” technology ensures that site content is only accessed by users that have acquired the proper credentials or visited the prerequisite pages. Users can only visit personal web pages if they have come from the say a user ID and password sign on web page. A9 Insufficient Transport Layer Protection Applications frequently fail to authenticate, encrypt, and protect the confidentiality and integrity of sensitive network traffic. When they do, they sometimes support weak algorithms, use expired or invalid certificates, or do not use them correctly. BIG-IP ASM significantly simplifies the implementation of SSL and certificate management by centralizing the location and administration of the server certificates in a single location rather than distributed over farms of servers. Also, by moving SSL handshaking and encryption to BIG-IP ASM, the Web servers gain an increased level of performance and efficiency. In addition ASM allows you to do the following : • Require SSL for all sensitive pages. Non-SSL requests to these pages redirected to the SSL page. Use BIG-IP SSL Acceleration in general for the whole application • Set the ‘secure’ flag on all sensitive cookies • Configure your SSL provider to only support strong (e.g., FIPS 140-2 compliant) algorithms. (Use BIG-IP 6900, 8900) • Ensure your certificate is valid, not expired, not revoked, and matches all domains used by the site. You can check with EM or scripts from Devcentral • Backend and other connections should also use SSL or other encryption technologies. Use re-encryption with Server-SSL-profile A10 Unvalidated Redirects and Forwards Web applications frequently redirect and forward users to other pages and websites, and use untrusted data to determine the destination pages. Without proper validation, attackers can redirect victims to phishing or malware sites, or use forwards to access unauthorized pages. BIG-IP ASM mitigates this issue by enforcing unique attack patterns, enforcing accurate values of parameters and enforcing dynamic parameters.
