iControl 101 - #24 - Folders

Bucket

Way back in time (well, not so way back), configuration objects were stored in one location in the configuration.  For the sake of this article, we’ll call this the root “bucket”.  This worked fine for small organizations but we found that as companies grew and, as a consequence, the number of applications they needed to support increased, it became more difficult to manage all the objects in a single “bucket”.

• vs_1
• vs_2
• pool_1
• pool_2
• monitor_1
• monitor_2

Buckets

In BIG-IP version 9.4, we introduced the concept of “Partitions”.  With Partitions, you could create additional “buckets” of configuration objects.  Each partition could contain objects and have it’s own set of authorization roles protecting them allowing the administrator to group them together and allow each application team to manage those objects without exposing access to objects they weren’t in control of.  I discussed this interface in my article titled: iControl 101 - #08 - Partitions.  A layout of the previously defined objects could now look like the following:

• /APP1
  • vs_1
  • pool_1
  • monitor_1
• /APP2
  • vs_2
  • pool_2
  • monitor_2

This still has the limitation in that it’s not easy to determine which groupings of objects (APP1, APP2, …) belong together.  The next logical extension is to allow arbitrary “buckets” of objects which is what I will talk about for the rest of this article.

Buckets of Buckets

BIG-IP version 11, introduced the concepts of Folders.  Here’s an excerpt from the iControl SDK’s reference page for the Folder interface:

A folder stores an arbitrary set of configuration objects. The system uses folders to control access to objects and to control synchronization of these objects within a device group. Folders are organized in a tree heirarchy, much like the folders or directories on a computer's file system. Objects stored in folders are referenced by the name of the individual object, preceded by its folder name, preceded by the names of any parent folders, up to the root folder (/), all separated by slashes (/), e.g., /george/server/virt-a. Note: methods to access the active folder for a session are found in the System::Session interface.

So, now we can have the objects look something like this

• /APPGROUP
  • /APP1
    • vs_1
    • pool_1
    • monitor_1
  • /APP2
    • vs_2
    • pool_2
    • monitor_2

Since I’m a console-kind-of-guy at heart, and how folders are very similar to directories in a file system, I figured, I’d write a companion sample for this article that emulated a directory shell allowing you to navigate through, create, remove, and modify folders while illustrating how to use the various iControl methods for those tasks.

The Application: A Folder Shell

I wrote this application in PowerShell, but it could have just as easily been coded in Java, Perl, Python, .Net, or whatever you tend to use with your projects.  Let’s take a look at the implementation, the actions that it performs, and some sample output

Initialization

This application will take only three parameters as input.  The address of the BIG-IP,  and the username and password for authentication.  The main application loop consists of:

1. Verifying the connection information is valid with the code in the Do-Initialize method.
2. Setting the prompt to the users current folder
3. Reading a command from the user and passing it to the Process-Input function described below.

param (
  $bigip = $null,
  $uid = $null,
  $pwd = $null
)

Set-PSDebug -strict;

# Global Script variables
$script:DEBUG = $false;
$script:FOLDER = $null;
$script:RECURSE = "STATE_DISABLED";

function Do-Initialize()
{
  if ( (Get-PSSnapin | Where-Object { $_.Name -eq "iControlSnapIn"}) -eq $null )
  {
    Add-PSSnapIn iControlSnapIn
  }
  $success = Initialize-F5.iControl -HostName $bigip -Username $uid -Password $pwd;
  
  return $success;
}

# Main Application Logic
if ( ($bigip -eq $null) -or ($uid -eq $null) -or ($pwd -eq $null) )
{
  usage;
}

if ( Do-Initialize )
{
  $s = Get-RecursiveState;
  while(1)
  {
    $prompt = Get-Prompt;
    # Not using Read-Host here so we can support responses starting with !
    $host.UI.Write($prompt);
    $i = $host.UI.ReadLine().Trim();
    Process-Input $i;
  }
}
else
{
  Write-Error "ERROR: iControl subsystem not initialized"
}

The Main Application Loop

The main application loop passes commands to the Process-Input function.   A wildcard match is performed against the passed in command and if there is a match, control is passed to the appropriate handler function.   I won’t describe them all here but they should be fairly self explanatory.  The main types of actions are described in the upcoming sections.

function Process-Input()
#----------------------------------------------------------------------------
{
  param($i);
  Debug-Message "< $($MyInvocation.MyCommand.Name) '$i' >";
  if ( $i.Length -gt 0 )
  {
    Debug-Message "CommandLine: '$i'...";
    switch -Wildcard ($i.Trim().ToLower())
    {
      ""        { break; }
      "cd *"    { Change-Folder (Get-Args $i); }
      "cd"      { Get-CurrentFolder; }
      "d"       { $script:DEBUG = -not $script:DEBUG; }
      "dir"     { Get-ChildFolders | Sort-Object Folder; }
      "gd *"    { Get-FolderDescription (Get-Args $i); }
      "h"       { Show-Help; }
      "ls"      { Get-ChildFolders | Sort-Object Folder; }
      "ls -l"   { Get-ChildFolders -Long | Sort-Object Folder; }
      "ls -lr"  { Get-ChildFolders -Recurse -Long | Sort-Object Folder; }
      "ls -r"   { Get-ChildFolders -Recurse | Sort-Object Folder; }
      "md *"    { Create-Folder (Get-Args $i); }
      "mkdir *" { Create-Folder (Get-Args $i); }
      "pwd"     { Get-CurrentFolder; }
      "r"       { Set-RecursiveState; }
      "r *"     { Set-RecursiveState (Get-Args $i); }
      "rd *"    { Remove-Folder (Get-Args $i); }
      "rmdir *" { Remove-Folder (Get-Args $i); }
      "sd *"    { Set-FolderDescription (Get-Args $i); }
      "q"       { exit; }
      "! *"     { Execute-Command (Get-Args $i); }
      "$*"      { Execute-Command $i.SubString(1); }
      ".."      { Move-Up; }
      default   { Show-Help; }
    }
  }
}

Querying The Current Folder

The location of the users current, or “active”, folder is determined by calling the System.Session.get_active_folder() method.  This is a server side variable that is stored during the lifetime of the iControl Portals instance and tied to the current authenticated user.  This Get-CurentFolder function does some internal caching with the $script:FOLDER variable to avoid repetitive calls to the server.  The caching can be overridden by passing in the “-Force” argument to the function causing a forced call to query the active folder.  The value is then returned to the calling code.

function Get-CurrentFolder()
{
  param([switch]$Force = $false);
  
  if ( ($($script:FOLDER) -eq $null) -or ($Force) )
  {
    $folder = (Get-F5.iControl).SystemSession.get_active_folder();
  }
  else
  {
    $folder = $script:FOLDER;
  }
  $folder;
}

Listing the Child Folders

The Get-ChildFolders function has several options you can use with it.  If “-Recurse” is passed to the function, the recursive query state is set to STATE_ENABLED telling the server to return all child objects in all child folders relative to the currently active one.  This is similar to a “dir /s” on Windows or “ls -r” on Unix.  The second parameter is “-Long”.  If this is passed in, then a “long” listing will be presented to the user containing the folder name, description, and device group.  Folder descriptions are described below while I’m leaving device groups to be a task for the reader to follow up on.

The function gathers the requested information and then packs the output into objects and returns them along the PowerShell pipeline to the calling code.

function Get-ChildFolders()
{
  param([switch]$Recurse = $false, [switch]$Long = $false);

  if ( $Recurse )
  {
    $oldstate = (Get-F5.iControl).SystemSession.get_recursive_query_state();
    (Get-F5.iControl).SystemSession.set_recursive_query_state("STATE_ENABLED");
  }
  $folders = (Get-F5.iControl).ManagementFolder.get_list();
  if ( $Recurse -and ($oldstate -ne "STATE_ENABLED") )
  {
    (Get-F5.iControl).SystemSession.set_recursive_query_state($oldstate);
  }
  
  $descriptions = (Get-F5.iControl).ManagementFolder.get_description($folders);
  $groups = (Get-F5.iControl).ManagementFolder.get_device_group($folders);
  
  $curfolder = Get-CurrentFolder;
  if ( $curfolder -eq "/" ) { $curfolder = "ZZZZZZZZ"; }
  
  for($i=0;$i-lt$folders.length;$i++)
  {
    if ( $Long )
    {
      $o = 1 | select "Folder", "Description", "DeviceGroup";
      $o.Folder = $folders[$i].Replace($curfolder, "");
      $o.Description = $descriptions[$i];
      $o.DeviceGroup = $groups[$i];
    }
    else
    {
      $o = 1 | select "Folder";
      $o.Folder = $folders[$i].Replace($curfolder, "");
    }
    $o;
  }
}

Changing The Current Folder

The Change-Folder function emulates the “cd” or “chdir” functionality in command shells.  The folder parameter can either be a fully qualified folder name (ie /PARTITION1/FOLDER1/SUBFOLDER), a relative child folder path (ie. FOLDER1/SUBFOLDER, SUBFOLDER, etc), or the special “..” folder which means to go up one level on the folder hierarchy.

function Change-Folder()
#----------------------------------------------------------------------------
{
  param($folder);
  
  Debug-Message "Setting active folder to '$folder'";
  if ( $folder -eq ".." )
  {
    Move-Up;
  }
  else
  {
    (Get-F5.iControl).SystemSession.set_active_folder($folder);
  }
  $f = Get-CurrentFolder -Force;
}

Moving Up A Folder

I wrote a special function to move up one folder from the currently active folder.  The Move-Up function parses the current folder and moves up a path using the PowerShell Split-Path cmdlet.  If the path isn’t the top folder (meaning it has a parent), then the call to System.Session.set_active_folder() is made and the new current folder is queried and cached for future use.

function Move-Up()
{
  $folder = Get-CurrentFolder;
  $parent = (Split-Path $folder).Replace('\', '/');
  if ( $parent.Length -gt 0 )
  {
    Debug-Message "Setting active folder to '$parent'";
    (Get-F5.iControl).SystemSession.set_active_folder($parent);
  }
  $f = Get-CurrentFolder -Force;
}

Creating And Removing Folders

Navigating folders is fun, but it’s more fun to create and destroy them!  The Create-Folder and Remove-Folder functions do just that.  They call the Management.Folder.create() and Management.Folder.delete_folder() methods to do these actions.

function Create-Folder()
{
  param($folder);
  Debug-Message "Creating folder '$folder'";
  (Get-F5.iControl).ManagementFolder.create($folder);
  Write-Host "Folder '$folder' successfully created!";
}

function Remove-Folder()
{
  param($folder);
  Debug-Message "Removing folder '$folder'";
  (Get-F5.iControl).ManagementFolder.delete_folder($folder);
  Write-Host "Folder '$folder' successfully removed!";
}

Folder Descriptions

One great feature of folders is the ability to attach a description to it.  A folder is just another type of object and it’s sometimes useful to be able to store some metadata in there with information that you can’t fit into the name.  The Set-FolderDescription and Get-FolderDescription functions call the Management.Folder.set_description() and Management.Folder.get_description() iControl methods to, well, get and set the descriptions.

function Set-FolderDescription()
{
  param($cmd);
  $tokens = $cmd.Split(" ");
  if ( $tokens.Length -eq 1 )
  {
    $f = $cmd;
    $d = "";
    Debug-Message "Setting folder '$folder' description to '$d'";
    (Get-F5.iControl).ManagementFolder.set_description($f, $d);
    Get-FolderDescription $f;
  }
  elseif ( $tokens.Length -gt 1 )
  {
    # folder description goes here
    $f = $tokens[0];
    $d = $tokens[1];
    for($i=2; $i-lt$tokens.Length; $i++)
    {
      $d += " ";
      $d += $tokens[$i];
    }
    Debug-Message "Setting folder '$f' description to '$d'";
    (Get-F5.iControl).ManagementFolder.set_description($f, $d);
    Get-FolderDescription $f;
  }
  else
  {
    Show-Help;
  }
}

function Get-FolderDescription()
{
  param($folder);
  Debug-Message "Retrieving folder description for '$folder'";
  $descriptions = (Get-F5.iControl).ManagementFolder.get_description($folder);
  $descriptions[0];
}

Controling The Recursive State For Queries

I touched at the recursive query option above when I was showing how to query child folders.  This recursive state not only applies to folder queries, but all queries across the iControl API!  The Set-RecursiveState function sets this configuration variable to either STATE_ENABLED or STATE_DISABLED.  If the recursive_query_state is set to STATE_DISABLED, then queries will only return objects in the current active folder.  But, if it’s set to STATE_ENABLED, then it will return all child objects in all child folders.  So, by setting this one value, you can determine whether you get just the objects (pools, virtuals, vlans, monitors, etc) in the current folder, or all of them in all child folders.  Cool stuff!

function Set-RecursiveState()
{
  param($state = $null);
  
  $newState = "STATE_DISABLED";
  if ( $state -eq $null )
  {
    # toggle
    $oldState = (Get-F5.iControl).SystemSession.get_recursive_query_state();
    if ( $oldState -eq "STATE_DISABLED" )
    {
      $newState = "STATE_ENABLED";
    }
  }
  else
  {
    # set
    if ( $state.ToLower().Contains("enable") )
    {
      $newState = "STATE_ENABLED";
    }
  }
  $script:RECURSE = $newState;
  (Get-F5.iControl).SystemSession.set_recursive_query_state($newState);
  Write-Host "Recursive State set to '$newState'";
}

function Get-RecursiveState()
{
    $oldState = (Get-F5.iControl).SystemSession.get_recursive_query_state();
    $script:RECURSE = $oldState;
    $oldState;
}

Executing Arbitrary iControl Commands

I threw this in to help illustrate how the recursive state described above works.  By passing in the command “! LocalLBVirtualServer.get_list()”, that iControl call will be executed and the results passed to the output.  By using the Invoke-Expression cmdlet, any iControl call can arbitrarily be made within the shell.  Again, cool stuff!

function Execute-Command()
{
  param($cmd);
  
  $fullcmd = $cmd;
  if ( -not $fullcmd.ToLower().StartsWith("(get-f5.icontrol).") )
  {
    $fullcmd = "(Get-F5.iControl).$cmd";
  }
  
  Debug-Message "Executing command '$fullcmd'";
  Invoke-Expression $fullcmd;
}

A Demo Walkthrough

Conclusion

Folders will allow administrators more control of grouping their objects together, thus enabling them with the long-term manageability of those objects.  This example illustrated how to use the iControl methods to interact with folders and, hopefully, in doing so, showed the ease at building powerful iControl solutions.

The Source Code

The Full Source can be found in the iControl Wiki in the CodeShare entry titled PowerShellManagementFolder. 

Related Content on DevCentral