Custom BIG-IP Object MetaData With Data Groups

An internal email came across a few weeks ago from DevCentral’s own L4L7 asking about putting extensions into the product to allow for custom attributes to be assigned to objects within the system.  The problem is pretty simple, you have a large configuration site and you would like to “tag” an object to be able to easily categorize it later.  A good example is to assign an owner to a virtual server, or maybe a timestamp of when the object was last updated.  By doing this you don’t only have a way to remotely query objects of a certain type, but you could use an iRule to use that metadata to make real time policy decisions.

Instead of digging into how we would incorporate this into the internal schemas of the core configuration database, I figured I’d tackle the problem in a different way.  One that doesn’t require an update to the product and will work backward on previous products.  Of course iControl came to mind.  I thought to myself about how it could be done.  Several ideas came to mind but, as it so often happens, the simplest solution satisfied all the requirements.  I ultimately decided on using Data Groups as the data store since those are able to be accessed from both iControl (for remote configuration) and from iRules (for policy decisions).

Prerequisites

This article assumes you have the iControl snapin registered in your system and you have initialized it with a connection to your BIG-IP.  You will also need to “dot source” the script to set the functions in your current PowerShell runspace.  This can be done with the following commands

PS> Add-PSSnapin iControlSnapin
PS> Initialize-F5.iControl –Hostname bigip –Username user –Password pass
PS> . .\PsObjectMetaData.ps1

The DataGroup And Storage Format

Since I chose the storage to be a DataGroup, we’ll have to allow the user to decide the DataGroup to use.  We will also have to include the code to create that DataGroup if it doesn’t already exist.  The Set-F5.MetaDataClass function sets the internal script variable to the specified Data Group name and then issues a call to the LocalLB.Class.create_string_class method for that new DataGroup.  If this fails, in the situation where the DataGroup already exists, the trap catches the exception and then allows the script to proceed.  For exceptions that do not include the string “already exists”, the exception will not be trapped and the error will be displayed to the console.

   1: function Set-F5.MetaDataClass()

   2: {

   3:   param($Class = $null);

   4:   if ( $null -eq $Class )

   5:   {

   6:     Write-Host "Usage: Set-F5.MetaDataClass -Class classname";

   7:     return;

   8:   }

   9:   

  10:   $StringClass = New-Object -TypeName iControl.LocalLBClassStringClass;

  11:   $StringClass.name = $Class

  12:   $StringClass.members = (,$null);

  13:   

  14:   trap {

  15:     if ( $_.Exception.Message.Contains("already exists") ) {

  16:       # Class member already exists, so eat the error.

  17:       $script:DataGroup = $Class;

  18:       continue;

  19:     }

  20:   }

  21:   (Get-F5.iControl).LocalLBClass.create_string_class( (,$StringClass) );

  22:   

  23:   $script:DataGroup = $Class;

  24: }

I’ve also included a Get-F5.MetaDataClass function to retrieve the script level variable.

   1: function Get-F5.MetaDataClass()

   2: {

   3:   $script:DataGroup;

   4: }

The Data Format

Since I’m just using a simple String Data Group, I’ve decided to implement it as a character delimited string.  The fields in the data set will be the following

• Product – This will be the product the object is associated with (LTM, GTM, ASM, etc).
• Type – The specific type of object (VIP, Pool, PoolMember, VLAN, WideIP, etc).
• Name – The object name (Pool Name, VIP Name, WideIP Name, etc).
• Key – The name of the metadata attribute (Owner, Location, etc).
• Value – The value for the metadata (Joe, Seattle, etc).

I’ve included the New-F5.DataObject to assist in the management of the fields within a PowerShell object.  This will be used in subsequent functions.  This is also handy when returning a result set to PowerShell.  By having an object representation, you can use the formatting and selection functions to choose how you want your results displayed.

   1: function New-F5.DataObject()

   2: {

   3:   param(

   4:     $Product = $null,

   5:     $Type = $null,

   6:     $Name = $null,

   7:     $Key = $null,

   8:     $Value = $null

   9:   );

  10:   if ( ($Product -eq $null) -or ($Type -eq $null) -or ($Name -eq $null) -or ($Key -eq $null) -or ($Value -eq $null) )

  11:   {

  12:     Write-Host "Usage: New-F5.DataObject -Product prod -Type type -Name objname -Key datakey -Value datavalue";

  13:     return;

  14:   }

  15:   

  16:   $o = 1 | select Product, Type, Name, Key, Value;

  17:   $o.Product = $Product;

  18:   $o.Type = $Type;

  19:   $o.Name = $Name;

  20:   $o.Key = $Key;

  21:   $o.Value = $Value;

  22:   $o;

  23: }

Assign MetaData To An Object

The first thing you will likely want to do is to assign some metadata to an object.  This is done with the Set-F5.MetaData function.  It takes the properties defined in the Data Format section above.  The record format is formed and the iControl LocalLB.Class.add_string_class_member method is called with the LocalLB.Class.StringClass structure containing the Data Group entry description.  As above, we eat the error if the entry already exists.

   1: function Set-F5.MetaData()

   2: {

   3:   param(

   4:     $Product = $null,

   5:     $Type = $null,

   6:     $Name = $null,

   7:     $Key = $null,

   8:     $Value = $null

   9:   );

  10:   if ( ($Product -eq $null)

  11:     -or ($Type -eq $null)

  12:     -or ($Name -eq $null)

  13:     -or ($Key -eq $null)

  14:     -or ($Value -eq $null) )

  15:   {

  16:     Write-Host "Usage: Set-F5.MetaData -Product prod -Type type -Name name -Key key -Value value";

  17:     return;

  18:   }

  19:   

  20:   $StringClass = New-Object -TypeName iControl.LocalLBClassStringClass;

  21:   $StringClass.name = $script:Datagroup;

  22:   $StringClass.members = ( ,"${Product}:${Type}:${Name}:${Key}:${Value}");

  23:   

  24:   trap {

  25:     if ( $_.Exception.Message.Contains("already exists") ) {

  26:       # Class member already exists, so eat the error.

  27:       continue;

  28:     }

  29:   }

  30:   (Get-F5.iControl).LocalLBClass.add_string_class_member( (,$StringClass) );

  31: }

Get The MetaData Associated With An Object

After assigning some metadata to objects, you’ll want to have a way to query them out.  I implemented this with the Get-F5.MetaData function.  It takes as input the 5 fields defined above.  But, in this case, all of the parameters are optional.  The function queries the DataGroup and then filters the entries based on which parameters you passed in.  This way you can query all “VIP” objects, or all objects owned by “Joe”, or any combination of the parameters.  The New-F5.DataObject function is used here to create PowerShell objects for the specific records that match the query.  As you’ll see below in the Example section, this allows filtering and display formatting of the output.

   1: function Get-F5.MetaData()

   2: {

   3:   param(

   4:     $Product = $null,

   5:     $Type = $null,

   6:     $Name = $null,

   7:     $Key = $null,

   8:     $Value = $null

   9:   );

  10:   

  11:   $Objs = @();

  12:   # Build list

  13:   

  14:   $StringClassA = (Get-F5.iControl).LocalLBClass.get_string_class( (,$script:Datagroup) );

  15:   $StringClass = $StringClassA[0];

  16:   

  17:   $classname = $StringClass.name;

  18:   $members = $StringClass.members;

  19:   

  20:   for($i=0; $i -lt $members.length; $i++)

  21:   {

  22:     $tokens = $members[$i].Split($Separator);

  23:     if ( $tokens.Length -eq 5 )

  24:     {

  25:       $oProd = $tokens[0];

  26:       $oType = $tokens[1];

  27:       $oName = $tokens[2];

  28:       $oKey = $tokens[3];

  29:       $oValue = $tokens[4];

  30:       

  31:       $o = New-F5.DataObject -Product $oProd -Type $oType -Name $oName -Key $oKey -Value $oValue;

  32:       

  33:       $match = $true;

  34:  

  35:       # Process filter parameters

  36:       if ( ($Product -ne $null) -and ($oProd  -notlike $Product) ) { $match = $false; }

  37:       if ( ($Type    -ne $null) -and ($oType  -notlike $Type   ) ) { $match = $false; }

  38:       if ( ($Name    -ne $null) -and ($oName  -notlike $Name   ) ) { $match = $false; }

  39:       if ( ($Key     -ne $null) -and ($oKey   -notlike $Key    ) ) { $match = $false; }

  40:       if ( ($Value   -ne $null) -and ($oValue -notlike $Value  ) ) { $match = $false; }

  41:       

  42:       if ( $match ) { $Objs += (,$o); }

  43:     }

  44:   }

  45:   

  46:   $Objs;

  47: }

Removing The MetaData From An Object 

Getting and setting the metadata is all fine and dandy, but there will be times when you want to delete the metadata associated with an item.  This can be done with the Remove-F5.MetaData function.  For this example, all fields are required so that you don’t accidentally delete the metadata for all the objects in the system by omitting a certain parameter.  It would be trivial to allow a more specific removal based on one or more of the supplied parameters.  I’ll leave that as an exercise for the reader.

The Remove-F5.MetaData function, calls the iControl LocalLB.Class.delete_string_class_member method with the supplied StringClass definition.  Errors for record not found are trapped since you probably aren’t concerned if you are deleting objects that don’t exist.  All other errors are presented to the console through the standard PowerShell error processing.

   1: function Remove-F5.MetaData()

   2: {

   3:   param($Product = $null, $Type = $null, $Name = $null, $Key = $null, $Value = $null);

   4:   if ( ($Product -eq $null)

   5:     -or ($Type -eq $null)

   6:     -or ($Name -eq $null)

   7:     -or ($Key -eq $null)

   8:     -or ($Value -eq $null) )

   9:   {

  10:     Write-Host "Usage: Remove-F5.MetaData -Product prod -Type type -Name name -Key key -Value value";

  11:     return;

  12:   }

  13:   

  14:   $StringClass = New-Object -TypeName iControl.LocalLBClassStringClass;

  15:   $StringClass.name = $script:Datagroup;

  16:   $StringClass.members = ( ,"${Product}:${Type}:${Name}:${Key}:${Value}");

  17:  

  18:   trap {

  19:     if ( $_.Exception.Message.Contains("was not found") ) {

  20:       # Class member doesn't exists, so eat the error.

  21:       continue;

  22:     }

  23:   }

  24:   (Get-F5.iControl).LocalLBClass.delete_string_class_member( (,$StringClass) );

  25:   

  26: }

Example Usage

PS> Add-PSSnapin iControlSnapin
PS> Initialize-F5.iControl –Hostname bigip –Username user –Password pass
PS> Set-F5.MetaDataClass –Class ObjectMetaData
PS> Set-F5.MetaData –Product LTM –Type Pool –Name OWA_Servers_1 –Key Owner –Value Joe
PS> Set-F5.MetaData –Product LTM –Type Pool –Name OWA_Servers_1 –Key Location –Value SEA
PS> Set-F5.MetaData –Product LTM –Type Pool –Name OWA_Servers_2 –Key Owner –Value Joe
PS> Set-F5.MetaData –Product LTM –Type Pool –Name OWA_Servers_2 –Key Location –Value LAS
PS> Set-F5.MetaData –Product GTM –Type Wideip –Name wip_OWA –Key Owner –Value Fred
PS> Set-F5.MetaData –Product GTM –Type Wideip –Name wip_OWA –Key Location –Value LAS

PS> Get-F5.MetaData | Format-Table
Product      Type         Name            Key        Value
-------      ----         ----            ---        -----
GTM          Wideip       wip_OWA         Location   LAS
GTM          Wideip       wip_OWA         Owner      Joe
LTM          Pool         OWA_Servers_1   Location   SEA
LTM          Pool         OWA_Servers_1   Owner      Joe
LTM          Pool         OWA_Servers_2   Location   LAS
LTM          Pool         OWA_Servers_2   Owner      Fred

PS> Get-F5.MetaData –Key Owner –Value Joe | Format-Table
Product      Type         Name            Key        Value
-------      ----         ----            ---        -----
GTM          Wideip       wip_OWA         Owner      Joe
LTM          Wideip       wip_OWA         Owner      Joe

PS> Get-F5.MetaData –Key Location –Value SEA | select Type, Name
Type         Name
----         ----
Pool         OWA_Servers_1

You can view the source library for this article in the iControl CodeShare under PsObjectMetaData.

Related Articles on DevCentral

• Working with Datagroup members in Powershell - DevCentral - F5 ...
• how to use data group - DevCentral - F5 DevCentral > Forums ...
• How do I edit Datagroups via PowerShell - DevCentral - F5 ...
• iControl 101 #13 - Data Groups > DevCentral > F5 DevCentral ...
• Forcing a reload of External Data Groups within an iRule ...
• Validating Data Group (Class) References > DevCentral > F5 ...
• DevCentral Wiki: Power Shell
• PowerShell - Getting Started > DevCentral > F5 DevCentral > Tech Tips
• Authoring an F5 Management Pack PowerShell Agent Task in SCOM 2007 ...
• Creating An iControl PowerShell Monitoring Dashboard With Google ...