iControl 101 - #18 - Stream Profile

One of the common uses for an iRule is to search for a string and replace it with another string.  For advanced searches that involve complex string parsing or regular expressions, iRules are the way to go.  But if you are just looking at modifying a hostname in a URI or maybe even just changing all returned URI's from http to https for a ssl offloaded server farm, then the often overlooked Stream Profile is a highly optimized way to do this.  This article will discuss how to programmatically create a Stream Profile to replace all occurances of "http://" with "https://", apply it to a virtual server, query statistics, and undo the process.

Initialization

This article uses PowerShell and the iControl Cmdlets for PowerShell as the client environment for accessing the BIG-IP.  The following setup will be required for the examples contained in this article.  Since we will be applying the stream profile to a virtual server, I will create a variable containing the VirtualServer interface as well.

PS C:\> Add-PSSnapin iControlSnapin
PS C:\> Initialize-F5.iControl -hostname theboss -Username admin -Password admin
True
PS C:\> $ProfileStream = (Get-F5.iControl).ProfileStream
PS C:\> $VirtualServer = (Get-F5.iControl).LocalLBVirtualServer

Creating the Stream Profile

The first step in the process is to create the stream profile.  I'll first call the get_list() method to return the list of current stream profiles to make sure mine has a unique name.  Next, we'll call the create() method that takes as a parameter a string array of profile names.  In this example I'll use the profile name of "http2https" since I'm converting http URls to https.

PS C:\> $ProfileStream.get_list()
stream
foobar

PS C:\> $ProfileStream.create( (,"http2https") )
PS C:\> $ProfileStream.get_list()
stream
http2https
foobar

Querying profile details

A newly created profile will be setup with inherited values from it's parent profile.  To determine the source (search) and target (replace) strings you can use the get_source_string() and get_target_string() methods that take as parameters the list of profile names you wish to request.  In the example of a newly created profile, the values will be empty with the default_flag set to true.  The default_flag value of true indicates that the value being used for that parameter is derived from it's default profile.  To determine the default parent profile you can use the get_default_profile() method.  If you wish to change the default parent profile, you can use the set_default_profile() method but since I'm overwriting all of the parameters in this example the parent profile is not material.

PS C:\> $ProfileStream.get_source_string( (,"http2https") ) | format-list
value        :
default_flag : True

PS C:\> $ProfileStream.get_target_string( (,"http2https") ) | format-list
value        :
default_flag : True

PS C:\> $ProfileStream.get_default_profile( (,"http2https") )
stream

Modifying profile details

For my profile to convert "http://" to "https://" references, I'll need to set the source (search) and target (replacement) strings.  This can be done with the set_source_string() and set_target_string() methods that each takes as input a list of profile names as well as a list of LocalLB.ProfileString structures.  You'll notice I'm setting the default_flag to 0 (or false) to indicate that we won't be using the default value anymore but rather the supplied strings.

PS C:\> $ProfileString = new-object -typename iControl.LocalLBProfileString
PS C:\> $ProfileString.default_flag = 0
PS C:\> $ProfileString.value = "http://"
PS C:\> $ProfileStream.set_source_string( (,"http2https"), (,$ProfileString) )

PS C:\> $ProfileString.value = "https://"
PS C:\> $ProfileStream.set_target_string( (,"http2https"), (,$ProfileString) )

PS C:\> $ProfileStream.get_source_string( (,"http2https") ) | format-list
value        : http://
default_flag : False

PS C:\> $ProfileStream.get_target_string( (,"http2https") ) | fl
value        : https://
default_flag : False

Applying the Stream Profile to a Virtual Server

A Stream Profile all by itself doesn't do much without being applied to a Virtual Server.  The add_profile() method in the LocalLB.VirtualServer interface is used to apply the newly configured Stream Profile to a Virtual Server.  This method takes as input a list of profile names as well as a 2-d array of LocalLB.VirtualServer.VirtualServerProfile structures containing the context in which the profile is applied as well as the name of the profile.  In this case I'm going to apply the context to all (as opposed to just client or just server) and add my http2https profile to the Virtual Server "xpbert-http".  I'll then call the VirtualServer.get_profile() method to make sure it took and was applied properly.

PS C:\> $VirtualServerProfile = new-object -typename iControl.LocalLBVirtualServerVirtualServerProfile
PS C:\> $VirtualserverProfile.profile_context = "PROFILE_CONTEXT_TYPE_ALL"
PS C:\> $VirtualserverProfile.profile_name = "http2https"
PS C:\> [iControl.LocalLBVirtualServerVirtualServerProfile[][]]$VSProfAofA =
>> @(, [iControl.LocalLBVirtualServerVirtualServerProfile[]]@(,$VirtualServerProfile))
PS C:\> $VirtualServer.add_profile( (,"xpbert-http"), $VSProfAofA)

PS C:\> $VirtualServer.get_profile( (,"xpbert-http") )
                           profile_type                         profile_context profile_name
                           ------------                         --------------- ------------
                    PROFILE_TYPE_STREAM                PROFILE_CONTEXT_TYPE_ALL http2https
                PROFILE_TYPE_STATISTICS                PROFILE_CONTEXT_TYPE_ALL maintenance_window
                      PROFILE_TYPE_HTTP                PROFILE_CONTEXT_TYPE_ALL my_http
                       PROFILE_TYPE_TCP                PROFILE_CONTEXT_TYPE_ALL tcp

Testing the stream profile

Now that the new profile has been applied to the Virtual Server, it's time to test it out.  Below is the file /file.html that I've loaded to my webserver.  In it I have a single HTTP URI that I'm looking at getting converted.  You should note that the Stream Profile will handle as many string replacements as you can hand to it.

URL: http://testvip/file.html

From a client machine I'll run the HTTP Query tool "curl" to query the test.html page though the virtual server.  As you see, the references to "http://" have been replaced with "https://" as expected.

PS C:\> curl http://xpbert-http/test.html
URL: https://testvip/file.html

Querying Statistics

If you want to keep track of how hard the Stream Profile is working, you can query the number of stream replacements made by calling the get_statistics() method in the ProfileStream interface.  This method takes as input a list of profile names and returns a ProfileStreamStatistics structure.  The following code queries the statistics and breaks apart the structure to list out all of the data showing that one replacement has been made.

PS C:\> $ProfileStreamStatistics = $ProfileStream.get_statistics( (,"http2https") )
PS C:\> $ProfileStreamStatistics.time_stamp
year   : 2008
month  : 6
day    : 19
hour   : 14
minute : 47
second : 49

PS C:\> $statistics = $ProfileStreamStatistics.statistics
PS C:\> $statistics.length
1
PS C:\> $statistics[0].profile_name
http2https

PS C:\> $stats = $statistics[0].statistics
PS C:\> $stats.length
1
PS C:\> $stats[0].type
STATISTIC_STREAM_REPLACEMENTS

PS C:\> $stats[0].value | format-list
high : 0
low  : 1

Cleaning things up

It's always good to know how to clean up house after you make a mess, so I'll go ahead and show you how to undo the above examples.  First, we'll want to remove the "http2https" profile from the Virtual Server. This can be done with the remove_profile() method in the VirtualServer interface.  This method takes the same parameters as the add_profile() method used above so I'll just reuse the variables I already created.

After the profile has been removed from the Virtual Server, you can safely delete the profile from the configuration.  This is done with the delete_profile() method.  Just pass in a list of profile names you want deleted and you are all set.

Oh, and just like some of the other interfaces I've covered in this series of articles, there is a "delete all" method to remove all of the Stream Profiles.  The delete_all_profiles() will do just what it's name implies, it will delete all of the stream profiles on the system.  If you feel the urge to use this method, do so at your own risk!

PS C:\> $VirtualServer.remove_profile( (,"xpbert-http"), $VSProfAofA)

PS C:\> $ProfileStream.delete_profile( (,"http2https") )

Conclusion

So, the next time you think about writing an iRule to do a simple string replacement, take a second to think about using the Stream Profile.  It's very simple to setup has significantly lower overhead than an iRule equivalent.