ARX File Change

Problem this snippet solves:

This example shows how to retrieve the set of ARX file changes that occurred in a managed volume between two snapshots.

This script is an example of how to use the iControl interfaces provided by an ARX to retrieve the set of file changes that occurred in a managed volume between two snapshots. The namespace name, volume path and notification rule name are specified by the user and the script will find the file changes that occurred between each set of snapshots that have been created for the notification rule.

How to use this snippet:

ARXFileChange.pl --url  --user  --pass  --ns  --vol  --rule

Prerequisites

SOAP::Lite perl module 2. An F5 ARX system configured with:
- A namespace and managed volume
- A notification rule in that managed volume
- Two snapshots created for the notification rule where file changes such as renames, removes, and creates took place between the creation time of the first snapshot and the creation time of the second snapshot.
- Management access must be permitted for HTTP-API and HTTPS-API services.

Code :

#!/usr/bin/perl
#-------------------------------------------------------------------------------
# The contents of this file are subject to the "END USER LICENSE AGREEMENT 
# FOR F5 Software Development Kit for iControl"; you may not use this file 
# except in compliance with the License. The License is included in the 
# iControl Software Development Kit.
#
# Software distributed under the License is distributed on an "AS IS"
# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
# the License for the specific language governing rights and limitations
# under the License.
#
# The Original Code is iControl Code and related documentation
# distributed by F5.
#
# The Initial Developer of the Original Code is F5 Networks,
# Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 1996-2010
# F5 Networks, Inc. All Rights Reserved.  iControl (TM) is a registered 
# trademark of F5 Networks, Inc.
#
# Alternatively, the contents of this file may be used under the terms
# of the GNU General Public License (the "GPL"), in which case the
# provisions of GPL are applicable instead of those above.  If you wish
# to allow use of your version of this file only under the terms of the
# GPL and not to allow others to use your version of this file under the
# License, indicate your decision by deleting the provisions above and
# replace them with the notice and other provisions required by the GPL.
# If you do not delete the provisions above, a recipient may use your
# version of this file under either the License or the GPL.
#-------------------------------------------------------------------------------
#
# Description:
#
# This script is an example of how to use the iControl interfaces provided by
# an ARX to retrieve the set of file changes that occurred in a managed volume
# between two snapshots.  The namespace name, volume path and notification rule
# name are specified by the user and the script will find the file changes
# that occurred between each set of snapshots that have been created for the
# notification rule.
#
# Usage: ARXFileChange.pl --url  --user  --pass 
#                         --ns  --vol  --rule 
#
# Prerequisites:
#
# This script requires the following:
#
#   * SOAP::Lite perl module
#   * An F5 ARX system configured with:
#     - A namespace and managed volume
#     - A notification rule in that managed volume
#     - Two snapshots created for the notification rule where file changes
#       such as renames, removes, and creates took place between the
#       creation time of the first snapshot and the creation time of the
#       second snapshot.
#     - Management access must be permitted for HTTP-API and HTTPS-API
#       services.
#
# For more information on ARX configuration, please consult the
# documentation that was provided with your ARX system.
#-------------------------------------------------------------------------------

# SOAP::Lite lets us send SOAP requests and parse them
use SOAP::Lite
    autotype => 0,
    default_ns => 'urn:iControl';

# If you need to debug problems with your script, you can use the +trace 
# option with SOAP::Lite and it will print the XML sent to and received from
# the server:
#
# use SOAP::Lite
#     autotype => 0,
#     default_ns => 'urn:iControl' + trace;

# Getopt::Long lets us easily parse command line options
use Getopt::Long;

# POST::strftime let us format a number of seconds into a date/time string
use POSIX qw(strftime);

#-------------------------------------------------------------------------------
# Main program logic
#-------------------------------------------------------------------------------

our ($url, $user, $pass, $namespaceName, $volumePath, $notificationRule);

# Load command line options - if the load fails, then we print the usage
# instructions and exit.
if (!GetOptions("url=s" =>  \$url,
                "user=s" => \$user,
                "pass=s" => \$pass,
                "ns=s" => \$namespaceName,
                "vol=s" => \$volumePath,
                "rule=s" => \$notificationRule)) {
    usage();
    exit(1);
}

# If any arguments were skipped, print the usage instructions and exit.
if (!defined $url || 
    !defined $user || 
    !defined $pass ||
    !defined $namespaceName ||
    !defined $volumePath ||
    !defined $notificationRule) {
    usage();
    exit(1);
}

# Build a security header for authentication
our $securityHeader = getSecurityHeader($user, $pass);

# The service path for interface "Interface" is the following:
#
# https://:/api/services/Interface
#
# In this case, we are going to use the FileChangeNotification
# interface.
my $fcnServiceUrl = $url . "/api/services/FileChangeNotification";

# In order for SOAP to access a web service, it needs to read the WSDL
# for the interface you want to use.  The WSDL file for an interface
# called "Interface" is available via http/https on the ARX at:
#
# https://:/api/services/Interface?wsdl
#
# If you need a WSDL 2.0 version, that is also available at:
#
# https://:/arx-api/wsdl/Interface.wsdl2
#
# In this case, we're using the FileChangeNotification interface and we're
# interested in using the WSDL 1.1 version.
my $fcnWsdlUrl = $fcnServiceUrl . "?wsdl";

# Now we build our SOAP::Lite object using the service and WSDL URLs
our $fcnSoap = SOAP::Lite->new(proxy   => $fcnServiceUrl,
                               service => $fcnWsdlUrl);
 
# Get a list of File Change Notification snapshots that exist on the ARX.
my $fcnSoapResult = $fcnSoap->get_snapshots(
                              SOAP::Data->name('namespace')->value($namespaceName),
                              SOAP::Data->name('volume')->value($volumePath),
                              SOAP::Data->name('rule')->value($notificationRule),
                              $securityHeader);

# Check if there were any faults encountered during the operation.  We find
# this by checking if the fault member of the result object is set.  If there
# is a fault, then we can print the detailed fault text using the faultstring
# member of the result object.
if ($fcnSoapResult->fault) {
    printf(STDERR "SOAP fault encountered:\n");
    printf(STDERR "%s\n", $fcnSoapResult->faultstring);
    exit(1);
}

# The get_list() call did not fail, so we build a list of fcn names from the
# result.  Note that the full result is a concatenation of the result and
# paramsout members of the SOAP result object.
my @snapshotList = ($fcnSoapResult->result, 
                    $fcnSoapResult->paramsout);

# If we didn't find any snapshots for the notification rule, then we can just
# exit.
if (@snapshotList == 0) {
    print "No snapshots were found.\n";
    exit(0);
}

# We can now print the list of snapshots
print "\nNamespace:         ", $namespaceName, "\n";
print "Volume:            ", $volumePath, "\n";
print "Notification Rule: ", $notificationRule, "\n";
print "Snapshot list:     ";
for (my $i = 0; $snapshotList[$i]; $i++) {
    if ($i > 0) {
        print ", ";
    }
    print $snapshotList[$i];
}
print "\n\n";

# Now we can iterate through each snapshot and get a list of file changes
# compared to the previous snapshot.  In order to do this, we need to retrieve
# information about all of the snapshots, specifically their creation times.
$fcnSoapResult = $fcnSoap->get_snapshot_definition(
                          SOAP::Data->name('namespace')->value($namespaceName),
                          SOAP::Data->name('volume')->value($volumePath),
                          SOAP::Data->name('rule')->value($notificationRule),
                          SOAP::Data->name('snapshots')->value(@snapshotList),
                          $securityHeader);

# Check for faults
if ($fcnSoapResult->fault) {
    printf(STDERR "SOAP fault encountered:\n");
    printf(STDERR "%s\n", $fcnSoapResult->faultstring);
    exit(1);
}

my @snapshotInfoList = ($fcnSoapResult->result, 
                        $fcnSoapResult->paramsout);

# For the first set of changes, the fromSnapshot value is just 0 because we
# want to do a full volume comparison.  Note that if you have a lot of files in
# the volume, this could take a long time.  As we iterate through each
# snapshot, we record the last creation time so that we can use it as the
# fromSnapshot time in the next call.
my $lastSnapshotTime = 0;
foreach my $snapshotInfo (@snapshotInfoList) {
    my $snapshotTime = $snapshotInfo->{'create_time'};
    printAllFileChanges($lastSnapshotTime,
                     $snapshotTime);
    $lastSnapshotTime = $snapshotTime;
}

#-------------------------------------------------------------------------------
# End of main program logic
#-------------------------------------------------------------------------------


#-------------------------------------------------------------------------------
# sub usage
#-------------------------------------------------------------------------------
sub usage
{
    print "\nUsage: ARXFileChange.pl --url  --user  --pass \n";
    print "                          --ns  --vol  --rule \n";
    print "\n";
    print "Argument  Description\n";
    print "--------  -----------\n";
    print "--url     The base URL of the web service on the ARX. Both http and https\n";
    print "          are supported (https is recommended for better security). The format\n";
    print "          is:\n";
    print "\n";
    print "          http(s)://:\n";
    print "\n";
    print "          : DNS resolvable hostname or IP address\n";
    print "          :     83 for http or 843 for https\n";
    print "\n";
    print "--user    The username for authentication.\n";
    print "--pass    The password for authentication.\n";
    print "--ns      The namespace that contains the volume and rule.\n";
    print "--vol     The managed volume that contains the notification rule.\n";
    print "--rule    The name of the notification rule used to track the file\n";
    print "          changes.\n";
    print "\n";
}

#-------------------------------------------------------------------------------
# sub getSecurityHeader(user, pass)
#
# This subroutine builds a security header that will be used for
# authentication.  This type of security header is required for all calls to
# iControl::ARX interfaces, so it makes sense to have this subroutine stored in
# a library for common access.
#-------------------------------------------------------------------------------
sub getSecurityHeader
{
    my $user = shift;
    my $pass = shift;
    my $now = time();
    my $then = time() + 60;
    my $created = strftime("%Y-%m-%dT%H:%M:%S", gmtime($now)) . 'Z';
    my $expires = strftime("%Y-%m-%dT%H:%M:%S", gmtime($then)) . 'Z';

    my $secExt = 'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd';
    my $secUtil = 'http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd';
    my $securityHeader = SOAP::Header->name("wsse:Security")->attr(
            {
                'xmlns:wsse'=> $secExt,
                'xmlns:wsu'=> $secUtil
            }
    );
    my $timestamp = SOAP::Data->name("wsu:Timestamp" =>
                            \SOAP::Data->value(
                                SOAP::Data->name('wsu:Created')->value($created)
                                                               ->type(''),
                                SOAP::Data->name('wsu:Expires')->value($expires)
                                                               ->type('')));
    my $usernameTokenType = 
        "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText";
    my $usernameToken = SOAP::Data->name("wsse:UsernameToken" =>
                            \SOAP::Data->value(
                                SOAP::Data->name('wsse:Username')->value($user)
                                                                 ->type(''),
                                SOAP::Data->name('wsse:Password')->value($pass)
                                                                 ->type('')
                                                                 ->attr({'Type'=>$usernameTokenType})));

    $securityHeader->value(\SOAP::Data->value($timestamp, $usernameToken));

    return $securityHeader;
}

#-------------------------------------------------------------------------------
# sub printOneFileChange(fileChange)
#
# This subroutine prints a hash that contains file changes.
#-------------------------------------------------------------------------------
sub printOneFileChange
{
    my $fileChange = shift;

    # Each file change object is a simple hash that contains the following:
    #
    # Name      Value
    # ----      -----
    # type      The type of file change.
    # src_path  The original path of the object.
    # dst_path  The new path of the object.
    # src_link  The original target that this object pointed at (NFS hard link)
    # dst_link  The new target that object points at (NFS hard link)
    #
    print "Type: ", $fileChange->{'type'}, "\n";
    print "Path (src -> dst): ", $fileChange->{'src_path'}, " -> ", $fileChange->{'dst_path'},
                    "\n";
    print "Link (src -> dst): ", $fileChange->{'src_link'}, " -> ", $fileChange->{'dst_link'}, 
                    "\n\n";

}

#-------------------------------------------------------------------------------
# sub printAllFileChanges(fromSnapshot, toSnapshot)
#
# This subroutine prints all file changes that took place between the creation
# times for fromSnapshot and toSnapshot.
#-------------------------------------------------------------------------------
sub printAllFileChanges
{
    my $fromSnapshot = shift;
    my $toSnapshot = shift;

    my $fromSnapshotStr = "None";
    if ($fromSnapshot > 0) {
        $fromSnapshotStr = strftime("%Y-%m-%d %H:%M:%S", gmtime($fromSnapshot));
    }

    print "----------------------------------------------------------------------\n";
    print "Start (from): ", $fromSnapshotStr, "\n";
    print "End (to):     ", strftime("%Y-%m-%d %H:%M:%S", gmtime($toSnapshot)), "\n";
    print "----------------------------------------------------------------------\n\n";
    
    my $fcnSoapResult = $fcnSoap->start_notifications(
                                  SOAP::Data->name('namespace')->value($namespaceName),
                                  SOAP::Data->name('volume')->value($volumePath),
                                  SOAP::Data->name('rule')->value($notificationRule),
                                  SOAP::Data->name('from_snapshot')->value($fromSnapshot),
                                  SOAP::Data->name('to_snapshot')->value($toSnapshot),
                                  SOAP::Data->name('all_files')->value(true),
                                  $securityHeader);
    # Check for faults
    if ($fcnSoapResult->fault) {
        printf(STDERR "SOAP fault encountered:\n");
        printf(STDERR "%s\n", $fcnSoapResult->faultstring);
        exit(1);
    }

    # The response from FileChangeNotification::start_notifications is a unique
    # session identifier.  We get batches of file changes each time we call
    # get_notifications, so the session identifier tells the ARX which session
    # it's dealing with.
    my $sessionId = $fcnSoapResult->result;

    # Since we're dealing with batches, we need a bookmark to keep track of
    # where we are in the list of file changes.  We provide that as a change
    # index to the ARX to let it know where to start for each call.
    my $changeIdx = "";

    my $totalCount = 0;

    # We need to keep track of whether there are more file changes
    # to retrieve.
    my $more = "true";
    while ($more eq "true") {
        $fcnSoapResult = $fcnSoap->get_notifications(
                                   SOAP::Data->name('session_id')->value($sessionId),
                                   SOAP::Data->name('count')->value(25),
                                   SOAP::Data->name('change_idx')->value($changeIdx),
                                   $securityHeader);
        # Check for faults
        if ($fcnSoapResult->fault) {
            printf(STDERR "SOAP fault encountered:\n");
            printf(STDERR "%s\n", $fcnSoapResult->faultstring);
            exit(1);
        }

        # The response is a hash that contains the following name, value
        # pairs:
        #
        # Name         Value
        # ----         -----
        # more         A boolean value that indicates if there are more 
        #              file changes left to retrieve.
        #
        # change_idx   A bookmark that indicates the next file change
        #              to start with.
        #
        # count        The number of file changes returned in this call.
        #
        # changes      A sequence of file change objects.

        my $fcResponse = $fcnSoapResult->result;
        $more = $fcResponse->{'more'};
        $changeIdx = $fcResponse->{'change_idx'};
        my $count = $fcResponse->{'count'};

        $totalCount = $totalCount + $count;

        # If we received at least one file change object, then we will process
        # them.  We can check if we received more than one object by checking
        # if the type of the reference is "ARRAY".  If it's not an array, then
        # we just have a single object.
        if ($count > 0) {
            my $fileChangesRef = $fcResponse->{'changes'};
            if (ref($fileChangesRef) eq 'ARRAY') {
                my @fileChanges = @{$fileChangesRef};
                foreach my $fileChange (@fileChanges) {
                    printOneFileChange($fileChange);
                }
            } else {
                printOneFileChange($fileChangesRef);
            }
        }
    }

    if ($totalCount == 0) {
        print "No changes found during this period.\n";
    }

    # In order to complete the session, we have to call finish_notifications so
    # we can inform the ARX that we are done and it can clean up the session.
    $fcnSoapResult = $fcnSoap->finish_notifications(
                               SOAP::Data->name('session_id')->value($sessionId),
                               $securityHeader);

}

Updated Jun 06, 2023

Version 2.0

Cirrus

Joined May 05, 2019

View Profile

ARX File Change

ABOUT DEVCENTRAL

RESOURCES

SUPPORT

PARTNERS