ASM All Policy Export
Problem this snippet solves:
ASM all policy export v1.4 - 15 Aug 2007
Writes out the active policy for each ASM web application to a gzip'd tar archive (see usage subrouting for details)
Tested on BIG-IP ASM versions: 9.2.4, 9.4.0, 9.4.1
Aaron Hooley - LODOGA Security Limited (hooleylists at gmail dot com)
Code :
#!/usr/bin/perl # # ASM policy export # v1.4 - 15 Aug 2007 # # Writes out the active policy for each ASM web application to a gzip'd tar archive # (see usage subrouting for details) # # Tested on BIG-IP ASM versions: 9.2.4, 9.4.0, 9.4.1 # # Aaron Hooley - LODOGA Security Limited (hooleylists@gmail.com) # use strict; use warnings; use DBI; use lib '/ts/packages/'; use GenUtils qw//; use POSIX qw/strftime/; use File::Path; # pass_through: all unknown options are left in @ARGV use Getopt::Long qw(:config pass_through); use File::Basename; $SIG{PIPE} = 'IGNORE'; use constant DEBUG => 1; # 1 = log debug messages to standard out; 0 = don't use debug use constant OUTPUT_DIR => '/shared/asm_all_policy_export/output/'; # Default output directory for finalized compressed tar archive containing each output file # options below these shouldn't need to be modified use constant UCS_DIR => '/var/local/ucs/'; # If writing output file as a UCS for access to files via the GUI, use the UCS directory use constant TMP_DIR => '/shared/tmp/'; # Temporary directory to write the individual policies to use constant VERSION_FILE => '/VERSION'; # File which contains BIG-IP version info use constant LOCK_FILE => '/ts/lock/asm_all_policy_export.lock'; # Lock file used to ensure only one copy of the script is executing at a time use constant USERS_CFG_FILE => '/ts/common/users.cfg'; # System file containing user and database details use constant VERSION_FILE => '/VERSION'; # System file which contains BIG-IP version info use constant DRIVER => GenUtils::cfg_get_config_item(USERS_CFG_FILE, 'DATABASE', 'Driver'); use constant DCC => GenUtils::cfg_get_config_item(USERS_CFG_FILE, 'DATABASE', 'Name'); use constant PLC => GenUtils::cfg_get_config_item(USERS_CFG_FILE, 'POLICY_DATABASE', 'Name'); use constant DB_USER => GenUtils::cfg_get_config_item(USERS_CFG_FILE, 'DATABASE', 'User'); use constant F5_EXPORT_SCRIPT => '/ts/tools/import_export_policy.pl'; # Stop this script if ASM is not licensed check_license(); # Create a lock so only one instance of script runs at a time my ($lock_sub,$unlock_sub,$cleanup_lock_sub) = GenUtils::lock_factory(LOCK_FILE) or fatal_error("Cannot open lockfile '".LOCK_FILE."': $!"); $lock_sub->(); # Stop the script if the F5 single policy export script doesn't exist if (not (-e F5_EXPORT_SCRIPT)){ fatal_error(qq{Cannot access F5 export script: }.F5_EXPORT_SCRIPT."\n\n"); } # Get command line options if the script was called correctly # Return variables (and their default values) are: debug ('not_set'), output_path ('not_set'), output_file ('not_set'), use_ucs_dir (0) # As the handle_commandline_arguments subroutine handles the help flag itself, we don't need to consider it my %args = handle_commandline_arguments(); # Set the debug level to the higher level of the constant and command line interface (CLI) option. See usage for debug levels. my $debug = $args{debug}; if ($debug eq 'not_set') { $debug = DEBUG; } print "Verbose debug enabled\n" if $debug > 1; # Get MySQL password (and verify the BIG-IP version is supported by this script) my $mysql_pass = get_mysql_password(); # Verify required config items were read successfully if (not defined DRIVER or not defined DCC or not defined PLC or not defined DB_USER or not defined $mysql_pass){ fatal_error(qq{Problems accessing database users configuration file\n\n}); } # Get current timestamp for output filename, formatted as %Y-%m-%d_%H-%M-%S my $date = get_date(); # Set output directory according to CLI option or default from constant. # If use_ucs_dir option is enabled, ignore any other setting for the path of the output and use UCS_DIR constant value my $use_ucs_dir = 0; my $output_path; # Check if the CLI option for using the UCS dir is set if ($args{use_ucs_dir}==1){ # It is; so we're writing out the output file to the UCS directory $use_ucs_dir = 1; $output_path = UCS_DIR; } elsif ($args{output_path} ne 'not_set' and length($args{output_path}) > 1){ # UCS option wasn't set, and another output path was set via CLI option $output_path= $args{output_path}; } else { # UCS option wasn't set and no output path was specified in CLI option, so use the default path from the constant $output_path = OUTPUT_DIR; } # Add a trailing slash to the output path if it's not there already if (substr($output_path,-1,1) ne "/") { $output_path .= "/"; } # Create the output path if it doesn't exist already (skip this for the UCS path as it has to already exist) if (not $use_ucs_dir){ make_output_path($output_path); } # Set the output fully qualified filename according to CLI if set, else default to the constant value my $output_file; if ($args{output_file} eq 'not_set' or length($args{output_file})==0){ # Output file wasn't specified in CLI option, so use the default $output_file = build_output_filename_with_path($output_path, $date); } else { # An output file was specified in the CLI option so use that for the filename $output_file = $output_path.$args{output_file}; } # Append .ucs to the filename if using the UCS path for output, so the GUI will present it in the archive list. # Check that the filename doesn't already end in .ucs if ($use_ucs_dir==1 and not ($output_file =~ /.*?\.ucs$/i)){ $output_file .= ".ucs"; } # Check if mysql is up. If not, start it my $restore_mysql_to_prior_state = check_mysql(); # Run the export subroutine export(); # Stop mysql if it wasn't running before we started $restore_mysql_to_prior_state->(); # Release the lock $unlock_sub->(); $cleanup_lock_sub->(); exit; # # Main subroutine which exports the active policies and writes the output to a compressed tar archive # sub export { # Initialize an array to store the full path/filename for the individual exported policies my @exported_filenames; # SQL query to retrieve the names of each web app, the active policy name and the active policy ID my $sql_statement = qq{ SELECT a.domain_name AS 'Web App', p.name AS 'Policy', p.id AS 'POLICY.ID' FROM DCC.ACCOUNTS a INNER JOIN PLC.PL_POLICIES p ON a.active_policy_id = p.id; }; # Connect to mysql database my $dbh = connect_to_db($mysql_pass); # Prepare the SQL statement my $sth = $dbh->prepare($sql_statement) or fatal_error(qq{Problems preparing sql: |$sql_statement|: $DBI::errstr\n\n}); # Execute the SQL statement $sth->execute() or fatal_error(qq{Problems executing sql: |$sql_statement|: $DBI::errstr\n\n}); print $sth->rows()." active policies in resultset\n" if $debug > 1; # Loop through each web app and export the active policy using the F5 export policy script while (my @row = $sth->fetchrow_array()) { # Parse the three column values for this record my ($webapp, $policy_name, $policy_id) = @row; print "\$webapp = $webapp; \$policy_name = $policy_name, \$policy_id = $policy_id\n" if $debug > 1; # Build file name for a single policy my $single_policy = $webapp.'_'.$policy_name.'_'.$date.'.plc'; # Add current single policy file to the array of files push (@exported_filenames, $single_policy); # Call the F5 supplied policy export command: # /ts/tools/import_export_policy.pl -a ACTION -p POLICY_ID -f OUTPUT_FILE # example: /ts/tools/import_export_policy.pl -a export -p 1 -f /shared/tmp/policy_export/my-web-app_policy-name_2007-08-01_12-00-00.plc my @export_cmd = ( 'nice -n 19', F5_EXPORT_SCRIPT, '-a export', '-p', $policy_id, '-f', TMP_DIR.$single_policy); print "@export_cmd \n" if $debug > 1; # Run the export command for a single policy my $out = `@export_cmd`; print "policy export result: $out\n" if $debug > 1; if (not ($out =~ "successfully")){ fatal_error( qq/Failed to export policy using '@export_cmd'\n\n/ ); } } # tar up the files (uses 'nice -n 19' to lower the CPU priority for the tar'ing) my @tar_cmd = ( 'nice -n 19 tar', '-z', '-c', '-f', $output_file, '-C', TMP_DIR, @exported_filenames); print @tar_cmd if $debug > 1; system("@tar_cmd") == 0 or fatal_error( qq/Failed to create tar using '@tar_cmd'/ ); # Clean up temp files cleanup(@exported_filenames); # Report success with the output file name print "Exported ".$sth->rows()." policies to " . $output_file . "\n" if $debug > 0; # Log a successful message to syslog GenUtils::ts_syslog( 'info', 'All active ASM policies exported to %s', $output_file ); } # # Other subroutines # sub get_mysql_password { # Verify BIG-IP version is supported and get MySQL password based on version my $bigip_version = get_bigip_version(); # Initialize a variable to store the MySQL password. MySQL password is different between (9.2.x/9.3.x) and 9.4.x+ my $mysql_pass; # If version is 9.4.x use F5 function get_mysql_password if (substr($bigip_version,0,3) eq "9.4"){ $mysql_pass = GenUtils::get_mysql_password(); } elsif (substr($bigip_version,0,3) eq "9.2" or substr($bigip_version,0,3) eq "9.3"){ # For 9.2.x and 9.3.x, get it from the users.cfg file $mysql_pass = GenUtils::cfg_get_config_item(USERS_CFG_FILE, 'DATABASE', 'Password'); } else { # Parsed version was unknown, so kill the script fatal_error(qq{Version $bigip_version unknown\n\n}); } print "\$mysql_pass: $mysql_pass\n" if $debug > 1; return $mysql_pass; } sub build_output_filename_with_path { # Build the path/filename for the output file. Expect the path and date as parameters. my ($output_path, $date) = @_; # Get BIG-IP hostname my $bigip_hostname = get_bigip_hostname(); # Build name of output tar ball with fully qualified path return ($output_path . $bigip_hostname."_policy-export_$date.tgz"); } sub get_bigip_hostname { # Get the BIG-IP hostname my $hostname = `hostname`; if (not defined $hostname){ fatal_error(qq{Cannot get BIG-IP hostname using hostname command}); } else { chomp($hostname); return $hostname; } } sub get_bigip_version { # Get the BIG-IP version to determine how to get MySQL root password my $version; # Check if version file exists if (-f VERSION_FILE){ # Open the version file or stop script open (my $fh, '<', VERSION_FILE) or fatal_error(qq{get_bigip_version: cannot open version file }.VERSION_FILE); # Read in the contents of the version file while (<$fh>){ # Do case insensitive search for "version: xxx" and get the match from a backreference ($1) if ($_ =~ /^version:\s+(\S+)$/i){ $version = $1; print "get_bigip_version: \$version: $version\n" if $debug > 1; } } if (not $version){ # Kill the script as we couldn't parse the version fatal_error(qq{get_bigip_version: Could not parse version from: }.VERSION_FILE."\n"); } close $fh; } else { fatal_error(qq{get_bigip_version: version file does not exist: }.VERSION_FILE); } return ("$version"); } sub cleanup { # Clean up temp files used during policy export print "Cleaning up\n" if $debug > 1; my (@files_to_cleanup) = @_; foreach my $file (@files_to_cleanup) { if ( -e TMP_DIR.$file ) { print "Deleting temp file: " . TMP_DIR . $file . "\n" if $debug > 1; unlink TMP_DIR.$file or warn qq/Failed to remove file '/. TMP_DIR . $file . qq/': $!/; } } } sub check_mysql { # Is mysql running? my $mysql_was_up = GenUtils::check_mysql(); return sub { GenUtils::stop_mysql() unless $mysql_was_up; }; } sub connect_to_db { # Open database connection #my $mysql_pass = @_; my $database = 'DCC'; my $dbh = DBI->connect( "DBI:".DRIVER.":".$database, DB_USER, $mysql_pass, { PrintError => 0, RaiseError => 0, FetchHashKeyName => 'NAME_lc', } ) or fatal_error( "Cannot connect to $database: $DBI::errstr\n" ); return $dbh; } sub check_license { # Kill script if ASM isn't licensed if ( not GenUtils::is_ts_licensed() ) { GenUtils::ts_syslog( 'info', 'ASM support is not enabled' ); fatal_error(qq{ASM is not licensed. Aborting script.\n}); } } sub get_date { # Get date in format year-month-day_hour-minute-second return POSIX::strftime( "%Y-%m-%d_%H-%M-%S", localtime ); } sub make_output_path { # Create the output path if it doesn't exist already my ($output_path) = @_; if (not (-d $output_path)){ eval { mkpath($output_path) }; if ($@) { fatal_error(qq{Could not create path $output_path: $@\n}); } else { print "make_output_path: made path $output_path" if $debug > 1; } } } sub fatal_error { # Write error details to syslog and exit script my ($msg,%args) = @_; GenUtils::ts_syslog('err', "%s", $msg); print $msg; exit(1); } sub handle_commandline_arguments { # Set some default values for the options my %arguments = (); $arguments{debug} = 'not_set'; $arguments{output_path} = 'not_set'; $arguments{output_file} = 'not_set'; $arguments{use_ucs_dir} = 0; $arguments{help} = 0; # Parse the options from the command line my $options = GetOptions ( "debug|d:i" => \$arguments{debug}, # match for debug or d as an integer "output_path|p:s" => \$arguments{output_path}, # match on output_path or p as a string "output_filename|f:s" => \$arguments{output_file}, # match on output_file or f as a string "use_ucs_directory|u" => \$arguments{use_ucs_dir}, # match on use_ucs_directory or u, binary type "help|h" => \$arguments{help}, # match on help or h, binary type ); # Debug printing of options if ($arguments{debug} ne 'not_set' and $arguments{debug} > 1){ while (my ($key, $value) = each(%arguments)){ print $key.": ".$value."\n"; } if (@ARGV){ print "extra args found: \n"; print "@ARGV\n"; } } # print the usage info if the options couldn't be parsed, help option was enabled, unknown options were used, or the output_path doesn't start with a / if ($arguments{help}){ print "Help option set\n".usage(); } elsif (@ARGV) { print "Unparsed options set: \n"; print "@ARGV\n"; print usage(); } elsif ($arguments{output_path} ne "not_set" and substr($arguments{output_path},0,1) ne "/"){ print "output_file value must start with a forward slash\n".usage(); } else { return %arguments; } } sub usage { # stop the script if this sub is called, as we're just printing the options for calling the script # get script name my ($script_name, $gar, $bage) = File::Basename::fileparse($0); die (qq{ -------------------------------------------------------------- NAME $script_name - ASM all policy export Write out each active policy to a single compressed tar archive DESCRIPTION $script_name - Write out the policy that is marked as active for each ASM web application. Each policy is exported from the internal database and written to a combined gzip compressed tar archive. The output path and filename can be configured using command line options or in constants defined in the script. OPTIONS -d --debug 0 - print nothing to standard output 1 - print output filenames to standard output 2 - print verbose logging to standard ouput Note: values greater than 2 are considered as 2 Default: 1 -p --output_path String containing the directory to write the output file. If the path does not exist it will be created. Must start with a leading forward slash. Note: this option is ignored if the -u/--use_ucs_dir option is enabled Default: }.OUTPUT_DIR.qq{ -f --output_filename String containing the name of the output file. Note: if the -u/--use_ucs_dir option is enabled, the file extension ".ucs" will be appended to the filename if not already present. Default: BIGIP_HOSTNAME_policy-export_DATE.tgz -u --use_ucs_directory If enabled, the output file will be written to the UCS directory with a file extension of ".ucs". This allows access to the output file via the BIG-IP admin GUI under System >> Archives. Note: use of this option overrides the use of the -f/--output_dir option Default: Disabled -h --help Print this help menu EXAMPLES $script_name -d 2 -p /path/to/ -f output.tgz Use debug level 2, write output to the specified path and file $script_name --debug 2 --ucs --output_filename asm_policy_export.tgz.ucs Use debug level 2, write output archive to the UCS directory with the specified filename. -------------------------------------------------------------- }); }
Published Mar 12, 2015
Version 1.0hooleylist
Cirrostratus
Joined September 08, 2005
hooleylist
Cirrostratus
Joined September 08, 2005
- paul_dawson_258Nimbostratus
Does this work on 11?