F5 Automated Backups - The Right Way
Hi all, Often I've been scouring the devcentral fora and codeshares to find that one piece of handywork that will drastically simplify my automated backup needs on F5 devices. Based on the works of Jason Rahm in his post "Third Time's the Charm: BIG-IP Backups Simplified with iCall" on the 26th of June 2013, I went ahead and created my own iApp that pretty much provides the answers for all my backup-needs. Here's a feature list of this iApp: It allows you to choose between both UCS or SCF as backup-types. (whilst providing ample warnings about SCF not being a very good restore-option due to the incompleteness in some cases) It allows you to provide a passphrase for the UCS archives (the standard GUI also does this, so the iApp should too) It allows you to not include the private keys (same thing: standard GUI does it, so the iApp does it too) It allows you to set a Backup Schedule for every X minutes/hours/days/weeks/months or a custom selection of days in the week It allows you to set the exact time, minute of the hour, day of the week or day of the month when the backup should be performed (depending on the usefulness with regards to the schedule type) It allows you to transfer the backup files to external devices using 4 different protocols, next to providing local storage on the device itself SCP (username/private key without password) SFTP (username/private key without password) FTP (username/password) SMB (using smbclient, with username/password) Local Storage (/var/local/ucs or /var/local/scf) It stores all passwords and private keys in a secure fashion: encrypted by the master key of the unit (f5mku), rendering it safe to store the backups, including the credentials off-box It has a configurable automatic pruning function for the Local Storage option, so the disk doesn't fill up (i.e. keep last X backup files) It allows you to configure the filename using the date/time wildcards from the tcl [clock] command, as well as providing a variable to include the hostname It requires only the WebGUI to establish the configuration you desire It allows you to disable the processes for automated backup, without you having to remove the Application Service or losing any previously entered settings For the external shellscripts it automatically generates, the credentials are stored in encrypted form (using the master key) It allows you to no longer be required to make modifications on the linux command line to get your automated backups running after an RMA or restore operation It cleans up after itself, which means there are no extraneous shellscripts or status files lingering around after the scripts execute I wasn't able to upload the iApp template to this article, so I threw it on pastebin: http://pastebin.com/YbDj3eMN Enjoy! Thomas Schockaert
F5 Archiver Ansible Playbook
Problem this snippet solves: Centralized scheduled archiving (backups) on F5 BIG-IP devices are a pain however, in the new world of Infrastructure as Code (IaC) and Super-NetOps tools like Ansible can provide the answer. I have a playbook I have been working on to allow me to backup off box quickly, UCS files are saves to a folder names tmp under the local project folder, this can be changed by editing the following line in the f5Archiver.yml file: dest: "tmp/{{ inventory_hostname }}-{{ date['stdout'] }}.ucs" The playbook can be run from a laptop on demand or via some scheduler (like cron ) or as part of a CI/CD pipelines. How to use this snippet: F5 Archiver Ansible Playbook Gitlab: StrataLabs: AnsibleF5Archiver Overview This Ansible playbook takes a list of F5 devices from a hosts file located within the inventory directory, creates a UCS archive and copies locally into the 'tmp' direcotry. Requirements This Ansible playbook requires the following: * ansible >= 2.5 * python module f5-sdk * F5 BIG-IP running TMOS >= 12 Usage Run using the ansible-playbook command using the inventory -i option to use the invertory directory instead of the default inventory host file. NOTE: F5 username and password are not set in the playbook and so need to be passed into the playbook as extra variables using the --extra-vars option, the variables are f5User for the username and f5Pwd for the password. The below examples use the default admin:admin . To check the playbook before using run the following commands ansible-playbook -i inventory --extra-vars "f5User=admin f5Pwd=admin" f5Archiver.yml --syntax-check ansible-playbook -i inventory --extra-vars "f5User=admin f5Pwd=admin" f5Archiver.yml --check Once happy run the following to execute the playbook ansible-playbook -i inventory --extra-vars "f5User=admin f5Pwd=admin" f5Archiver.yml Tested this on version: 12.1Download a BIG-IP UCS archive with "curl".
Problem this snippet solves: Download a BIG-IP UCS archive using the program "curl" and verifies the output file's signature. Tested on 13.1.1. How to use this snippet: Edit the code to input the hostname of your F5 UI, admin credentials, source UCS file name (defaults to config.ucs), and the output file name. Code : #!/bin/bash # # Download a UCS archive (across a stable network) with curl. # #------------------------------------------------------------------------- F5_HOST='myhost.example.com' CREDENTIALS='admin:admin' FINAL_FILE='/tmp/config.ucs' ARCHIVE_NAME_ON_SERVER='config.ucs' DEBUG='' #------------------------------------------------------------------------- # # Get the md5 checksum for the archive. # #------------------------------------------------------------------------- ARCHIVE_CHECKSUM=$(curl -sku $CREDENTIALS -X POST -H "Content-type: application/json" \ -d "{\"command\":\"run\", \"utilCmdArgs\": \"-c '/usr/bin/md5sum /var/local/ucs/$ARCHIVE_NAME_ON_SERVER'\"}" \ https://$F5_HOST/mgmt/tm/util/bash | awk -F':' '{print $NF}' | awk -F'"' '{ print $2 }' | awk '{print $1}') [ -z "$ARCHIVE_CHECKSUM" ] && echo "Failed to get archive signature. Aborting." && exit 1 [ ! -z "$DEBUG" ] && echo "Archive checksum: $ARCHIVE_CHECKSUM" #------------------------------------------------------------------------- # # Find out the size of the archive and the size of the data packet. # #------------------------------------------------------------------------- Content_Range=$(curl -I -kv -u $CREDENTIALS -H 'Content-Type: application/json' -X GET "https://$F5_HOST/mgmt/shared/file-transfer/ucs-downloads/$ARCHIVE_NAME_ON_SERVER" 2>/dev/null | grep "Content-Range: " | cut -d ' ' -f 2) FIRST_CONTENT_RANGE=$(echo -n $Content_Range | cut -d '/' -f 1 | tr -d '\r') [ ! -z "$DEBUG" ] && echo -n "FIRST_CONTENT_RANGE: " [ ! -z "$DEBUG" ] && echo $FIRST_CONTENT_RANGE NUMBER_OF_LAST_BYTE=$(echo -n $FIRST_CONTENT_RANGE | cut -d '-' -f 2) [ ! -z "$DEBUG" ] && echo -n "NUMBER_OF_LAST_BYTE: " [ ! -z "$DEBUG" ] && echo $NUMBER_OF_LAST_BYTE INITIAL_CONTENT_LENGTH=$NUMBER_OF_LAST_BYTE CONTENT_LENGTH=$(($NUMBER_OF_LAST_BYTE+1)) [ ! -z "$DEBUG" ] && echo -n "CONTENT_LENGTH: " [ ! -z "$DEBUG" ] && echo $CONTENT_LENGTH DFILE_SIZE=$(echo -n $Content_Range | cut -d '/' -f 2 | tr -d '\r' ) [ ! -z "$DEBUG" ] && echo -n "DFILE_SIZE: " [ ! -z "$DEBUG" ] && echo $DFILE_SIZE LAST_END_BYTE=$((DFILE_SIZE-1)) CUMULATIVE_NO=0 [ ! -z "$DEBUG" ] && echo "CUMULATIVE_NO: $CUMULATIVE_NO" SEQ=0 LAST=0 #------------------------------------------------------------------------- # # Clean up: Remove the previous output file. # #------------------------------------------------------------------------- /bin/rm $FINAL_FILE 2>/dev/null #------------------------------------------------------------------------- # # Get the archive file. # #------------------------------------------------------------------------- while true do if [ $LAST -gt 0 ]; then [ ! -z "$DEBUG" ] && echo 'End of run reached.' break fi if [ $SEQ -eq 0 ]; then NEXT_RANGE=$FIRST_CONTENT_RANGE CUMULATIVE_NO=$NUMBER_OF_LAST_BYTE CONTENT_LENGTH=$INITIAL_CONTENT_LENGTH else START_BYTE=$(($CUMULATIVE_NO+1)) END_BYTE=$(($START_BYTE + $CONTENT_LENGTH)) if [ $END_BYTE -gt $LAST_END_BYTE ]; then [ ! -z "$DEBUG" ] && echo "END_BYTE greater than LAST_END_BYTE: $END_BYTE:$LAST_END_BYTE" LAST=1 let END_BYTE=$LAST_END_BYTE [ ! -z "$DEBUG" ] && echo "Getting the last data packet." fi NEXT_RANGE="${START_BYTE}-${END_BYTE}" CUMULATIVE_NO=$END_BYTE fi [ ! -z "$DEBUG" ] && echo "NEXT_RANGE: $NEXT_RANGE" let SEQ+=1 [ ! -z "$DEBUG" ] && echo "SEQ: $SEQ" OUTPUT_FILE_NAME="/tmp/$$_downloaded_ucs_archive_file_part_$SEQ"; curl -H "Content-Range: ${NEXT_RANGE}/${DFILE_SIZE}" -s -k -u $CREDENTIALS -H 'Content-Type: application/json' -X GET "https://$F5_HOST/mgmt/shared/file-transfer/ucs-downloads/$ARCHIVE_NAME_ON_SERVER" -o $OUTPUT_FILE_NAME cat $OUTPUT_FILE_NAME >> $FINAL_FILE /bin/rm $OUTPUT_FILE_NAME [ ! -z "$DEBUG" ] && echo "End of loop $SEQ" done #------------------------------------------------------------------------- # # Verify downloaded file. # #------------------------------------------------------------------------- FINAL_FILE_CHECKSUM=$(/usr/bin/md5sum $FINAL_FILE | awk '{print $1}') if [ "$FINAL_FILE_CHECKSUM" == "$ARCHIVE_CHECKSUM" ]; then echo "Download completed and verified." else echo "Downloaded file has incorrect checksum." exit 1 fi # END -------------------------------------------------------------------- Tested this on version: 13.0
UCS saving process failed. Can't fork
Odd error after I attempted when attempting to create a UCS file without the private keys and encrypting. [/S1-green-P:Active:In Sync] ~ tmsh save sys ucs /var/tmp/DJ/$HOSTNAME"."$(date +%Y%m%d).ucs no-private-key passphrase g00b3r Saving active configuration... Can't fork at /usr/local/bin/im line 305. /var/tmp/configsync.spec: Error creating package WARNING:There are error(s) during saving. Not everything was saved. Be very careful when using this saved file! Error creating package Error during config save. Unexpected Error: UCS saving process failed.
automating UCS archive creation, with date in the file name
In an F5 BIG-IP load balancer, is there a way to achieve the following objectives? automate / schedule the process of generating / backing up its UCS archives, and automatically include / append the date (of generating the UCS archive) into the file name of the UCS archive
BIP-IP c2400 VCMP Archive Bug
We are currently facing issue to run archive process on all VCMP on c2400 chassis. f5 identified this to be an bug and provided an hotfix but we are unable to apply the fix due to maintenance window. We are running behind on backup for months now. Is there any other manual process that we can run a archive process so we can restore the box in case of any crash scenario ?
Error while creating UCS backups: no such file or directory
Hi, I'm trying to create a UCS backup and I get an error (gui and cli): Saving active configuration... md5sum: /var/sdm/plugin_store/plugins/:Common:sna_plugin_67132_6/extensions/mssql/node_modules/mssql/node_modules/tedious/node_modules/babel-runtime/node_modules/core-js/library/modules/\$.to-iobject.js: No such file or directory Fatal: executing: md5sum /var/sdm/plugin_store/plugins/:Common:sna_plugin_67132_6/extensions/mssql/node_modules/mssql/node_modules/tedious/node_modules/babel-runtime/node_modules/core-js/library/modules/\$.to-iobject.js Operation aborted. /var/tmp/configsync.spec: Error creating package I read that the "\" character can be a problem, but when I use ls to list the files in that directory I can see the $.to-iobject.js file and it has no "\" before it. I'm pretty sure this file is important for the node-js pluging installed so I don't want to delete it, also there are lots of files that start with $. in this folder. What can i do? ThanksUCS saving process failed.
Hey, Whenever i try to save archive my config, i get the following results: Saving active configuration... /bin/sh: -c: line 0: unexpected EOF while looking for matching `'' /bin/sh: -c: line 1: syntax error: unexpected end of file Fatal: executing: md5sum /config/monitors/' Operation aborted. /tmp/configsync.spec: Error creating package WARNING:There are error(s) during saving. Not everything was saved. Be very careful when using this saved file! Error creating package Error during config save. BIGpipe parsing error: 01020001:3: The requested operation failed. I've verified so i have disk free so its not that... anyone come across this before?Backup iApp questions
I'm fairly new to Big IP. I'm trying to automate backups of the config. We are running 11.4.1. I found an iApp (https://devcentral.f5.com/wiki/iapp.configuration_backup_scheduler_v11_2.ashx) and it seems to be working well. My problem is that it pushes the tar file to a direcoryton the Big IP device. How can I automate getting the backup files off of the device? Doesn't look like WinSCP has anything for scheduling/automating file copies. I'm not real Linux savvy but I know enough to get but. Need some guidance here. Thank you, caseymnpcMigrating 10.2.4 ASM config from 3600 to 4200
plan on changing hardware from a 3600 to a 4200 and would like to reconfigure as little as possible. would loading an archive be an option of will the fail because of the platform difference? what would be an alternative? specially having to redo the ASM config feels a challenge.