F5 Automated Backups - The Right Way
Hi all,
Often I've been scouring the devcentral fora and codeshares to find that one piece of handywork that will drastically simplify my automated backup needs on F5 devices. Based on the works of Jason Rahm in his post "Third Time's the Charm: BIG-IP Backups Simplified with iCall" on the 26th of June 2013, I went ahead and created my own iApp that pretty much provides the answers for all my backup-needs.
Here's a feature list of this iApp:
- It allows you to choose between both UCS or SCF as backup-types. (whilst providing ample warnings about SCF not being a very good restore-option due to the incompleteness in some cases)
- It allows you to provide a passphrase for the UCS archives (the standard GUI also does this, so the iApp should too)
- It allows you to not include the private keys (same thing: standard GUI does it, so the iApp does it too)
- It allows you to set a Backup Schedule for every X minutes/hours/days/weeks/months or a custom selection of days in the week
- It allows you to set the exact time, minute of the hour, day of the week or day of the month when the backup should be performed (depending on the usefulness with regards to the schedule type)
- It allows you to transfer the backup files to external devices using 4 different protocols, next to providing local storage on the device itself
- SCP (username/private key without password)
- SFTP (username/private key without password)
- FTP (username/password)
- SMB (using smbclient, with username/password)
- Local Storage (/var/local/ucs or /var/local/scf)
- It stores all passwords and private keys in a secure fashion: encrypted by the master key of the unit (f5mku), rendering it safe to store the backups, including the credentials off-box
- It has a configurable automatic pruning function for the Local Storage option, so the disk doesn't fill up (i.e. keep last X backup files)
- It allows you to configure the filename using the date/time wildcards from the tcl [clock] command, as well as providing a variable to include the hostname
- It requires only the WebGUI to establish the configuration you desire
- It allows you to disable the processes for automated backup, without you having to remove the Application Service or losing any previously entered settings
- For the external shellscripts it automatically generates, the credentials are stored in encrypted form (using the master key)
- It allows you to no longer be required to make modifications on the linux command line to get your automated backups running after an RMA or restore operation
- It cleans up after itself, which means there are no extraneous shellscripts or status files lingering around after the scripts execute
I wasn't able to upload the iApp template to this article, so I threw it on pastebin: http://pastebin.com/YbDj3eMN
Enjoy!
Thomas Schockaert
- JoadNimbostratus
Hello,
is it possible to inhibit hostname synchronization in HA pair?
Thanks all
- Shiran_Cohen_24Nimbostratus
Hi, 1. I have successfully setup this iapp - so thanks a lot. 2. What should i do to initiate a backup for test without waiting for schedule i setup ?
Thanks a lot.
- DTC_Manager_325Nimbostratus
Hi, I have the same problem than Sylvain Q. Does someone else have the same problem? To workaround this issue I put the following lines to the crontab:
0 * * * * find /var/local/ucs/*.ucs -type f -cmin +120 -delete >/dev/null 2>&1 0 * * * * find /var/local/scf/*.scf -type f -cmin +120 -delete >/dev/null 2>&1
In my case, I perform a backup each hour to an external FTP server. Then, I remove the temporary files older than 2 hours.
Do you have a better idea/solution ?
- Tariq_Iqbal_924Nimbostratus
We would like to avoid using IAPP. Is there anyway to accomplish this using F5's Python SDK?
- tabernarious_11Nimbostratus
This is now available on GitHub! Please look here for the latest version, and submit any bugs or questions as an "Issue" on GitHub:
 
https://github.com/tabernarious/f5-automated-backup-iapp
 
As a BACKUP, look in the DevCentral CodeShare. This MAY NOT HAVE THE LATEST VERSION:
 
https://devcentral.f5.com/s/articles/f5-iapp-automated-backup-1114
 
I needed to troubleshoot an issue and ended up rolling in many of the proposed and posted fixes into a new iApp.
 
iApp VERSIONS (From what I gathered perusing DevCentral) ~v2.0 - 20140312 - Initially posted releases (v11.4.0-11.6.x? compatibility). (Developed/posted by Thomas Schockaert) v2.1.1 - 20160916 - Retooled SMB upload from smbclient to "mount -t cifs" (v12.1+ compatibility). (Developed/posted by MAG) v2.2.1 - 20171214 - Allowed multiple instances of iApp by leveraging $tmsh::app_name to create unique object names. (Developed by Daniel Tavernier/tabernarious) v2.2.2 - 20171214 - Added "/" to "mount -t cifs" command and clarified/expanded help for SMB (CIFS) Destination Parameters. (Developed by Daniel Tavernier/tabernarious) v2.2.3 - 20171214 - Set many fields to "required" and set reasonable default values to prevent loading/configuration errors. Expanded help regarding private keys. (Developed by Daniel Tavernier/tabernarious) v2.2.4 - 20171214 - Added fix to force FTP to use binary upload. (Copied code posted by Roy van Dongen, posted by Daniel Tavernier/tabernarious) v2.2.4a - 20171215 - Added items to FUTURE list. v2.2.5 - 20171228 - Added notes about special characters in passwords. Added Deployment Information and ConfigSync sections. (Developed by Daniel Tavernier/tabernarious) v2.2.5a - 20180117 - Added items to FUTURE list. v2.2.5b4 - 20180118 - Moved encrypted values for SMB/CIFS to shell script which eliminates ConfigSync issues. Fixed long-password issue by using "-A" with openssl so that base64 encoded strings are written and read as a single line. (Developed by Daniel Tavernier/tabernarious) v2.2.5b4+ - 20180118 - Refining changes to SMB/CIFS and replicating to other remote copy types. (Developed by Daniel Tavernier/tabernarious) v3.0.0 - 20180124 - (Developed by Daniel Tavernier/tabernarious) - Eliminated ConfigSync issues and removed ConfigSync notes section. (Encrypted values now in $script instead of local file.) - Passwords now have no length limits. (Using "-A" with openssl which reads/writes base64 encoded strings as a single line.) - Added $script error checking for all remote backup types. (Using 'catch' to prevent tcl errors when $script aborts.) - Backup files are cleaned up after $script error due to new error checking. - Added logging. (Run logs sent to '/var/log/ltm' via logger command which is compatible with BIG-IP Remote Logging configuration (syslog). Run logs AND errors sent to '/var/tmp/scriptd.out'. Errors may include plain-text passwords which should not be in /var/log/ltm or syslog.) - Added custom cipher option for SCP. - Added StrictHostKeyChecking=no option. - Combined SCP and SFTP because they are both using SCP to perform the remote copy. v3.1.0 - 20180201 - (Developed by Daniel Tavernier/tabernarious) - Removed "app-service none" from iCall objects. The iCall objects are now created as part of the Application Service (iApp) and are properly cleaned up if the iApp is redeployed or deleted. - Reasonably tested on 11.5.4 HF2 (SMB worked fine using "mount -t cifs") and altered requires-bigip-version-min to match. - Fixing error regarding "script did not successfully complete: (can't read "::destination_parameters__protocol_enable": no such variable" by encompassing most of the "implementation" in a block that first checks $::backup_schedule__frequency_select for "Disable". - Added default value to "filename format". - Changed UCS default value for $backup_file_name_extension to ".ucs" and added $fname_noext. - Removed old SFTP sections and references (now handled through SCP/SFTP). - Adjusted logging: added "sleep 1" to ensure proper logging; added $backup_directory to log message. - Adjusted some help messages.
- comunicaciones_Nimbostratus
Hi all:
This iApp Works really fine and we have adopted it for all our F5, with the exception of a pair of them dedicated entirely to vCMP.
When we load the template in these F5, a warning triggers saying that "ltm is not provisioned/enabled". Given that the device is dedicated to vCMP, it is true that LTM is not provisioned, and the outcome is that we can't run this wonderful iApp in this pair of vCMP. Over these vCMP we run some guests, and in them we have loaded this iApp and we do the backups without any problem.
So, the obvious question is, is there any way of adapting this iApp to backup dedicated vCMP devices? It would be very useful for us.
Thanks a lot for your help.
- tabernarious_11Nimbostratus
@comunicaciones.datos, Unfortunately vCMP hosts cannot run iApps at all (as far as I know), so this cannot be adapted. If I have time I'll experiment to see if a vCMP host can run iCall scripts. If so, I might be able to devise a reasonable way to export a script to be imported to the vCMP host. Even if it is possible it would be a manual process to implement...which skirts around the entire premise. What I will say is that from my experience, things don't generally change much on vCMP hosts (not like they do on guests) so saving/storing a vCMP host's UCS Archive before and after each scheduled change is usually sufficient. Also, rebuilding a vCMP host from scratch is not nearly as onerous as rebuilding all the guests from scratch :)
- comunicaciones_Nimbostratus
Hi, Tabernarious. Yes, vCMP hosts change almost never, so, as you say and we always do, we take an UCS backup before and after some upgrade or change of the host, and for us it is more tan enough. The goal with the question in our post was to try to have only one way of taking automated backups in our installation, given that the iApp works really fine,... but, unfortunately, it seems it won't be possible. Thanks a lot for your answer and time. Your comments and improvements are very helpful for us all. Best regards.
- remosser_365878Nimbostratus
We are currently running 13.1.0.8 on our devices and this works great for us! It was very easy to deploy and is pretty straight forward to use. The only issue I see so far is that we would like to increase the pruning-handler interval. Our logs are getting pretty full with the pruning informational log entries. Can you please let me know how I can increase this value to at least every 10 minutes. Thanks!
- tabernarious_11Nimbostratus
@remosser, You will have to edit the template and reimport. The following is for v3.1.4:
- Open the template in a text editor (not in the F5 GUI).
- Rename the template (not the text file) by finding line 68 and changing
tof5.automated_backup.v3.1.4
or something.f5.automated_backup.v3.1.4_pruning_600
- Then find line 483 (or near) which should be
and changeinterval 60 \
to however many seconds you want.60
- Import the template to the F5 which should show up as a new template (with the new name you set).
- Open your iApp configuration, change the template, and redeploy. You may need to set the Frequency to
, finish, then go back and set the Frequency back to your desired values. I can't remember what exactly needs to be done to cause the iCall scripts to be regenerated.Disabled