F5 Automated Backups - The Right Way

We are currently running 13.1.0.8 on our devices and this works great for us! It was very easy to deploy and is pretty straight forward to use. The only issue I see so far is that we would like to increase the pruning-handler interval. Our logs are getting pretty full with the pruning informational log entries. Can you please let me know how I can increase this value to at least every 10 minutes. Thanks!

Hi, Tabernarious. Yes, vCMP hosts change almost never, so, as you say and we always do, we take an UCS backup before and after some upgrade or change of the host, and for us it is more tan enough. The goal with the question in our post was to try to have only one way of taking automated backups in our installation, given that the iApp works really fine,... but, unfortunately, it seems it won't be possible. Thanks a lot for your answer and time. Your comments and improvements are very helpful for us all. Best regards.

@comunicaciones.datos, Unfortunately vCMP hosts cannot run iApps at all (as far as I know), so this cannot be adapted. If I have time I'll experiment to see if a vCMP host can run iCall scripts. If so, I might be able to devise a reasonable way to export a script to be imported to the vCMP host. Even if it is possible it would be a manual process to implement...which skirts around the entire premise. What I will say is that from my experience, things don't generally change much on vCMP hosts (not like they do on guests) so saving/storing a vCMP host's UCS Archive before and after each scheduled change is usually sufficient. Also, rebuilding a vCMP host from scratch is not nearly as onerous as rebuilding all the guests from scratch :)

Hi all:

This iApp Works really fine and we have adopted it for all our F5, with the exception of a pair of them dedicated entirely to vCMP.

When we load the template in these F5, a warning triggers saying that "ltm is not provisioned/enabled". Given that the device is dedicated to vCMP, it is true that LTM is not provisioned, and the outcome is that we can't run this wonderful iApp in this pair of vCMP. Over these vCMP we run some guests, and in them we have loaded this iApp and we do the backups without any problem.

So, the obvious question is, is there any way of adapting this iApp to backup dedicated vCMP devices? It would be very useful for us.

Thanks a lot for your help.

This is now available on GitHub! Please look here for the latest version, and submit any bugs or questions as an "Issue" on GitHub:

 

https://github.com/tabernarious/f5-automated-backup-iapp

 

As a BACKUP, look in the DevCentral CodeShare. This MAY NOT HAVE THE LATEST VERSION:

 

https://devcentral.f5.com/s/articles/f5-iapp-automated-backup-1114

 

I needed to troubleshoot an issue and ended up rolling in many of the proposed and posted fixes into a new iApp.

 

 iApp VERSIONS (From what I gathered perusing DevCentral)
 ~v2.0  - 20140312 - Initially posted releases (v11.4.0-11.6.x? compatibility). (Developed/posted by Thomas Schockaert)
 v2.1.1 - 20160916 - Retooled SMB upload from smbclient to "mount -t cifs" (v12.1+ compatibility). (Developed/posted by MAG)
 v2.2.1 - 20171214 - Allowed multiple instances of iApp by leveraging $tmsh::app_name to create unique object names. (Developed by Daniel Tavernier/tabernarious)
 v2.2.2 - 20171214 - Added "/" to "mount -t cifs" command and clarified/expanded help for SMB (CIFS) Destination Parameters. (Developed by Daniel Tavernier/tabernarious)
 v2.2.3 - 20171214 - Set many fields to "required" and set reasonable default values to prevent loading/configuration errors. Expanded help regarding private keys. (Developed by Daniel Tavernier/tabernarious)
 v2.2.4 - 20171214 - Added fix to force FTP to use binary upload. (Copied code posted by Roy van Dongen, posted by Daniel Tavernier/tabernarious)
 v2.2.4a - 20171215 - Added items to FUTURE list.
 v2.2.5 - 20171228 - Added notes about special characters in passwords. Added Deployment Information and ConfigSync sections. (Developed by Daniel Tavernier/tabernarious)
 v2.2.5a - 20180117 - Added items to FUTURE list.
 v2.2.5b4 - 20180118 - Moved encrypted values for SMB/CIFS to shell script which eliminates ConfigSync issues. Fixed long-password issue by using "-A" with openssl so that base64 encoded strings are written and read as a single line. (Developed by Daniel Tavernier/tabernarious)
 v2.2.5b4+ - 20180118 - Refining changes to SMB/CIFS and replicating to other remote copy types. (Developed by Daniel Tavernier/tabernarious)
 v3.0.0 - 20180124 - (Developed by Daniel Tavernier/tabernarious)
                   - Eliminated ConfigSync issues and removed ConfigSync notes section. (Encrypted values now in $script instead of local file.)
                   - Passwords now have no length limits. (Using "-A" with openssl which reads/writes base64 encoded strings as a single line.)
                   - Added $script error checking for all remote backup types. (Using 'catch' to prevent tcl errors when $script aborts.)
                   - Backup files are cleaned up after $script error due to new error checking.
                   - Added logging. (Run logs sent to '/var/log/ltm' via logger command which is compatible with BIG-IP Remote Logging configuration (syslog). Run logs AND errors sent to '/var/tmp/scriptd.out'. Errors may include plain-text passwords which should not be in /var/log/ltm or syslog.)
                   - Added custom cipher option for SCP.
                   - Added StrictHostKeyChecking=no option.
                   - Combined SCP and SFTP because they are both using SCP to perform the remote copy.
 v3.1.0 - 20180201 - (Developed by Daniel Tavernier/tabernarious)
                   - Removed "app-service none" from iCall objects. The iCall objects are now created as part of the Application Service (iApp) and are properly cleaned up if the iApp is redeployed or deleted.
                   - Reasonably tested on 11.5.4 HF2 (SMB worked fine using "mount -t cifs") and altered requires-bigip-version-min to match.
                   - Fixing error regarding "script did not successfully complete: (can't read "::destination_parameters__protocol_enable": no such variable" by encompassing most of the "implementation" in a block that first checks $::backup_schedule__frequency_select for "Disable".
                   - Added default value to "filename format".
                   - Changed UCS default value for $backup_file_name_extension to ".ucs" and added $fname_noext.
                   - Removed old SFTP sections and references (now handled through SCP/SFTP).
                   - Adjusted logging: added "sleep 1" to ensure proper logging; added $backup_directory to log message.
                   - Adjusted some help messages.

We would like to avoid using IAPP. Is there anyway to accomplish this using F5's Python SDK?

Hi, I have the same problem than Sylvain Q. Does someone else have the same problem? To workaround this issue I put the following lines to the crontab:

0 * * * * find /var/local/ucs/*.ucs -type f -cmin +120 -delete >/dev/null 2>&1
0 * * * * find /var/local/scf/*.scf -type f -cmin +120 -delete >/dev/null 2>&1

In my case, I perform a backup each hour to an external FTP server. Then, I remove the temporary files older than 2 hours.

Do you have a better idea/solution ?

Hi, 1. I have successfully setup this iapp - so thanks a lot. 2. What should i do to initiate a backup for test without waiting for schedule i setup ?

Thanks a lot.

Hello,

is it possible to inhibit hostname synchronization in HA pair?

Thanks all

I'm running into the same problem, syncing between two HA members renders this iApp inoperational on the standby node. Even when using the same key for both machines, I always end up with only one ucs on the remote (SCP) server. reconfigure and re-apply the iApp/Application Service on the second node after configuring it on the first node as suggested above also won't help. Has anybody solved that issue? I'm running 11.6.0 HF6.

EDIT: After doing some analysis it appears that the standby machine (.210) fails to establish the SSH connection, whereas the active machine (.110) successfully can set it up (and transfer files over it):

Sep 23 12:58:05 my_upload_target sshd[58009]: Accepted publickey for lbupload from 10.x.x.110 port 40067 ssh2: RSA xxxxx [MD5]
Sep 23 12:58:05 my_upload_target sshd[58040]: Connection closed by 10.x.x.210 [preauth]

Obviously, the key synchronisation between both machines seems not to work...

EDIT2: In order to affirm that assumption, I replaced the private key on the active machine with rubbish to see if that provokes the same logging entry. The device-group is configured with config auto-sync, so this change will also immediately get synced to the standby device. Now, when the iCall script is running, these messages are logged by the upload server:

Sep 23 13:30:08 my_upload_target sshd[54899]: Failed password for my_upload_user from 10.x.x.110 port 42862 ssh2
Sep 23 13:30:08 my_upload_target sshd[54899]: Connection closed by 10.x.x.110 [preauth]
Sep 23 13:30:11 my_upload_target sshd[54904]: Connection closed by 10.x.x.210 [preauth]

As expected, the active machine is now rejected due to bad authentication (failed password), but the other one is not - it's just closing the connection during preauth and it seems that it does not even start any key exchange...

EDIT3: OK, it's actually that easy... The entry in /root/.ssh/known_hosts was missing on the failover unit, but was present in the active one. After manually establishing an ssh connection to the destination host, thus adding its public key to known_hosts, the automated backup now works for both.

EDIT4 (and that will probably be the last one - I hope this help anybody experiencing the same problems...): I added

-o StrictHostKeyChecking=no