on
24-Jan-2018
15:35
- edited on
15-Feb-2022
13:09
by
LiefZimmerman
Problem this snippet solves:
Please look on GitHub for the latest version, and submit any bugs or questions as an "Issue" on GitHub:
(Note: DevCentral admin update - Daniel's project appears abandoned so it's been forked and updated to the link below. @damnski on github added some SFTP code that has been merged in as well.)
https://github.com/f5devcentral/f5-automated-backup-iapp
Building on the significant work of Thomas Schockaert (and several other DevCentralites) I enhanced many aspects I needed for my own purposes, updated many things I noticed requested on the forums, and added additional documentation and clarification. As you may see in several of my comments on the original posts, I iterated through several 2.2.x versions and am now releasing v3.0.0. Below is the breakdown!
Also, I have done quite a bit of testing (mostly on v13.1.0.1 lately) and I doubt I've caught everything, especially with all of the changes. Please post any questions or issues in the comments.
Cheers!
Daniel Tavernier (tabernarious)
16.0
I've tried this with SCP and SMB and it is working great! I am wondering if it could also cleanup the files in the mount destination: Would I just write that bash logic somewhere in the iAPP?
I noticed a
Clean up local files
section and wondered if it's as easy as adding bash there that would rm items older than X days.
Something like:
find /var/tmp/cifs/$mountp -mtime +30 -exec rm {} \
@Darren Walker, I have added your request to the Issues list on the GitHub repository:
https://github.com/tabernarious/f5-automated-backup-iapp/issues
Feel free to leave comments or ideas there! There are some nuances/checks that would have to be worked out. If done improperly this could destroy other critical files on the remote backup volume.
Thanks for creating this! You have done a really excellent job on a very valuable tool.
It is embarrassing that an enterprise appliance does not include any methods for automated remote backups.
Tristan
@snl I have relaxed the destination IP restriction in iApp v3.1.2 (available in the Codeshare above and on GitHub)! You can now use destination FQDN as long as you have DNS configured and functional.
@Darren Walker, I have implemented SMB auto-pruning in iApp v3.1.3 (available in the Codeshare above and on GitHub)! Finally 🙂 Cheers! -Daniel
Hi,
I tried to use this iApp to backup ucs to remote CIFS server.
I encountered an issue with this iApp because the AD team deny browsing intermediate folders.
if I mount with commands :
mount -t cifs //${server}/${msshare}${mssubdir} ${mountp} -o user=${username}%${password},domain=${msdomain}
ls ${mountp}
I can read files in the mount point, but if I mount with commands (like in your script) :
mount -t cifs //${server}/${msshare} ${mountp} -o user=${username}%${password},domain=${msdomain}
ls ${mountp}${mssubdir}
I have following error:
ls: cannot access /var/tmp/cifs/MY/PATH/: Permission denied
after changing the line 296 to :
exec echo -e "\\!/bin/sh\nf5masterkey=\$(f5mku -K)\nusername=\$(echo \"ENCRYPTEDUSERNAME\" | openssl aes-256-ecb -salt -a -A -d -k \${f5masterkey})\npassword=\$(echo \"ENCRYPTEDPASSWORD\" | openssl aes-256-ecb -salt -a -A -d -k \${f5masterkey})\nmsdomain=\$(echo \"ENCRYPTEDMSDOMAIN\" | openssl aes-256-ecb -salt -a -A -d -k \${f5masterkey})\nserver=\$(echo \"ENCRYPTEDSERVER\" | openssl aes-256-ecb -salt -a -A -d -k \${f5masterkey})\nmsshare=\$(echo \"ENCRYPTEDMSSHARE\" | openssl aes-256-ecb -salt -a -A -d -k \${f5masterkey})\nmssubdir=\$(echo \"ENCRYPTEDMSSUBDIR\" | openssl aes-256-ecb -salt -a -A -d -k \${f5masterkey})\nmountp=\$(echo \"ENCRYPTEDMOUNTP\" | openssl aes-256-ecb -salt -a -A -d -k \${f5masterkey})\ncd /var/local/ucs\nif \[ \! -d \${mountp} \]\nthen\n\tmkdir -p \${mountp}\n\tif \[ \$? -ne 0 \]\n\tthen\n\t\trm -f ${fname_noext}BACKUPFILENAMEEXTENSION_WITHDOT\n\t\texit 1\n\tfi\nfi\nmount -t cifs //\${server}/\${msshare}\${mssubdir} \${mountp} -o user=\${username}%\${password},domain=\${msdomain} 2>> /var/tmp/scriptd.out\nif \[ \$? -ne 0 \]\n\tthen\n\trm -f ${fname_noext}BACKUPFILENAMEEXTENSION_WITHDOT\n\texit 1\nfi\nfONSMB=\$(ls -t \${mountp}/\*.ucs 2>/dev/null| head -n 1 2>/dev/null)\nif \[ \"X\"\${fONSMB} \!= \"X\" \]\n\tthen\n\tsum1=\$(md5sum ${fname_noext}BACKUPFILENAMEEXTENSION_WITHDOT | awk '{print \$1}')\n\tsum2=\$(md5sum \${fONSMB} | awk \'{print \$1}\')\n\tif \[ \${sum1} == \${sum2} \]\n\tthen\n\t\techo \"ERROR: File ${fname_noext}BACKUPFILENAMEEXTENSION_WITHDOT already exists in //\${server}/\${msshare}/\${mssubdir}\" >> /var/tmp/scriptd.out\n\t\tumount \${mountp}\n\t\trm -f ${fname_noext}BACKUPFILENAMEEXTENSION_WITHDOT\n\t\texit 1\n\tfi\nfi\ncp ${fname_noext}BACKUPFILENAMEEXTENSION_WITHDOT \${mountp}\nrm -f ${fname_noext}BACKUPFILENAMEEXTENSION_WITHDOT\n\nif \[ \"PRUNINGMODE\" \!= \"Disabled\" \]; then\n\n\tfiles_tokeep=\$(ls -t \${mountp}\${mssubdir}/*PRUNINGSUFFIX.BACKUPFILENAMEEXTENSION_NODOT 2>/dev/null | head -n CONSERVE\)\n\tfor current_archive_file in `ls \${mountp}\${mssubdir}/*PRUNINGSUFFIX.BACKUPFILENAMEEXTENSION_NODOT 2>/dev/null` ; do\n\t\tcurrent_archive_file_basename=`basename \$current_archive_file`\n\t\tcheck_file=\$(echo \$files_tokeep | grep -w \$current_archive_file_basename)\n\t\tif \[ \"\$check_file\" == \"\" \] ; then\n\t\t\trm -f \$current_archive_file\n\t\tfi\n\tdone\n\tif \[ \"BACKUPFILENAMEEXTENSION_NODOT\" == \"scf\" \] ; then\n\t\ttar_files_tokeep=\$(ls -t \${mountp}\${mssubdir}/*PRUNINGSUFFIX.BACKUPFILENAMEEXTENSION_NODOT.tar 2>/dev/null | head -n CONSERVE\)\n\t\tfor current_archive_tar_file in `ls \${mountp}\${mssubdir}/*PRUNINGSUFFIX.BACKUPFILENAMEEXTENSION_NODOT.tar 2>/dev/null` ; do\n\t\t\tcurrent_archive_tar_file_basename=`basename \$current_archive_tar_file`\n\t\t\tcheck_file=\$(echo \$tar_files_tokeep | grep -w \$current_archive_tar_file_basename)\n\t\t\tif \[ \"\$check_file\" == \"\" \] ; then\n\t\t\t\trm -f \$current_archive_tar_file\n\t\t\tfi\n\t\tdone\n\tfi\nfi\n\numount \${mountp}\n\nexit 0\n\n" > $scriptfile
it works!
I didn't worked with Pruning mode so I didn't try to correct the script in pruning section.
@Stanislas Piron, Awesome! I have added this as an issue on GitHub:
https://github.com/tabernarious/f5-automated-backup-iapp/issues/9
I'll see if I can work this out sooner than later while it's fresh on my mind.
Given that clustered BIG-IP appliances synchronize iApp templates and applications created from iApps, has anyone come across a method with this iApp to backup each appliance in the cluster independently?
@PhillyPDXmike, As long as you use the ${host} variable in the file name iApp setting (this is included in the default file name setting) it will generate archives with unique names per device (regardless of clustering). Are there other settings you are hoping to set independently?
@tabernarious, Thanks for the quick response!! I am using version 3.1.2 of the iApp and can confirm that the backup worked on both the active and standby units after I copied the public/private key from the active to the standby, edited the hostname in the public key (/root/.ssh/id_rsa.pub) on the standby, then added that public key to my sftp server's authorized_keys file.
My initial question was prompted from a cosmetic situation where the sample output for the "Select the filename format > Destination Parameters" setting carried over from the active appliance (dev02a) to the standby appliance (dev02b).
Your question: "Are there other settings you are hoping to set independently?" Since the UCS backup is specific to each appliance, it'd be nice to somehow have device independence within the iApp which is probably more of an F5 iApp software architecture thing than anything to do with this specific iApp. What are your thoughts?
I’m glad it’s working, though I don’t think you should have had to mess with the keys (copying between boxes)—I’ll test that too. I see what you mean about the cosmetic file name example issue. I’ll have to think about if that can be fixed.
As for your question about the overall architecture, I definitely agree that there are situations when it would be nice to keep apps independent, but I think the vast majority of cases support the existing keep-everything-in-sync model.
@aj1, This should definitely work on 13.1.x. Most of my recent testing has been on 13.1.0.x.
Hi,
I'm using v3.1.3, and getting this message:
script did not successfully complete: (can't read "::destination_parameters__pruning_mode": no such variable
while executing
"if { $::destination_parameters__pruning_mode eq "Only Prune iApp-Generated Archives" } {
set pruning_suffix $::destination_par..."
invoked from within
"if { $freq != "Disable" } {
Ensure a default $filename_format is set
if { $::destination_parameters__filenam..." line:44)
I Fixed it by changing line 54 from this:
if { $::destination_parameters__pruning_mode eq "Only Prune iApp-Generated Archives" } {
To this:
if { [info exists ::destination_parameters__pruning_mode] && ($::destination_parameters__pruning_mode eq "Only Prune iApp-Generated Archives") } {
Basically validating the variable exists as a condition to the 'if' statement.
I'm having some issues with authentication: In the logs, it shows the warning banner on the host I'm trying to send the backup to, but then shows this:
Permission denied, please try again.
Permission denied, please try again.
Received disconnect from xx.xx.xx.xx: 2: Too many authentication failures for backuphost
lost connection
f5.automated_backup iApp automated_backups_iapp:loadbalancer.com_20180601_085600.ucs REMOTE COPY (SCP) FAILED (check for errors above)
f5.automated_backup iApp automated_backups_iapp: FINISHED
We tried to enter the username/password, and I also copied the private key from the backuphost but still the same. Also, I checked /var/local/ucs for the local copy it would (should?) be copying over but there is only a "config.ucs" file (~70M) and the file's last dated a few weeks ago when we first installed it, I believe. It's been running every few minutes but failing and I thought it was just the copy part that fails. Any guidance would help. Thanks.
In my testing, using the backup files via SCP/SFTP option doesn't save a copy locally.
For the "Copy/Paste the SSH private key to be used for passwordless authentication:" option, I copied/pasted the entire RSA private key including -----BEGIN RSA PRIVATE KEY-----, -----END RSA PRIVATE KEY-----, and everything in between. Also, I found these links F5 K13454 and tecmint article very helpful in getting the ssh passwordless authentication working properly, although I didn't have to change any settings on my linux server.
@aj1, Regarding your search for the local copies of the UCS Archives, the iApp automatically cleans up the local copy whether the remote copy succeeds or fails. If you want local copies I would recommend deploying a second copy of the iApp that just backs up locally and auto-prunes to prevent filling up the /var partition.
Hello, I a using v3.1.3. When I attempt to implement the script using FTP for remote copy, I receive the following error.
script did not successfully complete: (can't read "::destination_parameters__pruning_mode": no such variable
while executing
"if { $::destination_parameters__pruning_mode eq "Only Prune iApp-Generated Archives" } {
set pruning_suffix $::destination_par..."
invoked from within
"if { $freq != "Disable" } {
Ensure a default $filename_format is set
if { $::destination_parameters__filenam..." line:44)
I am trying to understand the reason for this error. When I choose the "Select the Filename Format" drop-down, I am selecting an option that does include ${host}. Am using code 11.6.1 HF2 on my box. I saw @keithhubb post and resolution but would like to understand the reason for the
text
error before I implement on prod boxes. Thank you.
@cbrandys, see keithhubb’s earlier comment for a quick fix (you have to edit the template). I am planning to fix this officially with a release this week (possibly today). Stay tuned.
Released v3.1.4 to GitHub and updated CodeShare above. Release Notes:
@Silvain Q, If you're referring to the SCP/SFTP option, this could be manually configured relatively easily (it will take me more work to make this customizable through the iApp configuration). In a text editor open the iApp template (
f5.automated_backup.v3.1.4.tmpl.tcl
), find the line that starts with:
exec echo -e "scp_function()
...then find this part of the line:
\n\tscp -i /var/tmp/TMSHAPPNAME_scp.key
...and replace it with:
\n\tscp -P 224 -i /var/tmp/TMSHAPPNAME_scp.key
(NOTE:
-P 224
is the scp
option to modify the destination port.)
I would then recommend changing this line:
sys application template /Common/f5.automated_backup.v3.1.4 {
to
sys application template /Common/f5.automated_backup.v3.1.4_scp224 {
(NOTE: Changing the template name will allow you to have both the original
v3.1.4
and the new v3.1.4_scp224
templates on the BIG-IP at the same time.)
Finally, save the file you've been editing (I recommend selecting a new name to avoid confusion with the original) and upload it to the BIG-IP (you will have to select "overwrite" unless you edited the
sys application template
name above).
Let me know if you're looking for something else.
All,
I tried this but I got below error in /var/log/script.log
Saving active configuration... /var/local/ucs/abcd.ucs is saved. abcd.ucs GENERATING iApp F5autobackp: abcd.ucs SAVED LOCALLY (/var/local/ucs) iApp F5autobackp: abcd.ucs REMOTE COPY (SCP) STARTING
xxxxxxxxxx
Permission denied, please try again. Permission denied, please try again. Permission denied (publickey,gssapi-with-mic,password). lost connection iApp F5autobackp: abcd.ucs REMOTE COPY (SCP) FAILED (check for errors above) iApp F5autobackp: FINISHED
Any suggestion?
@Suresh Jo, Do you have the SCP/SFTP server set up for key-based authentication? And do you have the non-encrypted private key pasted into the iApp?
Hi,
Could you add a feature to specify the number of occurence you would like with remote backup ? I mean like a logrotate. It's just to avoid a space disk full.
As a workaround, I created a cron on the remote machine to delete file more than XX days.
Thanks,
@Support, This feature is currently available for "On this F5" and "Remotely via SMB/CIFS". I have not yet taken the time to devise a similar method for FTP or SFTP/SCP. If you have ideas feel free to share them here or on the GitHub repository: https://github.com/tabernarious/f5-automated-backup-iapp
Cheers!
Thanks a lot for that iApp, i realy appreciate your effort , i have deployed the iApp and worked from the first time but the second time i got the following error :
script did not successfully complete: (invalid command name "exec" while executing "exec f5mku -K" invoked from within "if { $freq != "Disable" } {
Ensure a default $filename_format is set
if { $::destination_parameters__filenam..." line:44)
Hi Suresh,
The issue is that the file you want to transfer doesn't exists, had the same issue.
If you debug, you will see that it tries to transfers a file xxx.ucs.ucs.
Replace {fname}BACKUPFILESCRIPTEXTENSION with {fname}
This will fix the issue.
Kind regards,
Anthony
@Suresh and @Anthony, This filename extension issue for FTP should have been fixed in v3.1.1 (see changelog) which is carried forward to the latest version (currently v3.1.4). What version(s) of the template are you running?
@efouli, I don't believe I've seen that "exec" error before. What version of this iApp are you running, and what version of TMOS are you running? And what method are you using for saving backups (local, SFTP, SMB, or FTP)?
Hi all,
Quick question on the iApp and the passwordless scp copy to remote storage.
I have the solution working fine on the first pair of BIGIPs where the keys are already stored on the device, bit doing a fresh pair now that do not have the private key in the default file but pasted into the iApp the device fails to upload to the scp destination, so it looks like I will have to copy the private key to each device to get this working.. Is this how it's meant to work, as I thought pasting it into the iApp config would do away with this part?
Currently using latest version of the template off GitHub.
Thanks..
@Markie Parkie, You should be able to just have the SSH key in the iApp, BUT by default this will fail unless the destination server's SSH fingerprint has been added to '/root/.ssh/known_hosts' (or you set "StrictHostKeyChecking" to "No" in the iApp (not recommended). Have you looked at the iApp logs (/var/tmp/scriptd.out)? If you log into the BIG-IP as root and SSH to the destination server are you prompted about 'Host verification' or similar?
Hi Tabernarious,
I have ssh to the destination server to add the fingerprint but still have the same issue. The log just says about too many Auth attempts and falls pass to password Auth.
Thanks..
iApp DEVICE_BACKUP: xxxxxxx.xxxxxxx.xxxxxxx_20181112_020000.ucs REMOTE COPY (SCP) STARTING
This system is for the use of authorized users only.
Permission denied, please try again.
Received disconnect from xx.xx.xx.xx: 2: Too many authentication failures
lost connection
iApp DEVICE_BACKUP: xxxxxxx.xxxxxxx.xxxxxxx_20181112_020000.ucs REMOTE COPY (SCP) FAILED (check for errors above)
iApp DEVICE_BACKUP: FINISHED
@Markie Parkie, I cannot replicate your issue, though I ran into something similar when the destination directory had the wrong permissions (username configured in iApp didn't have write permissions to the directory). Also ensure that you're pasting the OpenSSH b64 private key in this format (with newlines), though I know you've done this successfully elsewhere.
-----BEGIN RSA PRIVATE KEY-----
MIIEoAIBAAKCAQEAvnvuC/FWty8k6vrccyGvNP5uabqtT6CJNpKsfgnN0aHzPQ8T
xiLI007Bad6+2yW38zvUmXe2u49mFA3KGsOn02NIgehCdCQCJQEEuTW+T9W022Z1
dmAqfeFtz9H7tjq9JSoRfJXxl4lMOjB6QD0DhjE9YVpm8wSB8U4Fr560iKNLRBf6
v1UzqcxT2rqnsIlxEXVBCfC5waiNIMVO+Ipfj9ycNDLgrBpCymR6clc9IZmuJPpt
...
-----END RSA PRIVATE KEY-----
Please let me know if you figure anything out!
Does the username or path have any special characters?
These are the fixes/enhancements (I released v3.1.5 and v3.1.6 recently, so they are included):
v3.1.5 - 20181112
Has anyone run into cipher issues. My SCP server was recently hardened and now only except the following cipher "aes128-ctr,aes192-ctr,aes256-ctr" I am getting the following error. I also tried to change the cipher in the template but that gets same results. I am running auto_backup v3.1.8 and I am
generated this Tcl error: script did not successfully complete: (no matching cipher found: client aes128-cbc,aes256-cbc server aes128-ctr,aes192-ctr,aes256-ctr lost connection while executing
@Travis.Kamish, What version of TMOS are you running? Did you read through the help section under "Cipher" when configuring SCP/SFTP in the iApp? What is the output if you manually attempt to ssh or scp from the F5 CLI to the destination server?
@tabernarious I am running 13.1.0.8 and when I try to ssh from CLI to server I get same error. "no matching cipher found: client aes128-cbc,aes256-cbc server aes128-ctr,aes192-ctr,aes256-ctr"
I even tried to set the ssh ciphers.
(tmos) list sys ssh include sys sshd { include "Ciphers aes128-ctr,aes192-ctr,aes256-ctr" }
and with ssh set to none I get same issues with cipher mismatch.
Also if I do this test manually from F5 is works. ssh -vvv -c aes256-ctr username@X.X.X.X
@Travis.Kamish, If you paste "aes256-ctr" into the Cipher field of the iApp does it still fail?
Ok my fault I kept missing the cipher option on scp setup. It is working.
Thanks and sorry for wasting your time.
@Travis.Kamis, No problem at all. I'm just glad it's working for you!!
By the way, I just released v3.1.9 to fix an issue with SMB.
In regards to my issue.... This was fixed if I copied from the shell itself and into the iApp. Copying from notepad++ or any other way apart from within shell it failed.
Worked a treat when I did this.
Thanks..
@Markie Parkie, it may have to do with newline/carriage return formatting etc. I may do some testing to see if I can catch that.
Nice iAPP A cool extra feature would be e-mail notification after the backup was successfully copied to the remote location.