BigIP UCS Backup script; looking for some guidance on design
Greetings, I've began to work on a bash script, intended to be ran locally on each F5 appliance via a cron task. The criteria for this script has been, Saves the UCS /w encryption using {Hostname}-YYYY-MM-DD.ucs naming format. Uploads the generated UCS file to a SFTP server SFTP native commands are a MUST, SCP will not work due to it's reliance on command shell/login. Rollover after X # of saved files in order to prevent storage exhaustion on the target SFTP Server I strongly doubt any form of deduplication will work with a encrypted UCS Sends an email notification if the backup failed I've so far written a script that addresses the first 3 criteria and have been waiting for those to go through their paces in testing before adding in notification logic. The commands and logic being used have gotten more complex, the further I've gotten into the script's development. This has lead to some concerns about whether this is the best approach given the nature of the F5 BigIP systems being a vendor appliance and worry that there's a large possibility commands may stop working correctly after a major x. version update, requiring an overhaul of a fairly complex script. I'm almost wondering if setting up an AWX/Tower host in our environment and then using the f5networks Ansible Module for the majority of the heavy lifting followed by some basic logic for file rotation, would be a better long term approach. Ansible would also be a bit more flexible in that I wouldn't have to hardcore values that diverge between individual hosts into the script itself. It's however not clear if the F5networks ansible module supports SFTP as I only see SCP referenced. https://my.f5.com/manage/s/article/K35454259 Advice and insight is much appreciated! #!/bin/bash # F5 backup script based on https://my.f5.com/manage/s/article/K000138297 # User-configurable Variables UCS_DIR="/var/ucs" REMOTE_USER="svc_f5backup" REMOTE_HOST="myhost.contoso.local" REMOTE_DIR="/data/f5/dev" SSH_KEY="/shared/scripts/f5-backup/mykeys/f5user" ENCRYPTION_PASSPHRASE='' # Blank out the value to not encrypt the UCS backup. LOG_FILE="/var/log/backupscript.log" MAX_FILES=45 # Maximum number of backup files to keep # Dynamic Variables (do not edit) HOSTNAME=$(/bin/hostname) DATE=$(date +%Y-%m-%d) UCS_FILE="${UCS_DIR}/${HOSTNAME}-${DATE}.ucs" # Start logging echo "$(date +'%Y-%m-%d %H:%M:%S') - Starting backup script." >> ${LOG_FILE} # Save the UCS backup file if [ -n "${ENCRYPTION_PASSPHRASE}" ]; then echo "Running the UCS save operation (encrypted)." >> ${LOG_FILE} tmsh save /sys ucs ${UCS_FILE} passphrase "${ENCRYPTION_PASSPHRASE}" >> ${LOG_FILE} 2>&1 else echo "Running the UCS save operation (not encrypted)." >> ${LOG_FILE} tmsh save /sys ucs ${UCS_FILE} >> ${LOG_FILE} 2>&1 fi # Create a temporary batch file for SFTP commands BATCH_FILE=$(mktemp) echo "cd ${REMOTE_DIR}" > $BATCH_FILE echo "put ${UCS_FILE}" >> $BATCH_FILE echo "bye" >> $BATCH_FILE # Log that the transfer is starting echo "Starting SFTP transfer." >> ${LOG_FILE} # Execute SFTP command and capture the output transfer_command_output=$(sftp -b "$BATCH_FILE" -i "${SSH_KEY}" -oBatchMode=no "${REMOTE_USER}@${REMOTE_HOST}" 2>&1) transfer_status=$? # Extract the "Transferred:" line transfer_summary=$(echo "$transfer_command_output" | grep "^Transferred: sent") if [ $transfer_status -eq 0 ]; then if [ -n "$transfer_summary" ]; then echo "UCS file copied to the SFTP server successfully (remote:${REMOTE_HOST}:${REMOTE_DIR}/${UCS_FILE}). $transfer_summary" >> ${LOG_FILE} else echo "UCS file copied to the SFTP server successfully (remote:${REMOTE_HOST}:${REMOTE_DIR}/${UCS_FILE}). Please check the log for details." >> ${LOG_FILE} fi else echo "$transfer_command_output" >> ${LOG_FILE} echo "UCS SFTP copy operation failed. Please read the log for details." >> ${LOG_FILE} rm -f $BATCH_FILE exit 1 fi # Clean up the temporary batch file rm -f $BATCH_FILE # Rollover backup files if the number exceeds MAX_FILES echo "Checking and maintaining the maximum number of backup files." >> ${LOG_FILE} # Create a list of files to delete sftp -i "${SSH_KEY}" -oBatchMode=no "${REMOTE_USER}@${REMOTE_HOST}" <<EOF > file_list.txt cd ${REMOTE_DIR} ls -1 ${HOSTNAME}-*.ucs bye EOF # Filter out unwanted lines and sort the files alphanumerically grep -v 'sftp>' file_list.txt | grep -v '^cd ' | sort > filtered_file_list.txt # Determine files to delete files_to_delete=$(head -n -${MAX_FILES} filtered_file_list.txt) if [ -n "$files_to_delete" ]; then # Create a temporary batch file for SFTP cleanup commands CLEANUP_BATCH_FILE=$(mktemp) echo "cd ${REMOTE_DIR}" > $CLEANUP_BATCH_FILE for file in $files_to_delete; do echo "Deleting $file" >> ${LOG_FILE} echo "rm $file" >> $CLEANUP_BATCH_FILE done echo "bye" >> $CLEANUP_BATCH_FILE # Execute SFTP cleanup command and log the output cleanup_command_output=$(sftp -b "$CLEANUP_BATCH_FILE" -i "${SSH_KEY}" -oBatchMode=no "${REMOTE_USER}@${REMOTE_HOST}" 2>&1) echo "$cleanup_command_output" >> ${LOG_FILE} # Clean up the temporary batch file rm -f $CLEANUP_BATCH_FILE else echo "No files to delete. Total files within limit." >> ${LOG_FILE} fi # Clean up the file lists rm -f file_list.txt filtered_file_list.txt # Delete the local copy of the UCS archive tmsh delete /sys ucs ${UCS_FILE} >> ${LOG_FILE} 2>&1 echo "$(date +'%Y-%m-%d %H:%M:%S') - Backup script completed." >> ${LOG_FILE}Import PKCS 12 SSL to Device Certificate via API/Script or CLI on BIG-IP
We have more than 160 BIG-IP Virtual Edition with version 15.1.10.3 build 0.0.12. We need to import, in each one, an SSL Certificate in PFX/PKCS 12 format in the path System ›› Certificate Management: Device Certificate Management: Device Certificate. We looked in the documentation and the KB but we couldn't find a way to do it. Has anyone dealt with this and have a solution to do it via Script, CLI or API? Thank you.
Notifications via External Monitor
Hi, I have an external monitor that sends a notification (email) when it fails. I don't want it failing so I just had it create a file when if fails. If that file doesn't exist, it will notify. If it does exist, it just marks it down but doesn't send the alert. The way I thought to clean it up was just to rm-f the file when the status goes back to UP. This works fine if the script is run manually, but will not work when it runs through F5. Here are the basics of the script: curl -s -v -k -H "Host: ${HOST}" -k https://${NODE}${URI} 2>&1 > /dev/null | grep -i "${RECV}" STATUS=$? rm -f $PIDFILE if [ $STATUS -eq 0 ] then rm -f /shared/tmp/service-${NODE} echo "UP" else if [ ! -f /shared/tmp/service-${NODE} ] then touch /shared/tmp/service-${NODE} chmod 777 /shared/tmp/service-${NODE} curl -s -o /shared/tmp/service-${NODE}.html -H "Host: ${HOST}" -k https://${NODE}${URI} EMAIL NOTIFICATION PART fi fi exit Is there something that I am missing? Is there a better way?Upgrade Verification - CLI tips and tricks
I mentioned recently in a conversation that I use to use the CLI to generate a snapshot of the bigip run state for pre/post comparison after an upgrade. By accident, I ran across these scripts today so here they are for your enjoyment. They are circa big-ip v12 so update as needed, feel free to post updated versions in the reply and i will tick them as the answer even. Pre-Change (Now supports partitions) # tmsh -c "cd /; show ltm virtual recursive" | awk '/Ltm::Virtual/ { printf $NF } /(Availability|State)/ { printf ":"$NF } /Reason/ { print ":"$NF} ' > bigip-virtual-state.pre # tmsh -c "cd /; show ltm pool recursive" | awk '/Ltm::Pool/ { printf $NF } /(Availability|State)/ { printf ":"$NF } /Reason/ { print ":"$NF} ' > bigip-pool-state.pre Post Change # tmsh -c "cd /; show ltm virtual recursive" | awk '/Ltm::Virtual/ { printf $NF } /(Availability|State)/ { printf ":"$NF } /Reason/ { print ":"$NF} ' > bigip-virtual-state.post # tmsh -c "cd /; show ltm pool recursive" | awk '/Ltm::Pool/ { printf $NF } /(Availability|State)/ { printf ":"$NF } /Reason/ { print ":"$NF} ' > bigip-pool-state.post Comparison # diff bigip-virtual-state.pre bigip-virtual-state.post # diff bigip-pool-state.pre bigip-pool-state.post
string variable
Need to create variable from string with multiple delimiters Example: TXT=my Variable Value, with spaces and other characters, TXT2=More Data for Second variable, TXT3=Additional info, convert to: myVar1 = my Variable Value, with spaces and other characters myVar2 = More Data for Second variable myVar3 = Additional info if I use split with the = and , that works, but I can't use , as a delimiter as some of the strings might have commas, each string will end in a comma, but may have non delimited commas in the string as well. The = sign is always after the string name, and I need to assign the variable based on the string name, but don't want to keep the TXT= as part of the variable. TXT value is either one or two spaces, but for each variable it will always be the same, so myVar1, would always be 3 spaces plus the = as TXT= where myVar2 will always be 4 spaces as TXT1 and the = Would prefer to not use regular expressions, for performance reasons, but may have to if no other solution exists.
OS X F5 Access Scripting
Hello All, Is there any way to script the F5 Access client on the Mac? The documentation does not indicate that any scripting language can be used. AppleScript is not an option because all of tcc is now behind SIP, so we'd like to do BASH or Pyton, but Swift/Coca would also be acceptable if that was my only option. I would like to... Install the app via VPP (using my MDM for this) Configure it to launch through a LaunchAgent Create a new configuration Manage (Enable/Open and Disable/Close) a configuration If none of that is possible, does anybody know if the F5 environment can be configured to allow the Mac's built-in VPN (L2TP over IPSec, IKEv2 or Cisco IPSec) clients and what that configuration may look like. If I can be pointed to the right documentation or if anybody has examples, I would greatly appreciate the assist. Thank You, Nick Lucia
APM special cases
Hi All/DC Experts, I have a question 2 question regarding Access Policy Manager. First scenario, I have users that is member of multiple groups, does f5 can automatically merged resources if it detects that this user is member of multiple groups? Second I have a users of multiple groups and I just want that this user only can use this specific resources even though he is member of a multiple groups. THank you everyone, I am hoping that you can help me with this. -Nathanautomation - failing over wideips between two data centers
Hey guys, Been sort of at this on for a while now. Here Ill post a general outline of how we have our environment configured and questions to along with as it pertains to what we setup. As of now we have two DCs with GTMS that face internal and external. On the GTM we have the two vips in a pool configured with global availability. So say when an application needs to be failover from one DC to another I would go and change the weight of the member from 0 to 1 or 1 to 0 depending on the location of the DC. I have a script that contains a bunch of tmos commands that modifies the pool member to go failover from 1 to 0 or 0 to 1 that are actually in separate scripts that are on the internal and external facing gtms. I am looking for some ideas/suggestions as to ways to improve the script or maybe another way all together to preform this so that I can either run a script or automate it all together. The other tricky is that alot of applications are mission critical with most of the run on certain intervals either within the hour and we have to work with multiple teams to move pieces of the application/infrastructure over to our other DC. Tanks
