bash
27 TopicsRemoving AAM/WAM for a successful upgrade
If you are wanting to upgrade to version 16 or 17 of BIG-IP, one thing that can cause your config not to load, is any element of AAM/WAM/WOM. As I discovered via a customer of mine, even removing all AAM/WAM items from traffic objects is not enough. While I know how to identify things in the conf files and can see them in iHealth, that doesn't help Admins in the field assess if this is an issue for them, and if it is, how to document what needs to be changed for the necessary approvals. With some help, I wrote this knowledge article to meet these needs as well as provide a way to quickly make the changes - https://my.f5.com/manage/s/article/K000149084 I am sharing this in the forum to not only advertise this, but explain some of the commands and help the community understand how they might be used for other tasks. From spending time running a few BIG-IPs myself in a prior life and working with hundreds of customers, I knew that my solution needed to address partitions and even iApps. My coworker Fernando C provided me the syntax to crawl every partition and I quickly found ways to morph that into this document. Lets take a look at the syntax that can read the lan TCP profiles in the Common partition and then see the changes needed to read all partitions. In order to filter the results a bit better we run these from bash so that we have access to a number of tools like grep, awk, sed, etc. # Return all virtual server names in Common that use a TCP Profile from wam or wom (aka AAM) # grep to find the profile prefixes and then piping that to AWK to grab the third word in the output of each line tmsh list ltm virtual one-line | grep -E "(profiles.*(w(a|o)m-tcp-lan*))" | awk '{print $3}' This simply returns the virtual server name without the partition name. Now to read all partitions, the tmsh portion of the command has to change. Specifically, we pass the -c option to tmsh to tell it to run multiple commands. When you enter tmsh, by default you are in the Common partition, so we have to back out to the root. Because we are in the root directory, we need to add the recursive option to read all subfolders which in this case are the partitions. #Read all partitions and filter for virtual servers that use the wam/wom TCP profiles on the lan or server side tmsh -c 'cd /; list ltm virtual recursive one-line' | grep -E "(profiles.*(w(a|o)m-tcp-lan*))" | awk '{print $3}' Now the output is the partition name and virtual server name, or if iApps are involved, the appservice name as well. You can take the output from the first command and pass it to xarg to use your output as a variable in a command to execute. CAUTION, the following command will attempt to make changes to your config. #Read all partitions and filter for virtual servers that use the wam/wom TCP profiles on the lan or server side then insert new profiles and delete the original profile #This will cause an error tmsh -c 'cd /; list ltm virtual recursive one-line' | grep -E "(profiles.*(w(a|o)m-tcp-lan*))" | awk '{print $3}' | xargs -t -I vsName tmsh modify ltm virtual vsName profiles add { f5-tcp-lan { context serverside } } profiles delete { wam-tcp-lan-optimized } If you run this command, it will error out, because without the proper syntax, tmsh assumes you are referencing objects in the /Common partition and as a result it will help you by implicitly adding that to the beginning of every object in your xarg command. I added the -t option to xarg to output the command that it will execute. To correct the syntax error, in the awk command, you add a forward slash and now tmsh will treat your command as if you have explicitly declared the partition name for every object. Caution - This will make changes to your configuration, very fast... #Read all partitions and filter for virtual servers that use the wam/wom TCP profiles on the lan or server side then insert new profiles and delete the original profile #CAUTION - This will make changes to your system. tmsh -c 'cd /; list ltm virtual recursive one-line' | grep -E "(profiles.*(w(a|o)m-tcp-lan*))" | awk '{print "/" $3}' | xargs -t -I vsName tmsh modify ltm virtual vsName profiles add { f5-tcp-lan { context serverside } } profiles delete { wam-tcp-lan-optimized } When I first hit the wall with xarg beyond the /Common partition, I did not realize what the fix was. However my OCD wanted to see a slash in front of the partition name and I had modified the awk to add it, but had given up on the xarg to modify things outside of /Common. It wasn't until I went to show the error to a peer, Chad T., that I discovered I stumbled upon the proper syntax, and realized I could simplify the instructions quite a bit. Where I would love some help from the community would be on ways to crawl the iApps to quickly disable Strict Updates. The xarg commands to modify/delete objects associated with an iApp will fail if the default setting of "Strict Updates" is enabled. Hope this helps, Carl96Views3likes3CommentsLet's Encrypt with Cloudflare DNS and F5 REST API
Hi all This is a followup on the now very old Let's Encrypt on a Big-IP article. It has served me, and others, well but is kind of locked to a specific environment and doesn't scale well. I have been going around it for some time but couldn't find the courage (aka time) to get started. However, due to some changes to my DNS provider (they were aquired and shut down) I finally took the plunges and moved my domains to a provider with an API and that gave me the opportunity to make a more nimble solution. To make things simple I chose Cloudflare as the community proliferation is enormous and it is easy to find examples and tools. I though think that choosing another provide with an open API isn't such a big deal. After playing around with different tools I realized that I didn't need them as it ended up being much easier to just use curl. So, if the other providers have just a somewhat close resemblance it shouldn't be such a big task converting the scripts to fit. There might be finer and more advanced solutions out there, but my goal was that I needed a solution that had as few dependencies as possible and if I could make that only Bash and Curl it would be perfect. And that is what I ended up with 😎 Just put 5 files in the same directory, adjust the config to your environment, and BAM you're good to go!!😻 And if you need to run it somewhere else just copy the directory over and continue like nothing was changed. That is what I call portability 😁 Find all the details here: Let's Encrypt with Cloudflare DNS and F5 REST API Please just drop me a line if you have any questions or feedback or find any bugs.2.4KViews1like8CommentsBigIP UCS Backup script; looking for some guidance on design
Greetings, I've began to work on a bash script, intended to be ran locally on each F5 appliance via a cron task. The criteria for this script has been, Saves the UCS /w encryption using {Hostname}-YYYY-MM-DD.ucs naming format. Uploads the generated UCS file to a SFTP server SFTP native commands are a MUST, SCP will not work due to it's reliance on command shell/login. Rollover after X # of saved files in order to prevent storage exhaustion on the target SFTP Server I strongly doubt any form of deduplication will work with a encrypted UCS Sends an email notification if the backup failed I've so far written a script that addresses the first 3 criteria and have been waiting for those to go through their paces in testing before adding in notification logic. The commands and logic being used have gotten more complex, the further I've gotten into the script's development. This has lead to some concerns about whether this is the best approach given the nature of the F5 BigIP systems being a vendor appliance and worry that there's a large possibility commands may stop working correctly after a major x. version update, requiring an overhaul of a fairly complex script. I'm almost wondering if setting up an AWX/Tower host in our environment and then using the f5networks Ansible Module for the majority of the heavy lifting followed by some basic logic for file rotation, would be a better long term approach. Ansible would also be a bit more flexible in that I wouldn't have to hardcore values that diverge between individual hosts into the script itself. It's however not clear if the F5networks ansible module supports SFTP as I only see SCP referenced. https://my.f5.com/manage/s/article/K35454259 Advice and insight is much appreciated! #!/bin/bash # F5 backup script based on https://my.f5.com/manage/s/article/K000138297 # User-configurable Variables UCS_DIR="/var/ucs" REMOTE_USER="svc_f5backup" REMOTE_HOST="myhost.contoso.local" REMOTE_DIR="/data/f5/dev" SSH_KEY="/shared/scripts/f5-backup/mykeys/f5user" ENCRYPTION_PASSPHRASE='' # Blank out the value to not encrypt the UCS backup. LOG_FILE="/var/log/backupscript.log" MAX_FILES=45 # Maximum number of backup files to keep # Dynamic Variables (do not edit) HOSTNAME=$(/bin/hostname) DATE=$(date +%Y-%m-%d) UCS_FILE="${UCS_DIR}/${HOSTNAME}-${DATE}.ucs" # Start logging echo "$(date +'%Y-%m-%d %H:%M:%S') - Starting backup script." >> ${LOG_FILE} # Save the UCS backup file if [ -n "${ENCRYPTION_PASSPHRASE}" ]; then echo "Running the UCS save operation (encrypted)." >> ${LOG_FILE} tmsh save /sys ucs ${UCS_FILE} passphrase "${ENCRYPTION_PASSPHRASE}" >> ${LOG_FILE} 2>&1 else echo "Running the UCS save operation (not encrypted)." >> ${LOG_FILE} tmsh save /sys ucs ${UCS_FILE} >> ${LOG_FILE} 2>&1 fi # Create a temporary batch file for SFTP commands BATCH_FILE=$(mktemp) echo "cd ${REMOTE_DIR}" > $BATCH_FILE echo "put ${UCS_FILE}" >> $BATCH_FILE echo "bye" >> $BATCH_FILE # Log that the transfer is starting echo "Starting SFTP transfer." >> ${LOG_FILE} # Execute SFTP command and capture the output transfer_command_output=$(sftp -b "$BATCH_FILE" -i "${SSH_KEY}" -oBatchMode=no "${REMOTE_USER}@${REMOTE_HOST}" 2>&1) transfer_status=$? # Extract the "Transferred:" line transfer_summary=$(echo "$transfer_command_output" | grep "^Transferred: sent") if [ $transfer_status -eq 0 ]; then if [ -n "$transfer_summary" ]; then echo "UCS file copied to the SFTP server successfully (remote:${REMOTE_HOST}:${REMOTE_DIR}/${UCS_FILE}). $transfer_summary" >> ${LOG_FILE} else echo "UCS file copied to the SFTP server successfully (remote:${REMOTE_HOST}:${REMOTE_DIR}/${UCS_FILE}). Please check the log for details." >> ${LOG_FILE} fi else echo "$transfer_command_output" >> ${LOG_FILE} echo "UCS SFTP copy operation failed. Please read the log for details." >> ${LOG_FILE} rm -f $BATCH_FILE exit 1 fi # Clean up the temporary batch file rm -f $BATCH_FILE # Rollover backup files if the number exceeds MAX_FILES echo "Checking and maintaining the maximum number of backup files." >> ${LOG_FILE} # Create a list of files to delete sftp -i "${SSH_KEY}" -oBatchMode=no "${REMOTE_USER}@${REMOTE_HOST}" <<EOF > file_list.txt cd ${REMOTE_DIR} ls -1 ${HOSTNAME}-*.ucs bye EOF # Filter out unwanted lines and sort the files alphanumerically grep -v 'sftp>' file_list.txt | grep -v '^cd ' | sort > filtered_file_list.txt # Determine files to delete files_to_delete=$(head -n -${MAX_FILES} filtered_file_list.txt) if [ -n "$files_to_delete" ]; then # Create a temporary batch file for SFTP cleanup commands CLEANUP_BATCH_FILE=$(mktemp) echo "cd ${REMOTE_DIR}" > $CLEANUP_BATCH_FILE for file in $files_to_delete; do echo "Deleting $file" >> ${LOG_FILE} echo "rm $file" >> $CLEANUP_BATCH_FILE done echo "bye" >> $CLEANUP_BATCH_FILE # Execute SFTP cleanup command and log the output cleanup_command_output=$(sftp -b "$CLEANUP_BATCH_FILE" -i "${SSH_KEY}" -oBatchMode=no "${REMOTE_USER}@${REMOTE_HOST}" 2>&1) echo "$cleanup_command_output" >> ${LOG_FILE} # Clean up the temporary batch file rm -f $CLEANUP_BATCH_FILE else echo "No files to delete. Total files within limit." >> ${LOG_FILE} fi # Clean up the file lists rm -f file_list.txt filtered_file_list.txt # Delete the local copy of the UCS archive tmsh delete /sys ucs ${UCS_FILE} >> ${LOG_FILE} 2>&1 echo "$(date +'%Y-%m-%d %H:%M:%S') - Backup script completed." >> ${LOG_FILE}227Views0likes2CommentsIssue with a simple Bash Script for adding an iRule to a list of Virtual Servers.
Hello Community, I am having an issue with a bash script for an F5 BIG-IP Load Balancer, intended to read and iterate over a .txt list of Virtual Server names, look up the partition for a given VS, and add an iRule to it. When running the script I am only hitting the outermost 'else' statement for being unable to find the partition and VS name. My script logic is based on F5 Support solution K41961653: <p> #!/bin/bash # Prompt the user for the iRule name and read it into the 'new' variable echo "Please enter the iRule name:" read new noneRules='rules none' while IFS= read -r vs_name; do # Retrieve the partition and virtual server name full_vs_info=$(tmsh -c "cd /; list ltm virtual recursive" | grep "$vs_name" | grep -m1 "^ltm virtual") echo "Full VS Info Debug: $full_vs_info" # Extract the partition and virtual server name from the retrieved information if [[ $full_vs_info =~ ltm\ virtual\ (.+)/(.+) ]]; then partition="${BASH_REMATCH[1]}" vs_name="${BASH_REMATCH[2]}" # Format the tmsh command to include the partition rule=$(tmsh list ltm virtual /$partition/$vs_name rules | egrep -v "\{|\}" | xargs) if [[ "$rule" == "$noneRules" ]]; then tmsh modify ltm virtual /$partition/$vs_name rules { $new } echo "iRule $new was added to $vs_name in partition $partition" else# tmsh modify ltm virtual /$partition/$vs_name rules { $rule $new } echo "iRules $rule were conserved and added $new to $vs_name in partition $partition" fi else echo "Could not find partition and virtual server name for $vs_name" fi done < /shared/tmp/test_list.txt tmsh save sys config </p> As far as I was able to troubleshoot, the problem I am encountering appears to be with line 11 of my script where I attempt to assign the string "ltm virtual SomePartition/VS_Example.com {" to the "full_vs_info" variable using: full_vs_info=$(tmsh -c "cd /; list ltm virtual recursive" | grep "$vs_name" | grep -m1 "^ltm virtual") When I run the tmsh command [tmsh -c "cd /; list ltm virtual recursive" | grep "VS_Example.com" | grep -m1 "^ltm virtual"] on its own, from the F5's Bash shell, I am getting the output I expect: "ltm virtual SomePartition/VS_Example.com {" However, when I run the script with the debug echo , it only outputs "Full VS Info Debug:", and ends the script with "Could not find partition and virtual server name for $vs_name" and a sys config save. I am attempting to run this on a BIG-IP, version 15.1.10.2, build 0.44.2. I am quite new to both Bash scripting and F5 LBs. All feedback and criticism is highly appreciated! Thanks in advance!105Views0likes2Comments/var/ directory running out of space
I'm getting a broadcast message on the bash CLI that warns me about the /var/ directory running out of room. 011d0004:3: Disk partition /var has only 14% free There are four Tomcat files of identical size taking up almost a GB of space, but I don't know what purpose they serve. -rw-r--r--. 1 tomcat tomcat 240192271 2017-11-05 19:51 1509911444508.upload -rw-r--r--. 1 tomcat tomcat 240192271 2017-11-05 10:46 1509878765278.upload -rw-r--r--. 1 tomcat tomcat 240192271 2017-11-05 10:39 1509878372606.upload -rw-r--r--. 1 tomcat tomcat 240192271 2017-11-05 10:38 1509878267154.upload Does anyone know what the purpose of these files are, and whether or not it's safe to remove them?232Views0likes1CommentExternal monitor Realsec Cryptosec HSM
Hi all, I am trying to monitor a HSM appliance using the external monitor template provided on this link: link text I modified this part in the template so it should send CCCCNC and the response 00000000, I really am not sure if this is the correct line. I uploaded the script and attached it to the pool and it is available and actively sending monitor request towards the pool members, however looking at the payload in wireshark there is no data being send. I verify it in wireshark (Follow TCP stream). Send the request request and check the response echo -n 'CCCCNC' | nc $IP $PORT | grep "00000000" 2>&1 > /dev/null Someone over here with some bash scripting experience? Thanks in advance.258Views0likes1CommentCombine and Modify Grep Output
Hi, I'm trying to write a script to output virtual server stats. Can anyone help me modify the output? Right now the GREP output looks like this: Ltm::Virtual Server: www.website.com.443 Bits In 2.1G 0 - Bits Out 2.3G 0 - Ltm::Virtual Server: www.website.com.80 Bits In 1.4G 0 - Bits Out 740.8M 0 - I need help making the output like this: www.website.com.443 Bits In 2.1G Bits Out 2.3G www.website.com.80 Bits In 1.4G Bits Out 740.8M Afterwards, I can diff files from different dates, and figure out which VIPS are not being used.382Views0likes2CommentsWindows-File-Share-Monitor-SMB-CIFS
Hi, I am trying to use the: https://devcentral.f5.com/wiki/AdvDesignConfig.Windows-File-Share-Monitor-SMB-CIFS.ashx?lc=1 In the article the monitor for gtm is detailed as: monitor "smb_external_monitor" { defaults from "external" interval 10 timeout 40 probe_interval 1 probe_timeout 5 probe_num_probes 1 probe_num_successes 1 dest *:* "SEARCH_STRING" "got it" "DEBUG" "1" run "smb_monitor.bash" "USERNAME" "aaron" "FILE" "/share/test.txt" args "" "PASSWORD" "Test123!" partition "Common" } My monitor is 11.5.1 so the tmsh syntax is a little different: gtm monitor external /Common/smb_external_monitor { defaults-from /Common/external destination *:* interval 30 probe-timeout 5 run /Common/smb_monitor.bash timeout 120 user-defined DEBUG 1 user-defined FILE /F5GTM/F5GTMTST.txt user-defined PASSWORD ****** user-defined SEARCH_STRING up user-defined USERNAME f5gtm } I have also tried manually setting the debug to 1 in the script as suggested. I get nothing in /var/log/ltm and the monitor is failing. Any ideas? Thanks, Ben436Views0likes1CommentAnsible Module for bash against F5 LTM
Hi folks, I'm trying to find an Ansible module that will actually work for bash against F5 LTMs. I've tried command, shell, and ansible.builtin.shell with no luck. Alternatively an Ansible module that could execute a shell script already on the F5 LTMs would work as well. Here are a couple examples of the bash commands I'm trying to execute: tmsh save sys ucs lb1.ucs scp /var/local/ucs/lb1.ucs admin@192.168.0.1:/var/local/ucs/ tmsh load sys ucs base.ucs sleep 120 tmsh load sys ucs platform-migrate lb1.ucs sleep 120 tmsh modify cm traffic-group traffic-group-1 ha-order none tmsh modify cm device-group Employee_Sync_Failover devices none tmsh delete cm trust-domain all tmsh modify cm device lb1.fb configsync-ip none unicast-address none mirror-ip any6 tmsh delete net route all tmsh delete net self all tmsh delete net vlan all tmsh modify sys global-settings mgmt-dhcp enabled tmsh save sys ucs USE2-LBEMPL01A.ucs cd /opt/aws/awscli-2.2.29/bin/dist ./aws s3 cp /var/local/ucs/lb2.ucs s3://f5-bubble-sync-fb5095-us-east-2/lb2/lb2.ucs916Views0likes3CommentsMigration Test Script for LTM
Hello community! I have done quite a few F5 to F5 migrations now. But I have always wondered about utilizing some script to test the applications functionality after a cutover beyond just looking at if the VIP's, pools, nodes are marked as available and running one off curls or tcpdump/ssldumps. Let's start with just LTM. (Although having something similar for GTM would be nice). I was thinking of a bash script or something similar that would: 1) Run curls to all VIP's and the respective ports 2) Collect the relevant results for only important data so it's not so busy (perhaps HTTP response code and 2-3 lines of the body message) 3) Take file of output manually from both F5's (old and new) and compare content in diff editor (notepad++ or beyond compare, etc) A lot of these migrations I have done, all the IP addresses stay the same (VIP's & pool members). It would be nice if the script were able to just match the VIP name in the results as most migrations keep the name the same if the IP addresses do move. Any suggestions? Thanks!218Views0likes1Comment