BIG-IP Upgrade Procedure Using CLI (vCMP Guest & Host)

Problem this snippet solves:

Next article describes an upgrade procedure to perform only using CLI commands.


The idea is not to replace an official procedure, but to give a different approach for those guys who love using CLI and they want to execute an upgrade only using commands (without GUI access).


The procedure is separated in 4 sections:


  • Data Collection & Planning - for executing some days before the upgrade.
  • Pre-Upgrade Tasks - for executing just before the upgrade (applies to all devices in the cluster).
  • Upgrade Tasks - Only applies for one device in the cluster for each time (normally standby device).
  • Post-Upgrade Tasks - for executing just after the upgrade (applies to all devices in the cluster).


This procedure is valid for most of the BIP-IP set-ups:

  • Standalone & clusters
  • vCMP Host & vCMP Guests
  • GTM/DNS Synchronization Groups


Everything that helps to fix mistakes is great, so your comments are welcome.


OFFICIAL REFERENCES:


How to use this snippet:


>> DATA COLLECTION & PLANNING (ALL CLUSTER DEVICES)



>> PRE-UPGRADE TASKS (ALL CLUSTER DEVICES)



>> UPGRADE TASKS (ONE DEVICE AT TIME)



>> POST-UPGRADE TASKS (ALL CLUSTER DEVICES)



Code :

######################################################
## DATA COLLECTION & PLANNING (ALL CLUSTER DEVICES) ##
######################################################

## Capture Product Code & Serial Number
tmsh show sys hardware

## Capture Management IP & Blade State
tmsh show sys cluster

## Capture Provision State
tmsh list sys provision

## Capture Release and Volume Info
tmsh show sys software

## Capture Master-key
tmsh show sys crypto

## Check Relicensing Needed
tmsh show sys license | grep -i 'service check date'
REF - https://support.f5.com/csp/article/K7727

## Check Certificate Expiration
openssl x509 -noout -text -in /config/httpd/conf/ssl.crt/server.crt | grep Validity -A2
REF - https://support.f5.com/csp/article/K6353

## Check RAID Integrity
tmsh show sys raid
tmsh run util platform_check
cat /var/log/user.log
cat /var/log/kern.log

## Check Mirroring Enabled
tmsh show sys connection type mirror
tmsh show sys ha-mirror

## Check Upgrade Disk Space (At least 20Gb)
vgs

## Check ZebOS Module Running
vtysh
zebos/rdX/ZebOS.conf
>> 'X' REPRESENTS ROUTE DOMAIN ID

## ONLY GTM/DNS - Check Devices Managed by GTM
tmsh show gtm iquery all

## ONLY GTM/DNS - Check if DNSSEC keys in FIPS are Synchronized
tmsh show sys crypto fips

## Capture QKView (Upload to iHealth)
qkview
REF - https://ihealth.f5.com/qkview-analyzer/

## Check Release Notes For Specific Details
REF - https://support.f5.com/csp/knowledge-center/software/BIG-IP

## Upload Release Image
scp -p / @:/shared/images/

## Upload MD5 Hash Image
scp -p / @:/shared/images/

## Upload Script to Check Pool Status
scp -p /Check_Pool_Status.sh @:/shared/tmp/
REF - https://github.com/DariuSGB/F5_Bash/blob/master/Check_Pool_Status.sh


#############################################
## PRE-UPGRADE TASKS (ALL CLUSTER DEVICES) ##
#############################################

## Disable Virtual Server Mirroring
REF - https://support.f5.com/csp/article/K13478

## Disable Config Auto-Sync (if enabled)
tmsh modify cm device-group  auto-sync disabled

## ONLY GTM/DNS - Disable GSLB/ZoneRunner Synchronization
tmsh modify gtm global-settings general { synchronization no synchronize-zone-files no auto-discovery no }

## Save Running Config
tmsh save sys config

## Check HA Cluster Synchronization
tmsh show cm sync-status
tmsh run cm config-sync to-group 

## Check Release Image Integrity
cd /shared/images/
md5sum -c 

## Create Initial UCS (Backup)
tmsh save sys ucs /shared/tmp/$(date '+%Y%m%d')_initial.ucs

## Capture Initial Config
tmsh save sys config file /shared/tmp/$(date '+%Y%m%d')_initial.scf no-passphrase

## Capture Initial Pool Status
/shared/tmp/Check_Pool_Status.sh > /shared/tmp/$(date '+%Y%m%d')_initial_pools_output.txt

## Check No Upgrade Process Running
tmsh show sys software status

## OPTIONAL - Get More Free Disk Space (At least 20Gb)
tmsh delete sys software volume 
vgs


########################################
## UPGRADE TASKS (ONE DEVICE AT TIME) ##
########################################

## Restart AOM to Prevent Licensing Problems (iSeries)
ipmiutil reset -k
REF - https://support.f5.com/csp/article/K00415052

## ONLY VCMP HOST - Check That All Guests Are In Standby
tmsh show vcmp guest
>> ACCESS INDIVIDUALLY TO EACH GUEST
tmsh show cm sync-status

## ONLY VCMP HOST - Deprovision All Guests (Configured)
tmsh show vcmp guest
>> EXECUTE FOR EACH GUEST
tmsh modify vcmp guest  state configured
tmsh save sys config

## Re-licensing Device
>> BIG-IP WITH INTERNET ACCESS
tmsh install sys license registration-key  add-on-keys {  }
REF - https://support.f5.com/csp/article/K15055
>> BIG-IP WITHOUT INTERNET ACCESS
cp /config/bigip.license /config/bigip.license.backup
get_dossier -b  -a 
** ACCESS LICENSE ACTIVATION
https://activate.f5.com/license/dossier.jsp
** PASTE LICENSE FILE (ENTER 'CTRL+D' AFTER PASTING)
cat > /config/bigip.license
reloadlic
REF - https://support.f5.com/csp/article/K2595

## Force Offline Mode
tmsh run sys failover offline

## Verify Configuration Integrity
tmsh load sys config verify

## Install Image
tmsh install sys software image  create-volume volume 

## Check Installation State
tmsh show sys software status
cat /var/log/liveinstall.log

## OPTIONAL - Copy Configuration To New Volume
## (Only if you have made changes since installation)
clsh --slot=X,Y cpcfg 
>> FROM VIPRION
cpcfg 
>> FROM NOT VIPRION

## Boot On New Volume
tmsh reboot volume 

## ONLY VCMP GUEST - Check Boot Up Status
>> FROM VCMP HOST
vconsole  

## Check Logs (LTM, APM, ASM,...)
REF - https://support.f5.com/csp/article/K16197

## Capture Final Config
tmsh save sys config file /shared/tmp/$(date '+%Y%m%d')_final.scf no-passphrase

## Compare Initial-Final Config
tmsh show sys config-diff /shared/tmp/$(date '+%Y%m%d')_initial.scf /shared/tmp/$(date '+%Y%m%d')_final.scf | egrep -e "\s{3}\|\s{3}" -e "[<]$" -e "^\s*[>]"

## Disable Force Offline
tmsh run sys failover online

## ONLY GTM/DNS - Enable Metrics Collection
tmsh start sys service big3d

## Capture Final Pool Status
/shared/tmp/Check_Pool_Status.sh > /shared/tmp/$(date '+%Y%m%d')_final_pools_output.txt

## Compare Initial-Final Pool Status
diff /shared/tmp/$(date '+%Y%m%d')_initial_pools_output.txt /shared/tmp/$(date '+%Y%m%d')_final_pools_output.txt

## ONLY VCMP HOST - Deploy All Guests (Deployed)
tmsh show vcmp guest
tmsh modify vcmp guest  state deployed

## FROM ACTIVE NODE - Check Current Connections
tmsh show sys traffic raw

## FROM ACTIVE NODE - Force Failover Event
tmsh run sys failover standby

## Check CPU/Memory status
tmsh show sys cpu
tmsh show sys memory

## Check Current Connections
tmsh show sys traffic raw

## Perfom Other Custom Tests Here
...


##############################################
## POST-UPGRADE TASKS (ALL CLUSTER DEVICES) ##
##############################################

## OPTIONAL - Install Big3d daemon in all managed members
## (Only necessary if you upgrade GTM/DNS before its members)
big3d_install 
REF - https://support.f5.com/csp/article/K11661449#update-big3d

## ONLY GTM/DNS - Enable GSLB/ZoneRunner Synchronization
tmsh modify gtm global-settings general { synchronization yes synchronize-zone-files yes auto-discovery yes }

## Re-enable Virtual Server Mirroring
REF - https://support.f5.com/csp/article/K13478

## Synchronize HA Cluster
tmsh show cm sync-status
tmsh run cm config-sync force-full-load-push to-group 

## Re-enable Config Auto-Sync (if enabled)
tmsh modify cm device-group  auto-sync enabled

## Save running config
tmsh save sys config

## Create Final UCS (Backup)
tmsh save sys ucs /shared/tmp/$(date '+%Y%m%d')_final.ucs

## Delete Unused Images
delete sys software image 

## Delete Unused Volumes (Mandatory reboot)
delete sys software volume

Tested this on version:

12.1
Published Jun 20, 2019
Version 1.0
application delivery
BIG-IP
TMSH
upgrade
No CommentsBe the first to comment