BIGdiff Version 2

This article is written by, and published on behalf of, DevCentral MVP Leonardo Souza.

---

Introduction

I wrote the first version of BIGdiff script almost 2 years ago, since that release I have been using the script and keeping a list of new functionalities and improvements I would like to add. BIGdiff Version 2 includes all the things on my wish list which I will describe in this article.

First, if you have no idea what is BIGdiff, please read the article I wrote about version 1. https://devcentral.f5.com/s/articles/bigdiff-a-little-help-for-software-upgrades-31933

In this article I will only explain the new functionalities and improvements, so read the above article for the basics about the script.

Use Cases

When I wrote Version 1, the goal was to help with software upgrades, but I ended up using the script for any kind of change involving F5 devices. In Version 2, the scope is increased to include scenarios where you need to understand if something changed related to an F5 device.

Because the script points you to each object that has changed, the change can be anything; for example, a firewall.
You can basically, 

• run the script before the firewall change is made, 
• make the firewall change, 
• and run the script after the firewall change is complete.

The output will point you to problems for analysis and, if you conclude that the problem is not because of the F5 device, you can report problems to the application team at the time of the change rather than the next day.

This immediacy will save you a lot of time troubleshooting and reduce the pain your users may experience.

Associative Arrays

If you run the Version 2 script while upgrading a device, the script will be extremely fast. As indicated in the article about Version 1, it just takes a couple of minutes to compare 13 thousand objects. That is because the script was optimized for upgrades, where the list of objects before and after have the same order.

Internally the script will create an array (by type of object e.g., virtual server) listing objects before the change and another array listing objects after the change. The script then checks if the array index values match; if they have the same object name. If the index values match, like in the case of an upgrade, the search is done. If the index values do not match, the script scans the array with objects from after the change, this is a resource-intensive task and basically exponential.

The fix to reduce the 'cost' of the exponential back down to a linear cost is to use an associative array, but associative arrays were only introduced in Bash version 4. Version 2 of the BIGdiff script, supports versions starting from 11.0.0 but version 11.0.0 uses Bash 3. Somewhere in 11.5.x and 11.6.x, Bash was upgraded to version 4, and 12.0.0 uses Bash 4.

As version 11.x.x is becoming less common now, it is time to add associative arrays to the script. However, the script will only use associative arrays if it detects that the current Bash version is 4 or higher.

To give you an idea about the improvement in running times I created a 'worst-case scenario' where most of the objects have a new name and the script has to scan the whole array.

Note: These times were while using my slow virtual lab, so you will definitely get faster times in a hardware device or a virtual device with more resources.

Number of objects: 3398

• BIGdiff v1: 1203 seconds
• BIGdiff v2: 596 seconds
• Time reduction: around 50%

Number of objects: 6648

• BIGdiff v1: 2868 seconds
• BIGdiff v2: 1319 seconds
• Time reduction: around 54%

Number of objects: 9898

• BIGdiff v1: 6865 seconds
• BIGdiff v2: 2219 seconds
• Time reduction: around 68%

As the number of objects increases, so to does the reduction in time (by percentage) to run the script when compared with Version 1. Using associative arrays Bash will internally look for the object using a key, and that key is visible faster than scanning the entire array.

DataTables

BIGdiff shows the data with HTML tables.

• Version 1 used TableFilter (https://www.tablefilter.com/) to provide filtering functionality.
• Version 2 uses DataTables (https://datatables.net/) to provide filtering functionality.

DataTables provides more functionality, including breaking the table in multiple pages. Also, now there is no need to download an extra file to provide filtering options because when loading the HTML file the browser will download the required files from CDN networks.

Backup

Version 2 now generates UCS/QKView/Logs. When you run the script it will automatically generate those files, but you can also opt to run the script without generating those files. You may also choose to generate the log files individually using the script.

Object Names

LTM does not support creating an object with a space in the name. However, GTM does accept spaces in names for most objects. The Version 2 script now accepts space in the object name for all objects.

BIGdiff Remote

I created another script called BIGdiff Remote that will automatically

• upload BIGdiff to the destination devices
• run the BIGdiff
• download the files created.

You might use BIGdiff Remote when you are running BIGdiff against multiple devices, as the script will automate that task. Also, if you want to get a UCS/QKView/full logs from multiple devices, the script automates that activity as well.

Conclusion

You should use this BIGdiff script during an upgrade because you don’t lose anything and it will definitely help if things go wrong. I also suggest including the script as part of your change process, if you have one. Normally you (should) generate a UCS file before performing a change to an F5 device, so upgrade that process to use this script and it will not only generate the required UCS file but also QKView and full logs. All of this helps identify whether something was broken during your change.

 

Check out the new version on DevCentrals' Codeshare:

https://devcentral.f5.com/s/articles/bigdiff-1160