BIG-IP Geolocation Updates – Part 4

Introduction

Management of geolocation services within the BIG-IP require updates to the geolocation database so that the inquired IP addresses are correctly characterized for service delivery and security enforcement. Traditionally managed device, where the devices are individually logged into and manually configured can benefit from a bit of automation without having to describe to an entire CI/CD pipeline and change in operational behavior. Additionally, a fully fledged CI/CD pipeline that embraces a full declarative model would also need a strategy around managing and performing the updates. This could be done via BIG-IQ; however, many organizations prefer BIG-IQ to monitor rather than manage their devices and so a different strategy is required.

This article series hopes to demonstrate some techniques and code that can work in either a classically managed fleet of devices or fully automated environment. If you have embraced BIG-IQ fully, this might not be relevant but is hopefully worth a cursory review depending on how you leverage BIG-IQ.

Assumptions and prerequisites

There are a few technology assumptions that will be imposed onto the reader that should be mentioned:

The solution will be presented in Python, specifically 3.10.2 although some lower versions could be supported. The use of the ‘walrus operator” ( := ) was made in a few places which requires version 3.8 or greater. Support for earlier versions would require some porting.
Visual Studio Code was used to create and test all the code. A modest level of expertise would be valuable, but likely not required by the reader.
An understanding of BIG-IP is necessary and assumed.
A cursory knowledge of the F5 Automation Toolchain is necessary as some of the API calls to the BIG-IP will leverage their use, however this is NOT a declarative operation.
Github is used to store the source for this article and a basic understanding of retrieving code from a github repository would be valuable.

References to the above technologies are provided here:

Lastly, an effort was made to make this code high-quality and resilient. I ran the code base through pylint until it was clean and handle most if not all exceptional cases. However, no formal QA function or load testing was performed other than my own. The code is presented as-is with no guarantees expressed or implied. That being said, it is hoped that this is a robust and usable example either as a script or slightly modified into a library and imported into the reader’s project.

Credits and Acknowledgements

Mark_Menger , for his continued review and support in all things automation based.

Mark Hermsdorfer, who reviewed some of my initial revisions and showed me the proper way to get http chunking to work. He also has an implementation on github that is referenced in the code base that you should look at.

Article Series

DevCentral places a limit on the size of an article and having learned from my previous submission I will try to organize this series a bit more cleanly. This is an overview of the items covered in each section:

Part 1 - Design and dependencies

Basic flow of a geolocation update
The imports list
The API library dictionary
The status_code_to_msg dictionary
Custom Exceptions
Method enumeration

Part 2 – Send_Request()

Function - send_request

Part 3 - Functions and Implementation

Function – get_auth_token
Function – backup_geo_db
Function – get_geoip_version

Part 4 (This article) - Functions and Implementation Continued

Function – fix_md5_file

Part 5 - Functions and Implementation Continued

Function – upload_geolocation_update

Part 6 - Functions and Implementation Conclusion

Function – install_geolocation_update

Part 7 - Pulling it together

Function – compare_versions
Function – validate_file
Function – print_usage
Command Line script

Functions and Implementation Continued

This will be a short submission as the limitations of article submission size make it a little tricky on how to organize a large topic. I didn't want to split the discussion of any of the functions between articles and the upload and install routines are extensive. The routine covered in this article, frustratingly, didn't fit into the size limitations of the previous submission so I was left to a short article as my only course of action. Apologies in advance.

fix_md5_file()

The next function is fix_md5_file() and there needs to be an explanation as to why it exists before we get into it. When you run an md5sum, you can do so in which the utility will perform the calculation on the file and then look at the filename of the same name but ending in .md5 to compare the hash inside that md5 file and then return ‘OK’ if it matches. An md5 file has a simple format in which you have a checksum, two spaces, and then the filename. Through testing, it was discovered that if for some reason when doing this call, likely because of the way the current working directory is resolved, the utility could not resolve the hash unless the filename in the md5 file included the path. To resolve this, it was easier to just backup the local copy and then modify the md5 file so that the target working directory the operation will be performed is in the md5 file for the respective entry.

def fix_md5_file(filename, append_path, savebu=False):
    """
    Fixes the md5 file so that it will check the zip at the correct path
    If you run md5sum in a different directory, then the md5 file needs to
    specify the directory or it will be unable to find it.  This adds the
    full path in front.
    Note:  md5sum is insanely specific:
        <md5sum_checksum><space><space><file_name>

    Parameters
    ----------
    filename : str
        Valid filename with resolvable path from cwd
    append_path : str
        path to append in the file
    savebu : boolean, default False
        If you a bu of the file should be made prior to modification

    Returns
    -------
        Fixed str saved within the md5 file
    """

The function takes the filename of the md5 file, the path we want to append to it and a flag, savebu, that defaults to False. It will return the fixed string that is saved within the md5.

    with open(filename, 'r+', encoding='UTF-8') as fileobj:
        if savebu:
            bufilename = (f"{filename}-{datetime.now().strftime('%Y%m%d-%H%M%S')}.backup" )
            shutil.copy(filename, bufilename)

        old = fileobj.read().split()
        fileobj.seek(0)
        fpfn = f"{old[0]}  {append_path}/{old[1]}"
        fileobj.write(fpfn)
        fileobj.truncate()
        return fpfn

The function starts by opening the passed filename for read+ access and putting the file object into the fileobj. If this succeeds, the ‘with’ block continues. We first check to see if the savebu flag is True (set) and if it is it creates a backup filename with some timestamps and then does a copy of the existing file. Otherwise, we read the first line of the file and put it into the variable old. We then reset the file pointer to the beginning of the file. We then craft an appropriate rewrite of the entry and write it back into the file. Lastly, we return the string.

Of note, its assumed that the user will be using the md5 file that is included in the downloads section for the geolocation database images. These md5 files have a single entry for the respective file. If this is NOT consistent, this routine does not validate or parse the file to find a specific, correct, entry. So, if there are issues in comparing the md5 hashes, check this first.

Wrap up

This concludes part 4 of the series. Part 5 of the series will cover the upload_geolocation_update routine which is extensive and will be a more sizable submission. You can access the entire series here: