Download a BIG-IP UCS archive with "curl".

[JG]

Problem this snippet solves:

Download a BIG-IP UCS archive using the program "curl" and verifies the output file's signature.

Tested on 13.1.1.

How to use this snippet:

Edit the code to input the hostname of your F5 UI, admin credentials, source UCS file name (defaults to config.ucs), and the output file name.

Code :

#!/bin/bash
#
# Download a UCS archive (across a stable network) with curl.
#
#-------------------------------------------------------------------------

F5_HOST='myhost.example.com'
CREDENTIALS='admin:admin'
FINAL_FILE='/tmp/config.ucs'
ARCHIVE_NAME_ON_SERVER='config.ucs'
DEBUG=''

#-------------------------------------------------------------------------
#
# Get the md5 checksum for the archive.
#
#-------------------------------------------------------------------------

ARCHIVE_CHECKSUM=$(curl -sku $CREDENTIALS -X POST -H "Content-type: application/json" \
      -d "{\"command\":\"run\", \"utilCmdArgs\": \"-c '/usr/bin/md5sum /var/local/ucs/$ARCHIVE_NAME_ON_SERVER'\"}" \
      https://$F5_HOST/mgmt/tm/util/bash | awk -F':' '{print $NF}' | awk -F'"' '{ print $2 }' | awk '{print $1}')

[ -z "$ARCHIVE_CHECKSUM" ] && echo "Failed to get archive signature. Aborting." && exit 1
[ ! -z "$DEBUG" ] && echo "Archive checksum: $ARCHIVE_CHECKSUM"

#-------------------------------------------------------------------------
#
# Find out the size of the archive and the size of the data packet.
#
#-------------------------------------------------------------------------

Content_Range=$(curl -I -kv -u $CREDENTIALS -H 'Content-Type: application/json' -X GET "https://$F5_HOST/mgmt/shared/file-transfer/ucs-downloads/$ARCHIVE_NAME_ON_SERVER" 2>/dev/null | grep "Content-Range: " | cut -d ' ' -f 2)

FIRST_CONTENT_RANGE=$(echo -n $Content_Range | cut -d '/' -f 1 | tr -d '\r')
[ ! -z "$DEBUG" ] && echo -n "FIRST_CONTENT_RANGE: "
[ ! -z "$DEBUG" ] && echo $FIRST_CONTENT_RANGE

NUMBER_OF_LAST_BYTE=$(echo -n $FIRST_CONTENT_RANGE | cut -d '-' -f 2)
[ ! -z "$DEBUG" ] && echo -n "NUMBER_OF_LAST_BYTE: "
[ ! -z "$DEBUG" ] && echo $NUMBER_OF_LAST_BYTE

INITIAL_CONTENT_LENGTH=$NUMBER_OF_LAST_BYTE
CONTENT_LENGTH=$(($NUMBER_OF_LAST_BYTE+1))
[ ! -z "$DEBUG" ] && echo -n "CONTENT_LENGTH: "
[ ! -z "$DEBUG" ] && echo $CONTENT_LENGTH

DFILE_SIZE=$(echo -n $Content_Range | cut -d '/' -f 2 | tr -d '\r' )
[ ! -z "$DEBUG" ] && echo -n "DFILE_SIZE: "
[ ! -z "$DEBUG" ] && echo $DFILE_SIZE

LAST_END_BYTE=$((DFILE_SIZE-1))

CUMULATIVE_NO=0
[ ! -z "$DEBUG" ] && echo "CUMULATIVE_NO: $CUMULATIVE_NO"
SEQ=0
LAST=0

#-------------------------------------------------------------------------
#
# Clean up: Remove the previous output file. 
#
#-------------------------------------------------------------------------

/bin/rm $FINAL_FILE 2>/dev/null

#-------------------------------------------------------------------------
#
# Get the archive file.
#
#-------------------------------------------------------------------------

while true
do
    if [ $LAST -gt 0 ]; then
        [ ! -z "$DEBUG" ] && echo 'End of run reached.'
        break
    fi

    if [ $SEQ -eq 0 ]; then
        NEXT_RANGE=$FIRST_CONTENT_RANGE
        CUMULATIVE_NO=$NUMBER_OF_LAST_BYTE
        CONTENT_LENGTH=$INITIAL_CONTENT_LENGTH
    else
        START_BYTE=$(($CUMULATIVE_NO+1))
        END_BYTE=$(($START_BYTE + $CONTENT_LENGTH))

        if [ $END_BYTE -gt $LAST_END_BYTE ]; then
            [ ! -z "$DEBUG" ] && echo "END_BYTE greater than LAST_END_BYTE: $END_BYTE:$LAST_END_BYTE"
            LAST=1
            let END_BYTE=$LAST_END_BYTE
            [ ! -z "$DEBUG" ] && echo "Getting the last data packet."
        fi

        NEXT_RANGE="${START_BYTE}-${END_BYTE}"
        CUMULATIVE_NO=$END_BYTE
    fi

    [ ! -z "$DEBUG" ] && echo "NEXT_RANGE: $NEXT_RANGE"

    let SEQ+=1
    [ ! -z "$DEBUG" ] && echo "SEQ: $SEQ"
    OUTPUT_FILE_NAME="/tmp/$$_downloaded_ucs_archive_file_part_$SEQ";

    curl -H "Content-Range: ${NEXT_RANGE}/${DFILE_SIZE}" -s -k -u $CREDENTIALS -H 'Content-Type: application/json' -X GET "https://$F5_HOST/mgmt/shared/file-transfer/ucs-downloads/$ARCHIVE_NAME_ON_SERVER" -o $OUTPUT_FILE_NAME

    cat $OUTPUT_FILE_NAME >> $FINAL_FILE
    /bin/rm $OUTPUT_FILE_NAME
    [ ! -z "$DEBUG" ] && echo "End of loop $SEQ"
done

#-------------------------------------------------------------------------
#
# Verify downloaded file.
#
#-------------------------------------------------------------------------

FINAL_FILE_CHECKSUM=$(/usr/bin/md5sum $FINAL_FILE | awk '{print $1}')

if [ "$FINAL_FILE_CHECKSUM" == "$ARCHIVE_CHECKSUM" ]; then
    echo "Download completed and verified."
else
    echo "Downloaded file has incorrect checksum."
    exit 1
fi

# END --------------------------------------------------------------------

Tested this on version:

13.0

[Noel_McKenna]

​Great script.

I noticed that the UCS file is generated correctly in

/var/local/ucs/$ARCHIVE_NAME_ON_SERVER'

however the transfer appears to be from

/mgmt/shared/file-transfer/ucs-downloads/$ARCHIVE_NAME_ON_SERVER"

How is it moved?

Also is

END_BYTE=$(($START_BYTE + $CONTENT_LENGTH))

supposed to be

END_BYTE=$((START_BYTE + CONTENT_LENGTH))

[JG]

The origin location is dictated by the API.

The variables should be OK.

[Noel_McKenna]

Thanks.

​Does the UCS file need to be transfered from /var/local/ucs to /mgmt/shared/file-transfer/ucs-downloads on the F5?

That's an issue, as the directory "/mgmt/shared/file-transfer/ucs-downloads" doesn't appear to exist.

[JG]

I think you only need to specify the file name in the standard location /var/local/ucs/.

[Yulong]

hi Master, the way is good to me. great thanks to you.

but is there a url that we can upload ucs file ?