Decrypting BIG-IP Packet Captures Automatically

Back in April, I released the first of hopefully many tools (Automating Packet Captures on BIG-IP) that will assist those responsible for responding to all those directed "It's the BIG-IP!" and "It's the network!" accusations. In this article, I expand on that work by adding automatic decryption to the toolbelt.

Changes and Enhancements

The previous tool was focused on gathering support files for a case. It requested a virtual server name, a source IP for capturing test traffic, and a case number, which was used in naming the packet capture, sessions keys, and qkview files. For this tool, I stripped out the qkview and case-related steps to focus on just taking a capture against a virtual server. I retained the source IP for test traffic, though I might create another version that captures all sessions as well.

For enhancements, I have two minor changes and one major change. First was to ask for a capture duration so you can specify n number of seconds to run the capture, and second, asking for capture filters that will be applied in addition to the virtual server host IP.

def run_tcpdump(bigip, duration, virtual_name, filters):
    datestring = datetime.now().strftime('%Y%m%d-%H%M%S')
    vip = bigip.load(f'/mgmt/tm/ltm/virtual/{virtual_name}')
    virtual_ip = vip.properties.get('destination').split('/')[-1].split(':')[0]

    dump_string = TCPDUMP_BASH_STRING.replace('CAP_SECS', duration)
    if filters == '':
        dump_string = dump_string.replace('VIRTUAL_IP', f'host {virtual_ip}')
    else:
        dump_string = dump_string.replace('VIRTUAL_IP', f'host {virtual_ip} {filters} ')
        print(dump_string)
    dump_string = dump_string.replace('DATESTRING', datestring)

    try:
        print(f'\tStarting tcpdump...please reproduce your issue now.')
        data = {'command': 'run', 'utilCmdArgs': f'-c "{dump_string}"'}
        bigip.command('/mgmt/tm/util/bash', data)
    except RESTAPIError:
        pass
    sleep(5)
    print(f'\ttcpdump complete...continuing.')
    return f'autocap_{datestring}.pcap'

Thirdly, the major change was to use the key files and encrypted packet capture to generate a decrypted version of the packet capture. This requires that wireshark is installed on your system and that that installation also installed the editcap utility. 

def decrypt_capture(tcpdump_file, sessionkey_file):
    print(f'\tDecrypting capture {tcpdump_file} with session keys in {sessionkey_file}.')
    dir = 'output_files'
    cmd = f'editcap --inject-secrets tls,{dir}/{sessionkey_file} {dir}/{tcpdump_file} {dir}/decrypted_{tcpdump_file}'
    decrypt_file = subprocess.run(['editcap',
                                   '--inject-secrets',
                                   f'tls,{dir}/{sessionkey_file}',
                                   f'{dir}/{tcpdump_file}',
                                   f'{dir}/decrypted_{tcpdump_file}'],
                                  stdout=subprocess.DEVNULL,
                                  stderr=subprocess.STDOUT)
    print(f'\tDecrypted file: {dir}/decrypted_{tcpdump_file}...continuing.')

Script output:

    #################################################
    ### BIG-IP tcpdump capture collection utility ###
    #################################################

    -------------------------------------------------

    Virtual ext_nerdknobs.tech_443 has associated client-ssl profile cssl_nerdknobs.tech...continuing.
    Session keylogger iRule (cache disabled version) created...continuing.
    Session keylogger iRule applied to ext_nerdknobs.tech_443...continuing.
    Starting tcpdump...please reproduce your issue now.
    Session keylogger iRule removed from ext_nerdknobs.tech_443...continuing.
    keylogger iRule deleted...continuing.
    Secrets key file created (with cache disabled command)...continuing.
    Downloading capture and key files from BIG-IP.
        autocap_20221230-134941.pcap downloaded.
        autocap_sessionsecrets.pms downloaded.
    All files downloaded...continuing.
    Cleaning up capture and key files on BIG-IP.
        autocap_20221230-134941.pcap deleted.
        autocap_sessionsecrets.pms deleted.
    All files cleaned up on BIG-IP...complete.
    Decrypting capture autocap_20221230-134941.pcap with session keys in autocap_sessionsecrets.pms.
    Decrypted file: output_files/decrypted_autocap_20221230-134941.pcap...continuing.

And when I open up decrypted_autocap_20221230-134941.pcap and select an HTTP packet (which by itself is evidence I have a decrypted packet!) and select HTTP stream, I can see the request and response:

Decrypted HTTP Stream

 This script is capndecrypt.py and is available on GitHub in my pcap_utils repository.

Notes

1. The work here does not include TLS1.3 decryption.
2. There is an alternative approach to the iRules that uses a database key and a tcpdump flag to include the keys in the capture itself instead of having to glean them from log statements.
3. We'll take a look at numbers 1 and 2 in this list in an upcoming live coding session. If you aren't a registered member here on DevCentral, make that happen and watch for the details in the DevCentral Connects group.