Decrypting BIG-IP Packet Captures Automatically
Back in April, I released the first of hopefully many tools (Automating Packet Captures on BIG-IP) that will assist those responsible for responding to all those directed "It's the BIG-IP!" and "It's the network!" accusations. In this article, I expand on that work by adding automatic decryption to the toolbelt.
Changes and Enhancements
The previous tool was focused on gathering support files for a case. It requested a virtual server name, a source IP for capturing test traffic, and a case number, which was used in naming the packet capture, sessions keys, and qkview files. For this tool, I stripped out the qkview and case-related steps to focus on just taking a capture against a virtual server. I retained the source IP for test traffic, though I might create another version that captures all sessions as well.
For enhancements, I have two minor changes and one major change. First was to ask for a capture duration so you can specify n number of seconds to run the capture, and second, asking for capture filters that will be applied in addition to the virtual server host IP.
def run_tcpdump(bigip, duration, virtual_name, filters):
datestring = datetime.now().strftime('%Y%m%d-%H%M%S')
vip = bigip.load(f'/mgmt/tm/ltm/virtual/{virtual_name}')
virtual_ip = vip.properties.get('destination').split('/')[-1].split(':')[0]
dump_string = TCPDUMP_BASH_STRING.replace('CAP_SECS', duration)
if filters == '':
dump_string = dump_string.replace('VIRTUAL_IP', f'host {virtual_ip}')
else:
dump_string = dump_string.replace('VIRTUAL_IP', f'host {virtual_ip} {filters} ')
print(dump_string)
dump_string = dump_string.replace('DATESTRING', datestring)
try:
print(f'\tStarting tcpdump...please reproduce your issue now.')
data = {'command': 'run', 'utilCmdArgs': f'-c "{dump_string}"'}
bigip.command('/mgmt/tm/util/bash', data)
except RESTAPIError:
pass
sleep(5)
print(f'\ttcpdump complete...continuing.')
return f'autocap_{datestring}.pcap'
Thirdly, the major change was to use the key files and encrypted packet capture to generate a decrypted version of the packet capture. This requires that wireshark is installed on your system and that that installation also installed the editcap utility.
def decrypt_capture(tcpdump_file, sessionkey_file):
print(f'\tDecrypting capture {tcpdump_file} with session keys in {sessionkey_file}.')
dir = 'output_files'
cmd = f'editcap --inject-secrets tls,{dir}/{sessionkey_file} {dir}/{tcpdump_file} {dir}/decrypted_{tcpdump_file}'
decrypt_file = subprocess.run(['editcap',
'--inject-secrets',
f'tls,{dir}/{sessionkey_file}',
f'{dir}/{tcpdump_file}',
f'{dir}/decrypted_{tcpdump_file}'],
stdout=subprocess.DEVNULL,
stderr=subprocess.STDOUT)
print(f'\tDecrypted file: {dir}/decrypted_{tcpdump_file}...continuing.')
Script output:
#################################################
### BIG-IP tcpdump capture collection utility ###
#################################################
-------------------------------------------------
Virtual ext_nerdknobs.tech_443 has associated client-ssl profile cssl_nerdknobs.tech...continuing.
Session keylogger iRule (cache disabled version) created...continuing.
Session keylogger iRule applied to ext_nerdknobs.tech_443...continuing.
Starting tcpdump...please reproduce your issue now.
Session keylogger iRule removed from ext_nerdknobs.tech_443...continuing.
keylogger iRule deleted...continuing.
Secrets key file created (with cache disabled command)...continuing.
Downloading capture and key files from BIG-IP.
autocap_20221230-134941.pcap downloaded.
autocap_sessionsecrets.pms downloaded.
All files downloaded...continuing.
Cleaning up capture and key files on BIG-IP.
autocap_20221230-134941.pcap deleted.
autocap_sessionsecrets.pms deleted.
All files cleaned up on BIG-IP...complete.
Decrypting capture autocap_20221230-134941.pcap with session keys in autocap_sessionsecrets.pms.
Decrypted file: output_files/decrypted_autocap_20221230-134941.pcap...continuing.
And when I open up decrypted_autocap_20221230-134941.pcap and select an HTTP packet (which by itself is evidence I have a decrypted packet!) and select HTTP stream, I can see the request and response:
This script is capndecrypt.py and is available on GitHub in my pcap_utils repository.
Notes
- The work here does not include TLS1.3 decryption.
- There is an alternative approach to the iRules that uses a database key and a tcpdump flag to include the keys in the capture itself instead of having to glean them from log statements.
- We'll take a look at numbers 1 and 2 in this list in an upcoming live coding session. If you aren't a registered member here on DevCentral, make that happen and watch for the details in the DevCentral Connects group.