What do you do when you need to decrypt traffic, apply a security service chain policy to it, and then send it to an array of open source security tools and never send an unencrypted packet on the AWS “wire”? And what if these security tools did not support TLS? What if you had to do this transparently?
If we break it down, we need to:
- Decrypt the incoming traffic so we can apply the security service chain.
- Encrypt the traffic and send it to the security system.
- Decrypt the traffic from the security system
- Encrypt the traffic to the tap
- Encrypt the traffic to the backend
The Security Stack
In our deployment, we will use two open source security tools. These were picked by the client that I was working with and I will not cover how to configure them in detail.
Suricata
If you are not familiar with Suricata, it is an open-source IDS/IPS security tool that supports packet captures, alerting among many other functions. In our deployment, we will configure Suricata to be an inline IPS with netfilter.
Arkime
Arkime “is a large-scale, open-source, indexed packet capture and search tool” allowing for graphing, visibility and search of network traffic.
SSL Orchestrator
If you are reading this, you probably know what SSL Orchestrator is. If not, please refer to F5 CloudDocs or search DevCentral to learn about all of the amazing things that SSLO can do. This is where security service logic and traffic steering happens.
The Topology
We are going to use the same Gateway Load Balancer (GWLB) topology that was used in my article series Increase Security in AWS without Rearchitecting your Applications; with that in mind you can use all the items below and use SSLO in a reverse proxy topology.
The change is we are going to use GRE tunnels between Suricata and BIG-IP and a GRETAP tunnel between BIGIP and Arkime. We will then wrap the GRE tunnels in IPSEC to ensure that traffic is encrypted. The backend web servers are using SSL with BIG-IP and the clients are using SSL, which is terminated on SSLO.
Building Overlays in the Overlay of AWS VPC Networking
To bring all of this together, we are going to build tunnels in the VPC. The type of tunnel we are using depends on what we need it to do. For a network tap, we need to copy the same IP packet into an L2 frame that has a different destination mac address. Since we also need to encrypt it, we need to pick a tunnel. I used a GRETAP interface, but VXLAN should work too. To deploy the Suricata instance, we need to move untranslated IP packets AND we need to encrypt them. To accomplish this, I used standard GRE interfaces.
Arkime Network Interfaces and GRE
Here we create the GRETAP interface and assign it the created IP address from SSLO. We will need the mac address for the SSLO config.
ens5: <span class="token assign-left variable">flags</span><span class="token operator">=</span><span class="token number">416</span><span class="token operator"><span class="token file-descriptor important">3</span><</span>UP,BROADCAST,RUNNING,MULTICAST<span class="token operator">></span> mtu <span class="token number">9001</span>
inet <span class="token number">172.31</span>.100.228 netmask <span class="token number">255.255</span>.255.0 broadcast <span class="token number">172.31</span>.100.255
inet6 fe80::dd:5cff:fe7d:fe9b prefixlen <span class="token number">64</span> scopeid 0x2<span class="token operator"><span class="token file-descriptor important">0</span><</span>link<span class="token operator">></span>
ether 02:dd:5c:7d:fe:9b txqueuelen <span class="token number">1000</span> <span class="token punctuation">(</span>Ethernet<span class="token punctuation">)</span>
ens6: <span class="token assign-left variable">flags</span><span class="token operator">=</span><span class="token number">441</span><span class="token operator"><span class="token file-descriptor important">9</span><</span>UP,BROADCAST,RUNNING,PROMISC,MULTICAST<span class="token operator">></span> mtu <span class="token number">9001</span>
inet <span class="token number">172.31</span>.15.198 netmask <span class="token number">255.255</span>.255.0 broadcast <span class="token number">172.31</span>.15.255
inet6 fe80::99:d5ff:fe61:18b3 prefixlen <span class="token number">64</span> scopeid 0x2<span class="token operator"><span class="token file-descriptor important">0</span><</span>link<span class="token operator">></span>
ether 02:99:d5:61:18:b3 txqueuelen <span class="token number">1000</span> <span class="token punctuation">(</span>Ethernet<span class="token punctuation">)</span>
tun0: <span class="token assign-left variable">flags</span><span class="token operator">=</span><span class="token number">416</span><span class="token operator"><span class="token file-descriptor important">3</span><</span>UP,BROADCAST,RUNNING,MULTICAST<span class="token operator">></span> mtu <span class="token number">8963</span>
inet <span class="token number">198.19</span>.0.10 netmask <span class="token number">255.255</span>.255.0 broadcast <span class="token number">198.19</span>.0.255
inet6 fe80::70a5:76ff:fed2:a3ec prefixlen <span class="token number">64</span> scopeid 0x2<span class="token operator"><span class="token file-descriptor important">0</span><</span>link<span class="token operator">></span>
ether <span class="token number">72</span>:a5:76:d2:a3:ec txqueuelen <span class="token number">1000</span> <span class="token punctuation">(</span>Ethernet<span class="token punctuation">)</span>
<span class="token function">sudo</span> <span class="token function">ip</span> <span class="token function">link</span> <span class="token function">add</span> tun0 <span class="token builtin class-name">type</span> gretap <span class="token builtin class-name">local</span> <span class="token number">172.31</span>.15.198 remote <span class="token number">172.31</span>.15.10 ttl <span class="token number">255</span>
<span class="token function">sudo</span> <span class="token function">ifconfig</span> tun0 <span class="token number">198.19</span>.0.10
<span class="token function">sudo</span> <span class="token function">ifconfig</span> tun0 up
Arkime IPSEC
Now that we have the GRETAP interface, we need to configure one IPSEC process to encrypt the traffic.
ubuntu@ip-172-31-100-228:~$ <span class="token function">sudo</span> <span class="token function">cat</span> /etc/ipsec.conf
config setup
<span class="token assign-left variable">charondebug</span><span class="token operator">=</span><span class="token string">"ike 1, knl 1, cfg 0, net 1"</span>
<span class="token assign-left variable">strictcrlpolicy</span><span class="token operator">=</span>no
<span class="token assign-left variable">uniqueids</span><span class="token operator">=</span>yes
<span class="token assign-left variable">cachecrls</span><span class="token operator">=</span>no
conn ipsec-ikev2-vpn
<span class="token assign-left variable">authby</span><span class="token operator">=</span>secret
<span class="token assign-left variable">left</span><span class="token operator">=</span>%defaultroute
<span class="token assign-left variable">leftid</span><span class="token operator">=</span><span class="token number">172.31</span>.15.198
<span class="token assign-left variable">leftsubnet</span><span class="token operator">=</span><span class="token number">0.0</span>.0.0/0
<span class="token assign-left variable">right</span><span class="token operator">=</span><span class="token number">172.31</span>.15.10
<span class="token assign-left variable">rightsubnet</span><span class="token operator">=</span><span class="token number">0.0</span>.0.0/0
<span class="token assign-left variable">ike</span><span class="token operator">=</span>aes256-sha2_256-modp1024<span class="token operator">!</span>
<span class="token assign-left variable">esp</span><span class="token operator">=</span>aes256-sha2_256<span class="token operator">!</span>
<span class="token assign-left variable">keyingtries</span><span class="token operator">=</span><span class="token number">0</span>
<span class="token assign-left variable">ikelifetime</span><span class="token operator">=</span>1h
<span class="token assign-left variable">lifetime</span><span class="token operator">=</span>8h
<span class="token assign-left variable">dpddelay</span><span class="token operator">=</span><span class="token number">30</span>
<span class="token assign-left variable">dpdtimeout</span><span class="token operator">=</span><span class="token number">120</span>
<span class="token assign-left variable">dpdaction</span><span class="token operator">=</span>restart
<span class="token assign-left variable">auto</span><span class="token operator">=</span>start
ubuntu@ip-172-31-100-228:~$
Suricata GRE
For Suricata, we are routing IP packets and a standard GRE tunnel will work.
ens5: <span class="token assign-left variable">flags</span><span class="token operator">=</span><span class="token number">416</span><span class="token operator"><span class="token file-descriptor important">3</span><</span>UP,BROADCAST,RUNNING,MULTICAST<span class="token operator">></span> mtu <span class="token number">9001</span>
inet <span class="token number">172.31</span>.11.20 netmask <span class="token number">255.255</span>.255.0 broadcast <span class="token number">172.31</span>.11.255
inet6 fe80::80:36ff:fe34:3325 prefixlen <span class="token number">64</span> scopeid 0x2<span class="token operator"><span class="token file-descriptor important">0</span><</span>link<span class="token operator">></span>
ether 02:80:36:34:33:25 txqueuelen <span class="token number">1000</span> <span class="token punctuation">(</span>Ethernet<span class="token punctuation">)</span>
ens6: <span class="token assign-left variable">flags</span><span class="token operator">=</span><span class="token number">416</span><span class="token operator"><span class="token file-descriptor important">3</span><</span>UP,BROADCAST,RUNNING,MULTICAST<span class="token operator">></span> mtu <span class="token number">9001</span>
inet <span class="token number">172.31</span>.12.20 netmask <span class="token number">255.255</span>.255.0 broadcast <span class="token number">172.31</span>.12.255
inet6 fe80::d6:aff:fee3:c25f prefixlen <span class="token number">64</span> scopeid 0x2<span class="token operator"><span class="token file-descriptor important">0</span><</span>link<span class="token operator">></span>
ether 02:d6:0a:e3:c2:5f txqueuelen <span class="token number">1000</span> <span class="token punctuation">(</span>Ethernet<span class="token punctuation">)</span>
gre1: <span class="token assign-left variable">flags</span><span class="token operator">=</span><span class="token number">20</span><span class="token operator"><span class="token file-descriptor important">9</span><</span>UP,POINTOPOINT,RUNNING,NOARP<span class="token operator">></span> mtu <span class="token number">8977</span>
inet <span class="token number">100.64</span>.0.20 netmask <span class="token number">255.255</span>.255.0 destination <span class="token number">100.64</span>.0.20
inet6 fe80::ac1f:b14 prefixlen <span class="token number">64</span> scopeid 0x2<span class="token operator"><span class="token file-descriptor important">0</span><</span>link<span class="token operator">></span>
gre2: <span class="token assign-left variable">flags</span><span class="token operator">=</span><span class="token number">20</span><span class="token operator"><span class="token file-descriptor important">9</span><</span>UP,POINTOPOINT,RUNNING,NOARP<span class="token operator">></span> mtu <span class="token number">8977</span>
inet <span class="token number">100.64</span>.1.20 netmask <span class="token number">255.255</span>.255.0 destination <span class="token number">100.64</span>.1.20
inet6 fe80::ac1f:c14 prefixlen <span class="token number">64</span> scopeid 0x2<span class="token operator"><span class="token file-descriptor important">0</span><</span>link<span class="token operator">></span>
<span class="token comment">#!/bin/bash</span>
<span class="token function">sudo</span> <span class="token function">ip</span> tunnel <span class="token function">add</span> gre1 mode gre <span class="token builtin class-name">local</span> <span class="token number">172.31</span>.11.20 remote <span class="token number">172.31</span>.11.10
<span class="token function">sudo</span> <span class="token function">ip</span> tunnel <span class="token function">add</span> gre2 mode gre <span class="token builtin class-name">local</span> <span class="token number">172.31</span>.12.20 remote <span class="token number">172.31</span>.12.10
<span class="token function">sudo</span> <span class="token function">ifconfig</span> gre2 <span class="token number">100.64</span>.1.20 netmask <span class="token number">255.255</span>.255.0 up
<span class="token function">sudo</span> <span class="token function">ifconfig</span> gre1 <span class="token number">100.64</span>.0.20 netmask <span class="token number">255.255</span>.255.0 up
<span class="token function">sudo</span> <span class="token function">ip</span> route <span class="token function">add</span> <span class="token number">10.0</span>.0.0/8 via <span class="token number">100.64</span>.1.20
<span class="token function">sudo</span> <span class="token function">ip</span> route <span class="token function">add</span> <span class="token number">0.0</span>.0.0/0 via <span class="token number">100.64</span>.0.10
<span class="token function">sudo</span> iptables -I FORWARD -i gre1 -o gre2 -j NFQUEUE
<span class="token function">sudo</span> iptables -I FORWARD -i gre2 -o gre1 -j NFQUEUE
<span class="token function">sudo</span> suricata -c /etc/suricata/suricata.yaml -q <span class="token number">0</span> <span class="token operator">&</span>
SURICATA IPSEC
For the Suricata instance, we need two IPSEC peers to create our network topology.
ubuntu@ip-172-31-11-20:~$ <span class="token function">sudo</span> <span class="token function">cat</span> /etc/ipsec.conf
config setup
<span class="token assign-left variable">charondebug</span><span class="token operator">=</span><span class="token string">"ike 1, knl 1, cfg 0, net 1"</span>
<span class="token assign-left variable">strictcrlpolicy</span><span class="token operator">=</span>no
<span class="token assign-left variable">uniqueids</span><span class="token operator">=</span>yes
<span class="token assign-left variable">cachecrls</span><span class="token operator">=</span>no
conn ipsec-ikev2-vpn
<span class="token assign-left variable">authby</span><span class="token operator">=</span>secret
<span class="token assign-left variable">left</span><span class="token operator">=</span>%defaultroute
<span class="token assign-left variable">leftid</span><span class="token operator">=</span><span class="token number">172.31</span>.11.20
<span class="token assign-left variable">leftsubnet</span><span class="token operator">=</span><span class="token number">0.0</span>.0.0/0
<span class="token assign-left variable">right</span><span class="token operator">=</span><span class="token number">172.31</span>.11.10
<span class="token assign-left variable">rightsubnet</span><span class="token operator">=</span><span class="token number">0.0</span>.0.0/0
<span class="token assign-left variable">ike</span><span class="token operator">=</span>aes256-sha2_256-modp1024<span class="token operator">!</span>
<span class="token assign-left variable">esp</span><span class="token operator">=</span>aes256-sha2_256<span class="token operator">!</span>
<span class="token assign-left variable">keyingtries</span><span class="token operator">=</span><span class="token number">0</span>
<span class="token assign-left variable">ikelifetime</span><span class="token operator">=</span>1h
<span class="token assign-left variable">lifetime</span><span class="token operator">=</span>8h
<span class="token assign-left variable">dpddelay</span><span class="token operator">=</span><span class="token number">30</span>
<span class="token assign-left variable">dpdtimeout</span><span class="token operator">=</span><span class="token number">120</span>
<span class="token assign-left variable">dpdaction</span><span class="token operator">=</span>restart
<span class="token assign-left variable">auto</span><span class="token operator">=</span>start
conn ipsec-ikev2-vpn-2
<span class="token assign-left variable">authby</span><span class="token operator">=</span>secret
<span class="token assign-left variable">left</span><span class="token operator">=</span>%defaultroute
<span class="token assign-left variable">leftid</span><span class="token operator">=</span><span class="token number">172.31</span>.12.20
<span class="token assign-left variable">leftsubnet</span><span class="token operator">=</span><span class="token number">0.0</span>.0.0/0
<span class="token assign-left variable">right</span><span class="token operator">=</span><span class="token number">172.31</span>.12.10
<span class="token assign-left variable">rightsubnet</span><span class="token operator">=</span><span class="token number">0.0</span>.0.0/0
<span class="token assign-left variable">ike</span><span class="token operator">=</span>aes256-sha2_256-modp1024<span class="token operator">!</span>
<span class="token assign-left variable">esp</span><span class="token operator">=</span>aes256-sha2_256<span class="token operator">!</span>
<span class="token assign-left variable">keyingtries</span><span class="token operator">=</span><span class="token number">0</span>
<span class="token assign-left variable">ikelifetime</span><span class="token operator">=</span>1h
<span class="token assign-left variable">lifetime</span><span class="token operator">=</span>8h
<span class="token assign-left variable">dpddelay</span><span class="token operator">=</span><span class="token number">30</span>
<span class="token assign-left variable">dpdtimeout</span><span class="token operator">=</span><span class="token number">120</span>
<span class="token assign-left variable">dpdaction</span><span class="token operator">=</span>restart
<span class="token assign-left variable">auto</span><span class="token operator">=</span>start
ubuntu@ip-172-31-11-20:~$ <span class="token function">sudo</span> ipsec status
Security Associations <span class="token punctuation">(</span><span class="token number">4</span> up, <span class="token number">0</span> connecting<span class="token punctuation">)</span>:
ipsec-ikev2-vpn-2<span class="token punctuation">[</span><span class="token number">2015</span><span class="token punctuation">]</span>: ESTABLISHED <span class="token number">6</span> seconds ago, <span class="token number">172.31</span>.12.20<span class="token punctuation">[</span><span class="token number">172.31</span>.12.20<span class="token punctuation">]</span><span class="token punctuation">..</span>.172.31.12.10<span class="token punctuation">[</span><span class="token number">172.31</span>.12.10<span class="token punctuation">]</span>
ipsec-ikev2-vpn-2<span class="token punctuation">{</span><span class="token number">2015</span><span class="token punctuation">}</span>: INSTALLED, TUNNEL, reqid <span class="token number">2</span>, ESP SPIs: c0f72a1f_i 0e5d3a98_o
ipsec-ikev2-vpn-2<span class="token punctuation">{</span><span class="token number">2015</span><span class="token punctuation">}</span>: <span class="token number">172.31</span>.12.20/32 <span class="token operator">==</span><span class="token operator">=</span> <span class="token number">172.31</span>.12.10/32
ipsec-ikev2-vpn<span class="token punctuation">[</span><span class="token number">2014</span><span class="token punctuation">]</span>: ESTABLISHED <span class="token number">108</span> seconds ago, <span class="token number">172.31</span>.11.20<span class="token punctuation">[</span><span class="token number">172.31</span>.11.20<span class="token punctuation">]</span><span class="token punctuation">..</span>.172.31.11.10<span class="token punctuation">[</span><span class="token number">172.31</span>.11.10<span class="token punctuation">]</span>
ipsec-ikev2-vpn<span class="token punctuation">{</span><span class="token number">2014</span><span class="token punctuation">}</span>: INSTALLED, TUNNEL, reqid <span class="token number">1</span>, ESP SPIs: cd5853ac_i 0199a2f6_o
ipsec-ikev2-vpn<span class="token punctuation">{</span><span class="token number">2014</span><span class="token punctuation">}</span>: <span class="token number">172.31</span>.11.20/32 <span class="token operator">==</span><span class="token operator">=</span> <span class="token number">172.31</span>.11.10/32
ipsec-ikev2-vpn<span class="token punctuation">[</span><span class="token number">2013</span><span class="token punctuation">]</span>: ESTABLISHED <span class="token number">7</span> minutes ago, <span class="token number">172.31</span>.11.20<span class="token punctuation">[</span><span class="token number">172.31</span>.11.20<span class="token punctuation">]</span><span class="token punctuation">..</span>.172.31.11.10<span class="token punctuation">[</span><span class="token number">172.31</span>.11.10<span class="token punctuation">]</span>
ipsec-ikev2-vpn<span class="token punctuation">{</span><span class="token number">2013</span><span class="token punctuation">}</span>: INSTALLED, TUNNEL, reqid <span class="token number">1</span>, ESP SPIs: c78843ac_i 0df07e34_o
ipsec-ikev2-vpn<span class="token punctuation">{</span><span class="token number">2013</span><span class="token punctuation">}</span>: <span class="token number">172.31</span>.11.20/32 <span class="token operator">==</span><span class="token operator">=</span> <span class="token number">172.31</span>.11.10/32
ipsec-ikev2-vpn-2<span class="token punctuation">[</span><span class="token number">2012</span><span class="token punctuation">]</span>: ESTABLISHED <span class="token number">32</span> minutes ago, <span class="token number">172.31</span>.12.20<span class="token punctuation">[</span><span class="token number">172.31</span>.12.20<span class="token punctuation">]</span><span class="token punctuation">..</span>.172.31.12.10<span class="token punctuation">[</span><span class="token number">172.31</span>.12.10<span class="token punctuation">]</span>
ipsec-ikev2-vpn-2<span class="token punctuation">{</span><span class="token number">2012</span><span class="token punctuation">}</span>: INSTALLED, TUNNEL, reqid <span class="token number">2</span>, ESP SPIs: c0addbb2_i 0d653b4d_o
ipsec-ikev2-vpn-2<span class="token punctuation">{</span><span class="token number">2012</span><span class="token punctuation">}</span>: <span class="token number">172.31</span>.12.20/32 <span class="token operator">==</span><span class="token operator">=</span> <span class="token number">172.31</span>.12.10/32
ubuntu@ip-172-31-11-20:~$
BIG-IP GRE Tunnels
We need to configure two tunnels for Suricata and one GRETAP for Arkime.
admin@<span class="token punctuation">(</span>ip-172-31-100-10<span class="token punctuation">)</span><span class="token punctuation">(</span>cfg-sync Standalone<span class="token punctuation">)</span><span class="token punctuation">(</span>Active<span class="token punctuation">)</span><span class="token punctuation">(</span>/Common<span class="token punctuation">)</span><span class="token punctuation">(</span>tmos.net.tunnels<span class="token punctuation">)</span><span class="token comment"># list</span>
net tunnels geneve aws-geneve <span class="token punctuation">{</span>
app-service none
defaults-from geneve
description none
<span class="token punctuation">}</span>
net tunnels gre arkime-gre-profile <span class="token punctuation">{</span>
app-service none
defaults-from gre
description none
encapsulation transparent-ethernet-bridging
flooding-type none
rx-csum disabled
tx-csum disabled
<span class="token punctuation">}</span>
net tunnels tunnel arkime-tap <span class="token punctuation">{</span>
if-index <span class="token number">288</span>
local-address <span class="token number">172.31</span>.15.10
profile arkime-gre-profile
remote-address any
<span class="token punctuation">}</span>
net tunnels tunnel geneve-tunnel <span class="token punctuation">{</span>
if-index <span class="token number">304</span>
local-address <span class="token number">172.31</span>.10.10
profile aws-geneve
remote-address any
<span class="token punctuation">}</span>
net tunnels tunnel gre-tunnel-from-suricata <span class="token punctuation">{</span>
if-index <span class="token number">320</span>
local-address <span class="token number">172.31</span>.12.10
profile gre
remote-address <span class="token number">172.31</span>.12.20
<span class="token punctuation">}</span>
net tunnels tunnel gre-tunnel-to-suricata <span class="token punctuation">{</span>
if-index <span class="token number">336</span>
local-address <span class="token number">172.31</span>.11.10
profile gre
remote-address <span class="token number">172.31</span>.11.20
<span class="token punctuation">}</span>
BIG-IP IPSEC
We need to configure two IPSEC peers and two traffic selectors for Sericata and one peer and selector for Arkime.
net ipsec ike-peer arkime <span class="token punctuation">{</span>
my-id-value <span class="token number">172.31</span>.15.10
peers-id-value <span class="token number">172.31</span>.15.198
phase1-auth-method pre-shared-key
phase1-encrypt-algorithm aes256
phase1-hash-algorithm sha256
preshared-key-encrypted <span class="token variable">$M</span><span class="token variable">$36</span><span class="token variable">$sL8Pgqt7panBy0rKOskzfQ</span><span class="token operator">==</span>
prf sha256
remote-address <span class="token number">172.31</span>.15.198
traffic-selector <span class="token punctuation">{</span> /Common/arkime-ts <span class="token punctuation">}</span>
version <span class="token punctuation">{</span> v2 <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
net ipsec ike-peer from-suricata <span class="token punctuation">{</span>
my-id-value <span class="token number">172.31</span>.12.10
peers-id-value <span class="token number">172.31</span>.12.20
phase1-auth-method pre-shared-key
phase1-encrypt-algorithm aes256
phase1-hash-algorithm sha256
preshared-key-encrypted <span class="token variable">$M</span><span class="token variable">$NW</span><span class="token variable">$XupRAfQhku</span>/WuQx/sRTAaw<span class="token operator">==</span>
prf sha256
remote-address <span class="token number">172.31</span>.12.20
traffic-selector <span class="token punctuation">{</span> /Common/ts-from-suricata <span class="token punctuation">}</span>
version <span class="token punctuation">{</span> v2 <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
net ipsec ike-peer to-suricata <span class="token punctuation">{</span>
my-id-value <span class="token number">172.31</span>.11.10
peers-id-value <span class="token number">172.31</span>.11.20
phase1-auth-method pre-shared-key
phase1-encrypt-algorithm aes256
phase1-hash-algorithm sha256
preshared-key-encrypted <span class="token variable">$M</span><span class="token variable">$uk</span><span class="token variable">$gE48199ic7g0k3OwbrOK0w</span><span class="token operator">==</span>
prf sha256
remote-address <span class="token number">172.31</span>.11.20
traffic-selector <span class="token punctuation">{</span> /Common/ts-to-suricata <span class="token punctuation">}</span>
version <span class="token punctuation">{</span> v2 <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
net ipsec ipsec-policy ipsec-policy-from-suricata <span class="token punctuation">{</span>
ike-phase2-auth-algorithm sha256
ike-phase2-encrypt-algorithm aes256
mode tunnel
tunnel-local-address <span class="token number">172.31</span>.12.10
tunnel-remote-address <span class="token number">172.31</span>.12.20
<span class="token punctuation">}</span>
net ipsec traffic-selector arkime-ts <span class="token punctuation">{</span>
destination-address <span class="token number">172.31</span>.15.0/24
ip-protocol <span class="token number">47</span>
ipsec-policy ipsec-traffic-policy-arkime
order <span class="token number">1</span>
source-address <span class="token number">172.31</span>.15.0/24
<span class="token punctuation">}</span>
net ipsec traffic-selector ts-from-suricata <span class="token punctuation">{</span>
destination-address <span class="token number">172.31</span>.12.20/32
ipsec-policy ipsec-policy-from-suricata
order <span class="token number">3</span>
source-address <span class="token number">172.31</span>.12.10/32
<span class="token punctuation">}</span>
net ipsec traffic-selector ts-to-suricata <span class="token punctuation">{</span>
destination-address <span class="token number">172.31</span>.11.20/32
ipsec-policy ipsec-policy-to-suricata
order <span class="token number">2</span>
source-address <span class="token number">172.31</span>.11.10/32
<span class="token punctuation">}</span>
Moving Arkime into GRE
By default, your tap will be on a VLAN. You will need to update BIG-IP tunnels to move that mac address into the GRETAP. This mac address is pulled from the GRETAP interface on Linux and is also used in the SSLO wizard (which will show the VLAN interface in the GUI).
admin@<span class="token punctuation">(</span>ip-172-31-100-10<span class="token punctuation">)</span><span class="token punctuation">(</span>cfg-sync Standalone<span class="token punctuation">)</span><span class="token punctuation">(</span>LICENSE EXPIRES IN <span class="token number">2</span> DAYS:Active<span class="token punctuation">)</span><span class="token punctuation">(</span>/Common<span class="token punctuation">)</span><span class="token punctuation">(</span>tmos.net.fdb<span class="token punctuation">)</span><span class="token comment"># list</span>
net fdb tunnel arkime-tap <span class="token punctuation">{</span>
records <span class="token punctuation">{</span>
<span class="token number">72</span>:a5:76:d2:a3:ec <span class="token punctuation">{</span>
endpoint <span class="token number">172.31</span>.15.198
<span class="token punctuation">}</span>
<span class="token punctuation">}</span>
<span class="token punctuation">}</span>
Pool Members in SSL Orchestrator
If you have worked with SSL Orchestrator, youknow that you cannot select systems that are on tunnels. This means that we need to configure the security services and then edit those objects after we have created them in SSLO, and to do this we need to disable strictness. Here we add a GRE profile to Arkime and replace the pool members for Suricata.
ltm pool arkime<span class="token operator">-</span>outside <span class="token punctuation">{</span>
members <span class="token punctuation">{</span>
172.31.15.198:any <span class="token punctuation">{</span>
address 172.31.15.198
session monitor<span class="token operator">-</span>enabled
state up
<span class="token punctuation">}</span>
<span class="token punctuation">}</span>
monitor gateway_icmp
profiles <span class="token punctuation">{</span>
arkime<span class="token operator">-</span>gre<span class="token operator">-</span>profile
<span class="token punctuation">}</span>
<span class="token punctuation">}</span>
ltm pool suricata<span class="token operator">-</span>ipsec<span class="token operator">-</span>pool <span class="token punctuation">{</span>
members <span class="token punctuation">{</span>
100.64.0.20:any <span class="token punctuation">{</span>
address 100.64.0.20
session monitor<span class="token operator">-</span>enabled
state up
<span class="token punctuation">}</span>
<span class="token punctuation">}</span>
monitor gateway_icmp
<span class="token punctuation">}</span>
ltm virtual ssloS_suricate_ipsec.app<span class="token operator">/</span>ssloS_suricata_ipsec<span class="token operator">-</span>t<span class="token operator">-</span>4 <span class="token punctuation">{</span>
app<span class="token operator">-</span>service <span class="token operator">/</span>Common<span class="token operator">/</span>ssloS_suricata_ipsec.app<span class="token operator">/</span>ssloS_suricata_ipsec
creation<span class="token operator">-</span>time 2023<span class="token operator">-</span>06<span class="token operator">-</span>19:13:44:55
description <span class="token string">"In-line service (2023-6-19 13:44:47)"</span>
destination 0.0.0.0<span class="token operator">%</span>0:any
internal
ip<span class="token operator">-</span>protocol tcp
last<span class="token operator">-</span>modified<span class="token operator">-</span>time 2023<span class="token operator">-</span>06<span class="token operator">-</span>19:13:45:35
mask any
pool suricata<span class="token operator">-</span>ipsec<span class="token operator">-</span>pool
profiles <span class="token punctuation">{</span>
ssloS_suricate_ipsec.app<span class="token operator">/</span>ssloS_suricata_ipsec<span class="token operator">-</span>service <span class="token punctuation">{</span> <span class="token punctuation">}</span>
ssloS_suricate_ipsec.app<span class="token operator">/</span>ssloS_suricata_ipsec<span class="token operator">-</span>ss<span class="token operator">-</span>client <span class="token punctuation">{</span>
context serverside
<span class="token punctuation">}</span>
ssloS_suricate_ipsec.app<span class="token operator">/</span>ssloS_suricata_ipsec<span class="token operator">-</span>tcp<span class="token operator">-</span>lan <span class="token punctuation">{</span>
context clientside
<span class="token punctuation">}</span>
ssloS_suricate_ipsec.app<span class="token operator">/</span>ssloS_suricata_ipsec<span class="token operator">-</span>tcp<span class="token operator">-</span>wan <span class="token punctuation">{</span>
context serverside
<span class="token punctuation">}</span>
<span class="token punctuation">}</span>
serverssl<span class="token operator">-</span>use<span class="token operator">-</span>sni disabled
<span class="token keyword">source</span> 0.0.0.0<span class="token operator">/</span>0
translate<span class="token operator">-</span>address disabled
translate<span class="token operator">-</span>port disabled
vlans<span class="token operator">-</span>enabled
vs<span class="token operator">-</span>index 32
<span class="token punctuation">}</span>
The Results
Verify BIG-IP is only sending IPSEC traffic. From the BIG-IP perspective interface 1.2 is traffic TO the Suricata instance, 1.3 is traffic FROM the Suricata instance and 1.6 is traffic to the Arkime instance. Please note you need to look at the traffic from the interface and NOT the VLAN as packet captures at the VLAN will be by after TMM processing.
Please see the attached PDF for more info on BIG-IP config and realized network state.
Suricata Log
If we look in the eve.json file, we can see the http traffic
<span class="token property">"timestamp"</span><span class="token operator">:</span> <span class="token string">"2023-12-27T23:06:14.600407+0000"</span><span class="token punctuation">,</span>
<span class="token property">"flow_id"</span><span class="token operator">:</span> <span class="token number">1273598159638494</span><span class="token punctuation">,</span>
<span class="token property">"event_type"</span><span class="token operator">:</span> <span class="token string">"fileinfo"</span><span class="token punctuation">,</span>
<span class="token property">"src_ip"</span><span class="token operator">:</span> <span class="token string">"10.1.4.11"</span><span class="token punctuation">,</span>
<span class="token property">"src_port"</span><span class="token operator">:</span> <span class="token number">443</span><span class="token punctuation">,</span>
<span class="token property">"dest_ip"</span><span class="token operator">:</span> <span class="token string">"x.x.x.x"</span><span class="token punctuation">,</span>
<span class="token property">"dest_port"</span><span class="token operator">:</span> <span class="token number">55968</span><span class="token punctuation">,</span>
<span class="token property">"proto"</span><span class="token operator">:</span> <span class="token string">"TCP"</span><span class="token punctuation">,</span>
<span class="token property">"community_id"</span><span class="token operator">:</span> <span class="token string">"1:wA/9qp6gL7lAwPGHPYOVUBrRjVU="</span><span class="token punctuation">,</span>
<span class="token property">"http"</span><span class="token operator">:</span> <span class="token punctuation">{</span>
<span class="token property">"hostname"</span><span class="token operator">:</span> <span class="token string">"3.x.x.x6"</span><span class="token punctuation">,</span>
<span class="token property">"url"</span><span class="token operator">:</span> <span class="token string">"/stylesheets/style.css"</span><span class="token punctuation">,</span>
<span class="token property">"http_user_agent"</span><span class="token operator">:</span> <span class="token string">"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15"</span><span class="token punctuation">,</span>
<span class="token property">"http_content_type"</span><span class="token operator">:</span> <span class="token string">"text/css"</span><span class="token punctuation">,</span>
<span class="token property">"http_refer"</span><span class="token operator">:</span> <span class="token string">"https://3.x.x.x6/"</span><span class="token punctuation">,</span>
<span class="token property">"http_method"</span><span class="token operator">:</span> <span class="token string">"GET"</span><span class="token punctuation">,</span>
<span class="token property">"protocol"</span><span class="token operator">:</span> <span class="token string">"HTTP/1.1"</span><span class="token punctuation">,</span>
<span class="token property">"status"</span><span class="token operator">:</span> <span class="token number">200</span><span class="token punctuation">,</span>
<span class="token property">"length"</span><span class="token operator">:</span> <span class="token number">185</span>
<span class="token punctuation">}</span><span class="token punctuation">,</span>
<span class="token property">"app_proto"</span><span class="token operator">:</span> <span class="token string">"http"</span><span class="token punctuation">,</span>
<span class="token property">"fileinfo"</span><span class="token operator">:</span> <span class="token punctuation">{</span>
<span class="token property">"filename"</span><span class="token operator">:</span> <span class="token string">"/stylesheets/style.css"</span><span class="token punctuation">,</span>
<span class="token property">"sid"</span><span class="token operator">:</span> <span class="token punctuation">[</span><span class="token punctuation">]</span><span class="token punctuation">,</span>
<span class="token property">"gaps"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span>
<span class="token property">"state"</span><span class="token operator">:</span> <span class="token string">"CLOSED"</span><span class="token punctuation">,</span>
<span class="token property">"stored"</span><span class="token operator">:</span> <span class="token boolean">false</span><span class="token punctuation">,</span>
<span class="token property">"size"</span><span class="token operator">:</span> <span class="token number">176</span><span class="token punctuation">,</span>
<span class="token property">"tx_id"</span><span class="token operator">:</span> <span class="token number">3</span>
<span class="token punctuation">}</span>
<span class="token punctuation">}</span>
We can also see traffic travesing the GRE interface
ubuntu@ip-172-31-11-20:/var/log/suricata$ <span class="token function">sudo</span> tcpdump -ni gre2 port <span class="token number">443</span>
tcpdump: verbose output suppressed, use -v<span class="token punctuation">[</span>v<span class="token punctuation">]</span><span class="token punctuation">..</span>. <span class="token keyword">for</span> full protocol decode
listening on gre2, link-type LINUX_SLL <span class="token punctuation">(</span>Linux cooked v1<span class="token punctuation">)</span>, snapshot length <span class="token number">262144</span> bytes
01:13:31.173786 IP X.X.X.X.55405 <span class="token operator">></span> <span class="token number">10.1</span>.4.11.443: Flags <span class="token punctuation">[</span>P.<span class="token punctuation">]</span>, <span class="token function">seq</span> <span class="token number">3043973525</span>:3043974270, ack <span class="token number">3641341613</span>, win <span class="token number">18556</span>, options <span class="token punctuation">[</span>nop,nop,TS val <span class="token number">3523761960</span> ecr <span class="token number">3523749045</span><span class="token punctuation">]</span>, length <span class="token number">745</span>
01:13:31.175573 IP <span class="token number">10.1</span>.4.11.443 <span class="token operator">></span> X.X.X.X.55405: Flags <span class="token punctuation">[</span>.<span class="token punctuation">]</span>, ack <span class="token number">745</span>, win <span class="token number">25620</span>, options <span class="token punctuation">[</span>nop,nop,TS val <span class="token number">3523761962</span> ecr <span class="token number">3523761960</span><span class="token punctuation">]</span>, length <span class="token number">0</span>
01:13:31.182178 IP <span class="token number">10.1</span>.4.11.443 <span class="token operator">></span> X.X.X.X.55405: Flags <span class="token punctuation">[</span>P.<span class="token punctuation">]</span>, <span class="token function">seq</span> <span class="token number">1</span>:990, ack <span class="token number">745</span>, win <span class="token number">25620</span>, options <span class="token punctuation">[</span>nop,nop,TS val <span class="token number">3523761969</span> ecr <span class="token number">3523761960</span><span class="token punctuation">]</span>, length <span class="token number">989</span>
01:13:31.182178 IP <span class="token number">10.1</span>.4.11.443 <span class="token operator">></span> X.X.X.X.55405: Flags <span class="token punctuation">[</span>P.<span class="token punctuation">]</span>, <span class="token function">seq</span> <span class="token number">990</span>:995, ack <span class="token number">745</span>, win <span class="token number">25620</span>, options <span class="token punctuation">[</span>nop,nop,TS val <span class="token number">3523761969</span> ecr <span class="token number">3523761960</span><span class="token punctuation">]</span>, length <span class="token number">5</span>
01:13:31.183529 IP X.X.X.X.55405 <span class="token operator">></span> <span class="token number">10.1</span>.4.11.443: Flags <span class="token punctuation">[</span>.<span class="token punctuation">]</span>, ack <span class="token number">990</span>, win <span class="token number">19545</span>,
Arkime
In the graph below, we can see that I have captured 100,000+ flows over the last week
We can also see that I have the ability to look at details of the traffic and examine a packet capture form the system.
Proceed with Careful Consideration
This configuration required making many changes to what SSLO deploys and that you disable strictness. The other area of concern is that IPSEC tunnels end up being pinned to a single TMM; reducing capacity. If you are using this approach because you have decided that you cannot trust the network you do not have physical control of it is worth a discussion on should you deploy this security service next to the environment you do not trust, such as a colo that is cloud-adjacent where you can retain physical control of the network.
Simplification
By adjusting the approach and using an IPS solution that can support SSL; such as F5 Advanced Firewall Manager and leveraging request logging (if the apps are constrained to HTTP) to preclude the use of a network tap much of the complexity would be removed.
BIG-IP Flexibility
I have been working with the BIG-IP platform for many years and I continue to find new capabilities in the product. The flexibility the platform allows customers to solve the problems their organizations face in the environment they are choosing to use them.