Forum Discussion
f5gurunot
Cirrus
CirrusTCP Option 28 X-Forwarded-For Header
Using Akamai for DDOS mitigation and the requests sent to us show Akamai IP instead of the Client IP. I have enabled TCP Option 28 and trying to insert the Client IP into X-Forwarded-For header. I had to add this configuration into an existing iRule and getting Err_Connection_Reset. Any assistance on what I'm doing wrong would be greatly appreciated.
The entire iRule is below and I have bolded the new TCP Option 28 config. The rest of the iRule was working properly before I added this.
when CLIENT_ACCEPTED {
set proto 0
if { [PROFILE::exists clientssl] } {
set default_tls_pool [LB::server pool]
set detect_handshake 1
SSL::disable
TCP::collect
} else {
log local0. "This iRule is applied to a VS that has no clientssl profile."
set detect_handshake 0
}
set opt28 [TCP::option get 28]
if { [string length $opt28] == 5 } {
binary scan $opt28 c ver
if { $ver != 1 } {
log local0. "Unsupported Akamai version: $ver"
} else {
set optaddr [IP::addr parse -ipv4 $opt28 1]
}
}
}
when CLIENTSSL_CLIENTHELLO {
set proto 1
}
when HTTP_REQUEST {
if {$proto} {
if { [info exists optaddr] } {
HTTP::header insert X-Forwarded-Proto https
HTTP::header insert X-Forwarded-For $optaddr
} else {
HTTP::header insert X-Forwarded-Proto https
}
} else {
if { [info exists optaddr] } {
HTTP::header insert X-Forwarded-Proto http
HTTP::header insert X-Forwarded-For $optaddr
} else {
HTTP::header insert X-Forwarded-Proto http
}
}
}
when CLIENT_DATA {
if { ($detect_handshake) } {
binary scan [TCP::payload] cSS tls_xacttype tls_version tls_recordlen
switch $tls_version {
"769" -
"770" -
"771" {
if { ($tls_xacttype == 22) } {
binary scan [TCP::payload] @5c tls_action
if { not (($tls_action == 1) && ([TCP::payload length] > $tls_recordlen)) } {
set detect_handshake 0
}
}
}
default {
set detect_handshake 0
}
}
if { ($detect_handshake) } {
set record_offset 43
binary scan [TCP::payload] @${record_offset}c tls_sessidlen
set record_offset [expr {$record_offset + 1 + $tls_sessidlen}]
binary scan [TCP::payload] @${record_offset}S tls_ciphlen
set record_offset [expr {$record_offset + 2 + $tls_ciphlen}]
binary scan [TCP::payload] @${record_offset}c tls_complen
set record_offset [expr {$record_offset + 1 + $tls_complen}]
if { ([TCP::payload length] >= $record_offset) } {
binary scan [TCP::payload] @${record_offset}S tls_extenlen
set record_offset [expr {$record_offset + 2}]
binary scan [TCP::payload] @${record_offset}a* tls_extensions
for { set x 0 } { $x < $tls_extenlen } { incr x 4 } {
set start [expr {$x}]
binary scan $tls_extensions @${start}SS etype elen
if { ($etype == "00") } {
set grabstart [expr {$start + 9}]
set grabend [expr {$elen - 5}]
binary scan $tls_extensions @${grabstart}A${grabend} tls_servername
set start [expr {$start + $elen}]
} else {
set start [expr {$start + $elen}]
}
set x $start
}
if { ([info exists tls_servername] ) } {
set ssl_profile [class match -value [string tolower $tls_servername] ends_with tls_servername]
set tls_pool [class match -value [string tolower $tls_servername] ends_with tls_servername_pool]
if { $ssl_profile == "" } {
SSL::enable
} else {
set ssl_profile_enable "SSL::profile $ssl_profile"
catch { eval $ssl_profile_enable }
if { not ($tls_pool == "") } {
pool $tls_pool
} else {
pool $default_tls_pool
}
SSL::enable
}
} else {
SSL::enable
}
} else {
SSL::enable
}
set detect_handshake 0
TCP::release
} else {
set detect_handshake 0
SSL::enable
TCP::release
}
}
}
when CLIENT_DATA {
set opt28 [TCP::option get 28]
if { [string length $opt28] == 4 } {
binary scan $opt28 H8 addr
scan $addr "%2x%2x%2x%2x" ip1 ip2 ip3 ip4
set optaddr "$ip1.$ip2.$ip3.$ip4"
log local0. "optaddr is $optaddr"
log local0. "ip addr parse result is [IP::addr parse -ipv4 $opt28]"
}
}
- xuwen
Cumulonimbus
Have you bound the created tcp_option profile in this VS？
create ltm profile tcp PROFILE_NAME tcp-options “{option <first|last>} {option <first|last>}”
Can you give the configuration of list ltm virtual xxx and list ltm profile tcp tcp_option_XX and the error log in /var/log/ltm
f5gurunotJust ran the following and not getting connection reset anymore.
tmsh create ltm profile tcp tcp_opt tcp-options "{28 first}"However, still not seeing the Client IP.
Also, tried changing the HTTP_REQUEST to:
when HTTP_REQUEST {
if {$proto} {
HTTP::header insert X-Forwarded-Proto https
}
else {
HTTP::header insert X-Forwarded-Proto http
}
if { [info exists optaddr] } {
HTTP::header insert X-Forwarded-For $optaddr
}
}- xuwen
can you insert a code below "set opt28 [TCP::option get 28]"
log local0. "tcp option 28 length is [string length $opt28]"and show the log give me, tail -f /var/log/ltm
- f5gurunot
ltm virtual test_443 {
destination 5.5.5.5:https
ip-protocol tcp
last-modified-time 2023-02-02:18:49:31
mask 255.255.255.255
policies {
asm_auto_l7_policy__test_443 { }
}
pool test_80
profiles {
ASM_WebASM_US_ONLY { }
analytics { }
http { }
manual_test.com {
context clientside
}
tcp-analytics { }
tcp_opt { }
websecurity { }
websocket { }
}
rules {
Akamai_Opt28
}
security-log-profiles {
"Log illegal requests"
}
serverssl-use-sni disabled
source 0.0.0.0/0
source-address-translation {
type automap
}
translate-address enabled
translate-port enabled
vs-index 40
}
ltm profile tcp tcp_opt {
app-service none
tcp-options "{28 first}"
}
- xuwen
The simplest way is to perform tcpdump on the outside vlan(clientside) of F5. Wireshark observes whether have tcp option 28 field.
For some company network, F5 only performs tcp forwarding, F5 performs tcp option 254 insertion, pool members nginx performs SSL offload and reads the value of F5's tcp option field, and then inserts it into HTTP XFF header. One of the pits is:
Nginx downloads the open-source TOA module to read the value of tcp option, which can only be read in the tcp three handshakes. Unlike F5, F5 is convenient to directly use the TCP:: option function.F5 must be in SERVER_INIT event in V14+ version, execute tcp option 254 insertion. If F5 is lower than V14, nginx will be unable to read the value of tcp option 254
- f5gurunot
We are on version 17, will get a tcpdump.
- xuwen
Observe the clientside tcpdump file of F5 to see if there is a tcp option 28 field.
If not, check whether there is a firewall or other security cleaning equipment or The reverse proxy device in front of F5，It is better to restore iRules to a state without tcp option 28 and exec tcpdump command.
if wireshark pcap file has tcp option 28 field:
Because your iRules in CLIENT_ACCEPED event executed TCP:: collect command, I personally estimate that your tcp option 28 code should be placed in CLIENT_DATA event(personally advise, have not test in BIGIP VE environment)
when CLIENT_DATA { set opt28 [TCP::option get 28] log local0. "tcp option 28 length is [string length $opt28]" ........ }
- xuwen
i just used two BIGIP V16 VE test,
F5-1(set/insert tcp option 28) ====> F5-2(read/get tcp option 28 value)
i find that:
if F5-1 set tcp option 28 not in SERVER_INIT event(test in SERVER_CONNECTED), it will insert tcp option 28 in tcp data(not insert in tcp three handshake process), F5-2 in CLIENT_ACCEPTED event log [string length [TCP::option get 28]] is 0 and in CLIENT_DATA or HTTP_REQUEST event log [string length [TCP::option get 28]] is 4, This corresponds to the picture Akamai gave you the pcap.png content，Akamai insert tcp option 28 is in tcp data, not in tcp three handshake process，so you use set opt28 [TCP::option get 28]; log local0. "tcp option 28 length is [string length $opt28]"
in CLIENT_ACCEPTED event, the value is 0
Feb 3 23:30:49 f5 info tmm[10530]: Rule /Common/gslb_http <CLIENT_ACCEPTED>: tcp option 28 length in CLIENT_ACCEPTED is 0 Feb 3 23:30:49 f5 info tmm[10530]: Rule /Common/gslb_http <CLIENT_DATA>: tcp option 28 length in CLIENT_DATA is 4 Feb 3 23:30:49 f5 info tmm[10530]: Rule /Common/gslb_http <CLIENT_ACCEPTED>: tcp option 28 length in CLIENT_ACCEPTED is 0 Feb 3 23:30:49 f5 info tmm[10530]: Rule /Common/gslb_http <HTTP_REQUEST>: tcp option 28 length in HTTP_REQUEST is 4 Feb 3 23:30:49 f5 info tmm[10530]: Rule /Common/gslb_http <CLIENT_DATA>: tcp option 28 length in CLIENT_DATA is 4 Feb 3 23:30:49 f5 info tmm[10530]: Rule /Common/gslb_http <HTTP_REQUEST>: tcp option 28 length in HTTP_REQUEST is 4if F5-1 set tcp option 28 in SERVER_INIT event, it will insert tcp option 28 in (tcp syn, tcp ack, tcp data), and F5-2 in CLIENT_ACCEPTED, CLIENT_DATA, HTTP_REQUEST event is log [string length [TCP::option get 28]] is 4
Feb 3 23:32:11 f5 info tmm[10530]: Rule /Common/gslb_http <CLIENT_ACCEPTED>: tcp option 28 length in CLIENT_ACCEPTED is 4 Feb 3 23:32:11 f5 info tmm[10530]: Rule /Common/gslb_http <CLIENT_DATA>: tcp option 28 length in CLIENT_DATA is 4 Feb 3 23:32:11 f5 info tmm[10530]: Rule /Common/gslb_http <CLIENT_ACCEPTED>: tcp option 28 length in CLIENT_ACCEPTED is 4 Feb 3 23:32:11 f5 info tmm[10530]: Rule /Common/gslb_http <HTTP_REQUEST>: tcp option 28 length in HTTP_REQUEST is 4 Feb 3 23:32:11 f5 info tmm[10530]: Rule /Common/gslb_http <CLIENT_DATA>: tcp option 28 length in CLIENT_DATA is 4 Feb 3 23:32:11 f5 info tmm[10530]: Rule /Common/gslb_http <HTTP_REQUEST>: tcp option 28 length in HTTP_REQUEST is 4 - Paulius
MVP
f5gurunot From my understanding Akamai is the one that would insert the true client-IP into the HTTP header because they are the source of the connection as far as your F5 is concerned. If the true client IP doesn't exist in the HTTP header that is squarly on Akamai. It is possible that Akamai has provided you the incorrect HTTP header field that they use to inject the client-IP into the HTTP header or it is possible that your pool members are not searching for the correct header field in the HTTP header. The only thing that inserting the client-IP into the HTTP header on the F5 will achieve would be to include the Akamai IP. Based on your virtual server configuration your pool members think the F5 is the source of the communication but should have an HTTP header inserted by Akamai and one inserted by the iRule named X-Forwarded-For. It is possible that you are overwriting the HTTP header field value with what is in the iRule if you all are using the same header field name.
- f5gurunot
If you are referring to the True-Client-IP header that Akamai uses instead of the X-Forwarded-For header that is not an option because our Akamai solution operates in TCP mode, not HTTP mode. It only supports the following methods of source IP forwarding: CIP, Option 28, Proxy Protocol v1 & v2. We have enabled Option 28.
