TLS Fingerprinting JA3 iRule Application: Rate limit and block malicious traffic based on TLS signature
In this article, we use the same techniques, as some previous authors, to enable a TLS Fingerprinting iRule and proc to rate limit and block TLS clients based on generated TLS signatures. Related Resources F5SIRT Aaron Brailsford's https://devcentral.f5.com/s/articles/TLS-Fingerprinting-to-profile-SSL-TLS-clients-without-decryption article shows a sample proc iRule that will produce (unique) TLS signatures from clients accessing a TLS Virtual Server. The F5SIRT also have the https://devcentral.f5.com/s/articles/HTTP-Brute-Force-Mitigation-Playbook-BIG-IP-LTM-Mitigation-Options-for-HTTP-Brute-Force-Attacks-Chapter-3 and https://devcentral.f5.com/s/articles/HTTP-Brute-Force-Mitigation-Playbook-Appendix where a TLS Fingerprinting iRule and proc iRule was also used to rate limit and block known malicious fingerprints from HTTP based clients. Credit to the original post in https://devcentral.f5.com/s/articles/tls-fingerprinting-a-method-for-identifying-a-tls-client-without-decrypting-24598 Sample Application: Protecting IMAPS/POP3S service IMAPS/POP3S has been around for a long time and are also a target of brute force attacks. We will use the TLS Fingerprinting iRule and proc to generate a TLS signature and then rate limit a specific client or block a specific TLS signature . Using the "Library Rule" from https://devcentral.f5.com/s/articles/TLS-Fingerprinting-to-profile-SSL-TLS-clients-without-decryption, we create the proc iRule, I will name it "fingerprintTLSproc". You can name it as you per your needs, just note that it is important to remember the name of the proc iRule as it will be referenced in next iRule - the rate limiting/block iRule. This will be listed as iRule#1. Note that this iRule does not need to be applied to a Virtual Server. iRule#1 - fingerprintTLSproc ## Library-Rule ## JA3 TLS Fingerprint Procedure ################# ## ## Author: Aaron Brailsford, 06/2020 ## Based on the TLS Fingerprinting iRule by Kevin Stewart @ https://devcentral.f5.com/s/articles/tls-fingerprinting-a-method-for-identifying-a-tls-client-without-decrypting-24598 ## Derived from Lee Brotherston's "tls-fingerprinting" project @ https://github.com/LeeBrotherston/tls-fingerprinting ## Purpose: to identify the user agent based on unique characteristics of the TLS ClientHello message ## Input: ## Full TCP payload collected in CLIENT_DATA event of a TLS handshake ClientHello message ## Record length (rlen) ## TLS inner version (sslversion) ############################################## proc fingerprintTLS { payload rlen sslversion } { ## The first 43 bytes of a ClientHello message are the record type, TLS versions, some length values and the ## handshake type. We should already know this stuff from the calling iRule. We're also going to be walking the ## packet, so the field_offset variable will be used to track where we are. set field_offset 43 ## The first value in the payload after the offset is the session ID, which may be empty. Grab the session ID length ## value and move the field_offset variable that many bytes forward to skip it. binary scan ${payload} @${field_offset}c sessID_len set field_offset [expr {${field_offset} + 1 + ${sessID_len}}] ## The next value in the payload is the ciphersuite list length (how big the ciphersuite list is. binary scan ${payload} @${field_offset}S cipherList_len ## Now that we have the ciphersuite list length, let's offset the field_offset variable to skip over the length (2) bytes ## and go get the ciphersuite list. set field_offset [expr {${field_offset} + 2}] binary scan ${payload} @${field_offset}S[expr {${cipherList_len} / 2}] cipherlist_decimal ## Next is the compression method length and compression method. First move field_offset to skip past the ciphersuite ## list, then grab the compression method length. Then move field_offset past the length (2) ## Finally, move field_offset past the compression method bytes. set field_offset [expr {${field_offset} + ${cipherList_len}}] binary scan ${payload} @${field_offset}c compression_len set field_offset [expr {${field_offset} + 1}] set field_offset [expr {${field_offset} + ${compression_len}}] ## We should be in the extensions section now, so we're going to just run through the remaining data and ## pick out the extensions as we go. But first let's make sure there's more record data left, based on ## the current field_offset vs. rlen. if { [expr {${field_offset} < ${rlen}}] } { ## There's extension data, so let's go get it. Skip the first 2 bytes that are the extensions length set field_offset [expr {${field_offset} + 2}] ## Make a variable to store the extension types we find set extensions_list "" ## Pad rlen by 1 byte set rlen [expr {${rlen} + 1}] while { [expr {${field_offset} <= ${rlen}}] } { ## Grab the first 2 bytes to determine the extension type binary scan ${payload} @${field_offset}S ext set ext [expr {$ext & 0xFFFF}] ## Store the extension in the extensions_list variable lappend extensions_list ${ext} ## Increment field_offset past the 2 bytes of the extension type set field_offset [expr {${field_offset} + 2}] ## Grab the 2 bytes of extension lenth binary scan ${payload} @${field_offset}S ext_len ## Increment field_offset past the 2 bytes of the extension length set field_offset [expr {${field_offset} + 2}] ## Look for specific extension types in case these need to increment the field_offset (and because we need their values) switch $ext { "11" { ## ec_point_format - there's another 1 byte after length ## Grab the extension data binary scan ${payload} @[expr {${field_offset} + 1}]s ext_data set ec_point_format ${ext_data} } "10" { ## elliptic_curves - there's another 2 bytes after length ## Grab the extension data binary scan ${payload} @[expr {${field_offset} + 2}]S[expr {(${ext_len} - 2) / 2}] ext_data set elliptic_curves ${ext_data} } default { ## Grab the otherwise unknown extension data binary scan ${payload} @${field_offset}H[expr {${ext_len} * 2}] ext_data } } ## Increment the field_offset past the extension data length. Repeat this loop until we reach rlen (the end of the payload) set field_offset [expr {${field_offset} + ${ext_len}}] } } ## Now let's compile all of that data. ## The cipherlist values need masking with 0xFFFF to return the unsigned integers we need foreach cipher $cipherlist_decimal { lappend cipd [expr {$cipher & 0xFFFF}] } set cipd_str [join $cipd "-"] if { ( [info exists extensions_list] ) and ( ${extensions_list} ne "" ) } { set exte [join ${extensions_list} "-"] } else { set exte "" } if { ( [info exists elliptic_curves] ) and ( ${elliptic_curves} ne "" ) } { set ecur [join ${elliptic_curves} "-"] } else { set ecur "" } if { ( [info exists ec_point_format] ) and ( ${ec_point_format} ne "" ) } { set ecfp [join ${ec_point_format} "-"] } else { set ecfp "" } set ja3_str "${sslversion},${cipd_str},${exte},${ecur},${ecfp}" ## binary scan [md5 ${ja3_str}] H* ja3_digest ## Un-comment this line to display the fingerprint string in the LTM log for troubleshooting #log local0. "ja3 = ${ja3_str}" return ${ja3_str} } Here is the rate limiting / blocking iRule. This iRule will monitor TLS signatures and a corresponding IP address and if it exceeds the defined maximum rate of requests - the maxRate variable - the iRule will drop the traffic from the specific client IP and TLS signature. There is also a logic to check a known malicious TLS signature defined in a iRule Datagroup and if it matches, the iRule will drop the connection. I have named this iRule fingerprintTLSirule-ratelimit and listed as iRule#2. Note in this iRule, you will have to properly reference proc iRule for the detected signatures be checked - see the section of this iRule commented "## Call the fingerprintTLS proc". Note the syntax in the reference line for calling the proc iRule is "call <iRule>:<proc>" . Note as well that this fingerprintTLSirule-ratelimit iRule need to be applied to a Virtual Server. Note the "static::maxRate" variable as this controls the maxim number of requests before iRule rate limits a TLS signature hash and IP address combination. Adjust this value as per your needs. iRule#2: fingerprintTLSirule-ratelimit when RULE_INIT { # Default rate to limit requests set static::maxRate 15 # Default rate to set static::warnRate 12 # During this many seconds set static::timeout 1 } when CLIENT_ACCEPTED { ## Collect the TCP payload TCP::collect } when CLIENT_DATA { ## Get the TLS packet type and versions if { ! [info exists rlen] } { ## We actually only need the recort type (rtype), record length (rlen) handshake type (hs_type) and 'inner' SSL version (inner_sslver) here ## But it's easiest to parse them all out of the payload along with the bytes we don't need (outer_sslver & rilen) binary scan [TCP::payload] cSScH6S rtype outer_sslver rlen hs_type rilen inner_sslver if { ( ${rtype} == 22 ) and ( ${hs_type} == 1 ) } { ## This is a TLS ClientHello message (22 = TLS handshake, 1 = ClientHello) ## Call the fingerprintTLS proc set ja3_fingerprint [call fingerprintTLSproc::fingerprintTLS [TCP::payload] ${rlen} ${inner_sslver}] binary scan [md5 ${ja3_fingerprint}] H* ja3_digest ### Do Something here ### log local0. "[IP::client_addr]:[TCP::client_port] ja3 ${ja3_fingerprint}->${ja3_digest}" #check if fingerprint matches a known malicious fingerprint, if yes, drop connection if {[class match ${ja3_fingerprint} equals malicious_fingerprintdb]}{ set malicious_fingerprint [class match -value ${ja3_fingerprint} equals malicious_TLSfingerprintdb] drop log local0. "known malicious fingerprint matched $malicious_fingerprint - Action:DROP!" } #use generated digest of the signature for rate limiting set suspicious_fingerprint ${ja3_digest} #rate limit fingerprint # Increment and Get the current request count bucket #set epoch [clock seconds] #monitor an unrecognized fingerprint and rate limit it set currentCount [table incr -mustexist "Count_[IP::client_addr]_${suspicious_fingerprint}"] if { $currentCount eq "" } { # Initialize a new request count bucket table set "Count_[IP::client_addr]_${suspicious_fingerprint}" 1 indef $static::timeout set currentCount 1 } # Actually check fingerprint for being over limit if { $currentCount >= $static::maxRate } { log local0. "ERROR: fingerprint:[IP::client_addr]_${suspicious_fingerprint} exceeded ${static::maxRate} requests per second. Rejecting request. Current requests: ${currentCount}." event disable all drop } if { $currentCount > $static::warnRate } { log local0. "WARNING: fingerprint:[IP::client_addr]_${suspicious_fingerprint} exceeded ${static::warnRate} requests per second. Will reject at ${static::maxRate}. Current requests: ${currentCount}." } log local0. "fingerprint:[IP::client_addr]_${suspicious_fingerprint}: currentCount: ${currentCount}" ### Do Something here ### } } # Collect the rest of the record if necessary if { [TCP::payload length] < $rlen } { TCP::collect $rlen } ## Release the paylaod TCP::release } Sample Test Output A curl client simulates as an imaps (secure) client and successfully lists the folders for the sample user [root@curlclient] config # curl -k --url = "imaps://172.16.0.30/" --user "lala:lala" -v * Rebuilt URL to: =/ * Could not resolve host: = * Closing connection 0 curl: (6) Could not resolve host: = * Trying 172.16.0.30... * Connected to 172.16.0.30 (172.16.0.30) port 993 (#1) * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): * TLSv1.2 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Client hello (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS change cipher, Client hello (1): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * Server certificate: * subject: C=US; ST=WA; L=Seattle; O=MyCompany; OU=IT; CN=localhost.localdomain; emailAddress=root@localhost.localdomain * start date: May 13 13:57:07 2020 GMT * expire date: May 11 13:57:07 2030 GMT * issuer: C=US; ST=WA; L=Seattle; O=MyCompany; OU=IT; CN=localhost.localdomain; emailAddress=root@localhost.localdomain * SSL certificate verify result: self signed certificate (18), continuing anyway. < * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot (Ubuntu) ready. > B001 CAPABILITY < * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN < B001 OK Pre-login capabilities listed, post-login capabilities have more. > B002 AUTHENTICATE PLAIN bGFsYQBsYWxhAGxhbGE= < * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE < B002 OK Logged in > B003 LIST "" * < * LIST (\HasNoChildren \Drafts) "." Drafts * LIST (\HasNoChildren \Drafts) "." Drafts < * LIST (\HasNoChildren \Sent) "." Sent * LIST (\HasNoChildren \Sent) "." Sent < * LIST (\HasNoChildren \Trash) "." Trash * LIST (\HasNoChildren \Trash) "." Trash < * LIST (\HasNoChildren) "." Templates * LIST (\HasNoChildren) "." Templates < * LIST (\HasNoChildren) "." INBOX * LIST (\HasNoChildren) "." INBOX < B003 OK List completed (0.001 + 0.000 secs). * Connection #1 to host 172.16.0.30 left intact The iRule fingerprintTLSirule-ratelimitwill log the TLS signature generated when it called the fingerprintTLSproc::fingerprintTLS proc. The reference log can be seen in /var/log/ltm file of the BIG-IP as per configured in the iRule. The logs can be also be sent to a high speed logging server. Jul 24 02:40:09 behavioral-dos-v15 info tmm1[11152]: Rule /Common/fingerprintTLSirule <CLIENT_DATA>: 172.16.7.31:59158 ja3 771,49200-49196-49192-49188-49172-49162-163-159-107-106-57-56-136-135-49202-49198-49194-49190-49167-49157-157-61-53-132-49199-49195-49191-49187-49171-49161-162-158-103-64-51-50-154-153-69-68-49201-49197-49193-49189-49166-49156-156-60-47-150-65-49170-49160-22-19-49165-49155-10-255,11-10-13-15-13172,25-24-22-23-20-21-18-19-15-16-17,256->19e387a2748bc0f70bc463d3af4cd04a the TLS signature here is: 771,49200-49196-49192-49188-49172-49162-163-159-107-106-57-56-136-135-49202-49198-49194-49190-49167-49157-157-61-53-132-49199-49195-49191-49187-49171-49161-162-158-103-64-51-50-154-153-69-68-49201-49197-49193-49189-49166-49156-156-60-47-150-65-49170-49160-22-19-49165-49155-10-255,11-10-13-15-13172,25-24-22-23-20-21-18-19-15-16-17,256 This TLS signature can be defined in an iRule Datagroup and be matched as either a known good or bad TLS signature. As noted earlier, the iRule fingerprintTLSirule-ratelimit includes a logic block to drop known malicious TLS signature. mutt is a mail client in linux and if it connects to the reference Virtual Server where the fingerprintTLSirule-ratelimit iRule is applied, this is the sample TLS signature in the generated log Jul 24 02:37:35 behavioral-dos-v15 info tmm1[11152]: Rule /Common/fingerprintTLSirule <CLIENT_DATA>: 172.16.10.31:51844 ja3 771,4866-4867-4865-4868-49196-52393-49325-49162-49195-49324-49161-49200-52392-49172-49199-49171-157-49309-53-156-49308-47-159-52394-49311-57-158-49310-51,5-10-11-13-22-23-35-51-43-65281-0-45-28,23-24-25-29-30-256-257-258-259-260,0->f35ce21b44ac0b87d3266294bb1b0e20 mutt client's TLS signature is: 4866-4867-4865-4868-49196-52393-49325-49162-49195-49324-49161-49200-52392-49172-49199-49171-157-49309-53-156-49308-47-159-52394-49311-57-158-49310-51,5-10-11-13-22-23-35-51-43-65281-0-45-28,23-24-25-29-30-256-257-258-259-260 nmap has a NSE script that can brute force an imap service. This can be used ethically, however, it also possible to be used for malicious purpose. For testing purpose, I ran a nmap imap-brute NSE scan on a Virtual Server and as it is expected to send brute force traffic, there were multiple instances of the generated TLS signature. Jul 24 02:49:04 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule <CLIENT_DATA>: 172.16.10.31:51974 ja3 771,4866-4867-4865-51-57-53-47-49196-49200-163-159-52393-52392-52394-49327-49325-49315-49311-49245-49249-49239-49235-49195-49199-162-158-49326-49324-49314-49310-49244-49248-49238-49234-49188-49192-107-106-49267-49271-196-195-49187-49191-103-64-49266-49270-190-189-49162-49172-56-136-135-49161-49171-50-154-153-69-68-157-49313-49309-49233-156-49312-49308-49232-61-192-60-186-132-150-65-255,11-10-35-22-23-13-43-45-51-21,29-23-30-25-24,256->912a836a48eb490e243eb28eef562687 nmap imap-brute generated TLS signature is: 771,4866-4867-4865-51-57-53-47-49196-49200-163-159-52393-52392-52394-49327-49325-49315-49311-49245-49249-49239-49235-49195-49199-162-158-49326-49324-49314-49310-49244-49248-49238-49234-49188-49192-107-106-49267-49271-196-195-49187-49191-103-64-49266-49270-190-189-49162-49172-56-136-135-49161-49171-50-154-153-69-68-157-49313-49309-49233-156-49312-49308-49232-61-192-60-186-132-150-65-255,11-10-35-22-23-13-43-45-51-21,29-23-30-25-24,256 The occurrence of nmap imap-brute TLS signature was increasing as the nmap script brute forces the IMAP Virtual Server. Note in this output, the hash of the signature "912a836a48eb490e243eb28eef562687" was used as the search string, [root@behavioral-dos-v15:Active:Standalone] config # grep 912a836a48eb490e243eb28eef562687 /var/log/ltm | wc -l 122 [root@behavioral-dos-v15:Active:Standalone] config # grep 912a836a48eb490e243eb28eef562687 /var/log/ltm | wc -l 262 [root@behavioral-dos-v15:Active:Standalone] config # grep 912a836a48eb490e243eb28eef562687 /var/log/ltm | wc -l 402 [root@behavioral-dos-v15:Active:Standalone] config # grep 912a836a48eb490e243eb28eef562687 /var/log/ltm | wc -l 522 As in this test nmap scan, if we want to block nmap from scanning the IMAP Virtual Server, we can define the detected TLS Signature to a iRule Datagroup and when its matched, the traffic will be dropped. Here is the sample iRule Datagroup of type String. The TLS signature is added in the String part and the value is a name for the TLS signature Here is a sample log when fingerprintTLSirule-ratelimit iRule drops the connection from the known malicious TLS signature Jul 24 04:06:04 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: 172.16.10.31:57434 ja3 771,4866-4867-4865-51-57-53-47-49196-49200-163-159-52393-52392-52394-49327-49325-49315-49311-49245-49249-49239-49235-49195-49199-162-158-49326-49324-49314-49310-49244-49248-49238-49234-49188-49192-107-106-49267-49271-196-195-49187-49191-103-64-49266-49270-190-189-49162-49172-56-136-135-49161-49171-50-154-153-69-68-157-49313-49309-49233-156-49312-49308-49232-61-192-60-186-132-150-65-255,11-10-35-22-23-13-43-45-51-21,29-23-30-25-24,256->912a836a48eb490e243eb28eef562687 Jul 24 04:06:04 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: known malicious fingerprint matched nmapscanner - Action:DROP! Jul 24 04:06:04 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687: currentCount: 1 Jul 24 04:06:04 behavioral-dos-v15 warning tmm[11152]: 01260009:4: 172.16.10.31:57434 -> 172.16.0.30:993: Connection error: hud_ssl_handler:1202: alert(40) invalid profile unknown on VIP /Common/dos-vs-v15 You may want to use this when you have determined a TLS signature to be malicious. The fingerprintTLSirule-ratelimit iRule also have a rate limiting logic. TLS signatures hash can be generated and along with the client IP address, you can isolate and rate limit traffic should it exceeds the defined maximum rate of requests. Here is the sample log where I ran the nmap script imap-brute, it was fingerprinted and applied rate limiting thru the iRule fingerprintTLSirule-ratelimit [root@behavioral-dos-v15:Active:Standalone] config # grep -i Rejecting /var/log/ltm Jul 24 03:16:43 behavioral-dos-v15 info tmm1[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 15. Jul 24 03:16:43 behavioral-dos-v15 info tmm1[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 16. Jul 24 03:16:43 behavioral-dos-v15 info tmm1[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 17. Jul 24 03:16:43 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 18. Jul 24 03:16:43 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 19. Jul 24 03:16:43 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 20. Jul 24 03:17:29 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 15. Jul 24 03:17:51 behavioral-dos-v15 info tmm1[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 15. Jul 24 03:19:20 behavioral-dos-v15 info tmm1[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 15. Jul 24 03:19:20 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 16. Jul 24 03:19:20 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 17. Jul 24 03:19:20 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 18. Jul 24 03:19:21 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 20. Jul 24 03:19:32 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 15. Jul 24 03:19:32 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 16. Jul 24 03:19:32 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 17. Jul 24 03:19:33 behavioral-dos-v15 info tmm1[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 18. Jul 24 03:20:18 behavioral-dos-v15 info tmm1[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 15. Jul 24 03:20:18 behavioral-dos-v15 info tmm[11152]: Rule /Common/fingerprintTLSirule-ratelimit <CLIENT_DATA>: ERROR: fingerprint:172.16.10.31_912a836a48eb490e243eb28eef562687 exceeded 15 requests per second. Rejecting request. Current requests: 16. [root@behavioral-dos-v15:Active:Standalone] config # Considerations iRule operation are CPU intensive, thus, expect an increase in CPU usage on the BIG-IP. The sample iRule here were tested in a controlled lab environment. Please test the iRules before applying to your production traffic. These iRules can be useful to quickly mitigate an attack or unexpected traffic and a trade off of additional CPU resource usage increase for the protected service availability and security. The reference iRules also produces insight on the TLS signatures that accesses the TLS Virtual Server and may be useful to define a block or allow list thru a iRule Datagroup and optimize the access to the protected Virtual Server.TLS Fingerprinting to profile SSL/TLS clients without decryption
In the F5 SIRT we are always looking for new and better ways to profile incoming traffic to try and sort the wheat from the chaff; or in our case, usually, legitimate from illegitimate traffic in order to apply some kind of blocklist/allowlist to the traffic – and if we can do that automatically based on some thresholds, all the better. If the traffic is being decrypted – either in front of or on the BIG-IP – and it is some easily inspectable format like HTTP then we have plenty of tools at our disposal and could, for example, look for unusual User-Agent strings or other artefacts. There are times, though, when we need to look at something lower down the OSI model to classify the traffic, either because the traffic isn’t being decrypted or because the adversary is employing a wide range of L7 evasions (picking from a large list of potentially valid User-Agent strings for example). Often, though, malicious tools are built upon a standard set of lower-layer libraries or have hard coded TLS handshake parameters, and that’s where fingerprinting traffic at the TLS layer has great power. F5 already has a solution for TLS fingerprinting (https://devcentral.f5.com/s/articles/tls-fingerprinting-a-method-for-identifying-a-tls-client-without-decrypting-24598) based on work by Lee Brotherston, and that works extremely well when you want to consume the fingerprints directly in some way – for example, have a Class full of fingerprints and use matches to direct traffic – but I wanted a solution that would produce a smaller, easier to handle hash of the fingerprint that could be easily counted in tables or shipped off to a log aggregator. Enter JA3! Originally developed at Salesforce, but now an open-source methodology (https://github.com/salesforce/ja3), JA3 produces a nice easy-to-use hash. For example, the hash of the handshake used by Trickbot malware is: 6734f37431670b3ab4292b8f60f29984 As I say, you can then either use that fingerprint directly (for example you could count how many times you’ve seen that fingerprint in any given space of time and rate limit specific client groups) or use HSL to send it off to a remote logger. So to get into the iRules themselves, we’ll have two parts – a library rule and a rule that calls this library on each TLS handshake. Credit where credit is due, these are both heavily based on Kevin Stewarts earlier work as well as the JA3 algorithm. Library Rule Create the following iRule PROC – remember, whatever you call it in the TMUI is what appears in the first part of the [call] statement. Our call statement says [call Library-Rule::fingerprintTLS] so you’d want to save it as Library-Rule – if you already have Kevin’s rule deployed you’ll need to adjust to accommodate, of course. ## Library-Rule ## JA3 TLS Fingerprint Procedure ################# ## ## Author: Aaron Brailsford, 06/2020 ## Based on the TLS Fingerprinting iRule by Kevin Stewart @ https://devcentral.f5.com/s/articles/tls-fingerprinting-a-method-for-identifying-a-tls-client-without-decrypting-24598 ## Derived from Lee Brotherston's "tls-fingerprinting" project @ https://github.com/LeeBrotherston/tls-fingerprinting ## Purpose: to identify the user agent based on unique characteristics of the TLS ClientHello message ## Input: ##Full TCP payload collected in CLIENT_DATA event of a TLS handshake ClientHello message ##Record length (rlen) ##TLS inner version (sslversion) ############################################## proc fingerprintTLS { payload rlen sslversion } { ## The first 43 bytes of a ClientHello message are the record type, TLS versions, some length values and the ## handshake type. We should already know this stuff from the calling iRule. We're also going to be walking the ## packet, so the field_offset variable will be used to track where we are. set field_offset 43 ## The first value in the payload after the offset is the session ID, which may be empty. Grab the session ID length ## value and move the field_offset variable that many bytes forward to skip it. binary scan ${payload} @${field_offset}c sessID_len set field_offset [expr {${field_offset} + 1 + ${sessID_len}}] ## The next value in the payload is the ciphersuite list length (how big the ciphersuite list is. binary scan ${payload} @${field_offset}S cipherList_len ## Now that we have the ciphersuite list length, let's offset the field_offset variable to skip over the length (2) bytes ## and go get the ciphersuite list. set field_offset [expr {${field_offset} + 2}] binary scan ${payload} @${field_offset}S[expr {${cipherList_len} / 2}] cipherlist_decimal ## Next is the compression method length and compression method. First move field_offset to skip past the ciphersuite ## list, then grab the compression method length. Then move field_offset past the length (2) ## Finally, move field_offset past the compression method bytes. set field_offset [expr {${field_offset} + ${cipherList_len}}] binary scan ${payload} @${field_offset}c compression_len set field_offset [expr {${field_offset} + 1}] set field_offset [expr {${field_offset} + ${compression_len}}] ## We should be in the extensions section now, so we're going to just run through the remaining data and ## pick out the extensions as we go. But first let's make sure there's more record data left, based on ## the current field_offset vs. rlen. if { [expr {${field_offset} < ${rlen}}] } { ## There's extension data, so let's go get it. Skip the first 2 bytes that are the extensions length set field_offset [expr {${field_offset} + 2}] ## Make a variable to store the extension types we find set extensions_list "" ## Pad rlen by 1 byte set rlen [expr {${rlen} + 1}] while { [expr {${field_offset} <= ${rlen}}] } { ## Grab the first 2 bytes to determine the extension type binary scan ${payload} @${field_offset}S ext set ext [expr {$ext & 0xFFFF}] ## Store the extension in the extensions_list variable lappend extensions_list ${ext} ## Increment field_offset past the 2 bytes of the extension type set field_offset [expr {${field_offset} + 2}] ## Grab the 2 bytes of extension lenth binary scan ${payload} @${field_offset}S ext_len ## Increment field_offset past the 2 bytes of the extension length set field_offset [expr {${field_offset} + 2}] ## Look for specific extension types in case these need to increment the field_offset (and because we need their values) switch $ext { "11" { ## ec_point_format - there's another 1 byte after length ## Grab the extension data binary scan ${payload} @[expr {${field_offset} + 1}]s ext_data set ec_point_format ${ext_data} } "10" { ## elliptic_curves - there's another 2 bytes after length ## Grab the extension data binary scan ${payload} @[expr {${field_offset} + 2}]S[expr {(${ext_len} - 2) / 2}] ext_data set elliptic_curves ${ext_data} } default { ## Grab the otherwise unknown extension data binary scan ${payload} @${field_offset}H[expr {${ext_len} * 2}] ext_data } } ## Increment the field_offset past the extension data length. Repeat this loop until we reach rlen (the end of the payload) set field_offset [expr {${field_offset} + ${ext_len}}] } } ## Now let's compile all of that data. ## The cipherlist values need masking with 0xFFFF to return the unsigned integers we need foreach cipher $cipherlist_decimal { lappend cipd [expr {$cipher & 0xFFFF}] } set cipd_str [join $cipd "-"] if { ( [info exists extensions_list] ) and ( ${extensions_list} ne "" ) } { set exte [join ${extensions_list} "-"] } else { set exte "" } if { ( [info exists elliptic_curves] ) and ( ${elliptic_curves} ne "" ) } { set ecur [join ${elliptic_curves} "-"] } else { set ecur "" } if { ( [info exists ec_point_format] ) and ( ${ec_point_format} ne "" ) } { set ecfp [join ${ec_point_format} "-"] } else { set ecfp "" } set ja3_str "${sslversion},${cipd_str},${exte},${ecur},${ecfp}" ## binary scan [md5 ${ja3_str}] H* ja3_digest ## Un-comment this line to display the fingerprint string in the LTM log for troubleshooting #log local0. "ja3 = ${ja3_str}" return ${ja3_str} } Exactly as in Kevin’s rules, the PROC takes the full TCP payload as input along with the record length and the “inner” TLS version. The PROC then walks the payload extracting the values we need to calculate the hash before calculating and passing the hash back to the calling iRule. Create the caller iRule Now there’s just one thing left to do, again just as with Kevin’s rules. This iRule has enough code to detect and collect the entire TLS ClientHello and then send the fingerprint payload off to the PROC. This is really just an example – if you’re using this then you want to put your logic, or code to ship the hash off to a logger, between the “Do Something here” lines. when CLIENT_ACCEPTED { ## Collect the TCP payload TCP::collect } when CLIENT_DATA { ## Get the TLS packet type and versions if { ! [info exists rlen] } { ## We actually only need the recort type (rtype), record length (rlen) handshake type (hs_type) and 'inner' SSL version (inner_sslver) here ## But it's easiest to parse them all out of the payload along with the bytes we don't need (outer_sslver & rilen) binary scan [TCP::payload] cSScH6S rtype outer_sslver rlen hs_type rilen inner_sslver if { ( ${rtype} == 22 ) and ( ${hs_type} == 1 ) } { ## This is a TLS ClientHello message (22 = TLS handshake, 1 = ClientHello) ## Call the fingerprintTLS proc set ja3_fingerprint [call Library-Rule::fingerprintTLS [TCP::payload] ${rlen} ${inner_sslver}] binary scan [md5 ${ja3_fingerprint}] H* ja3_digest ### Do Something here ### log local0. "[IP::client_addr]:[TCP::client_port] ja3 ${ja3_fingerprint}->${ja3_digest}" ### Do Something here ### } } # Collect the rest of the record if necessary if { [TCP::payload length] < $rlen } { TCP::collect $rlen } ## Release the paylaod TCP::release } Caveats and considerations The first thing to mention is that binary operations in iRules are very computationally expensive – to the extent that on a system under attack you might not be able to deploy something like this if the BIG-IP is already under significant stress. That said, we are most often in situations where the BIG-IP isn’t under stress but the back end applications are and in those situations, these rules can be very handy indeed. Because we aren’t comparing hashes against a pre-computed or pre-compiled list (as the work by Kevin & Lee does) there is no worry about having an incomplete hash table, but it does mean that you aren’t getting any 0th-second intelligence until you’ve had a chance to look at traffic volumes per hash and collect some information. Hopefully you find these rules useful and if you have any feedback let us know!
