For more information regarding the security incident at F5, the actions we are taking to address it, and our ongoing efforts to protect our customers, click here.

Forum Discussion

beefy80's avatar
beefy80
Icon for Nimbostratus rankNimbostratus
Apr 03, 2014

Calculating a two byte header in message payload

Hello I am currently writing a logging event to capture a message but I am finding that by enabling the TCP::collect event the processing is being slowed down by as much as 2 seconds. As the messages are variable length we have a two byte prefix in front of the message that gives the payload length minus the two byte header e.g. <0x00 0x04> <\x02 12 \x03> I want to try capturing the prefix and calculate the message length from this and then apply a check to see once the message length is reached call TCP::release to try and speed this process up. I am struggling to calculate the two byte header and hoping someone may be able to assist me. I have been dabbling with Binary Scan but cannot get the correct result.

 

Thank you James

 

9 Replies

  • John_Alam_45640's avatar
    John_Alam_45640
    Historic F5 Account

    James

    I would start by collecting only 2 bytes. Convert those from hex to decimal and then collect that number of bytes before you release.

    when CLIENT_ACCEPTED {
       TCP::collect 2
       set header_collected "false"
    }
    
    when CLIENT_DATA {
    
       if { not ($header_collected) } {
           Comment out one of the two binary scan lines below.
           use this form if lower order byte is last (big endian order)
          binary scan [TCP::payload] S message_length
           use this form if lower order byte is first (little endian order)
          binary scan [TCP::payload] s message_length
    
          if { $message_length > 0 } {
             set header_collected "true"
             log local0. "Collecting message with length $message_length"
             TCP::collect $message_length
           }
        } else {
           if we are here the actual message is collected.
          set actual_message_text [TCP::payload]
          TCP::release
        }
     }
    

    Feel free to post what you have and we take a look.

    Examples of binary scan:

    % binary scan \x02\x00 s var1
    1
    % puts $var1
    2
    % binary scan \x00\x03 S var1
    1
    % puts $var1
    3
    %
    

    http://tmml.sourceforge.net/doc/tcl/binary.html

    HTH.
    • beefy80's avatar
      beefy80
      Icon for Nimbostratus rankNimbostratus
      Thank you for this John. I am going to look at this further and will post back how I get on.
    • beefy80's avatar
      beefy80
      Icon for Nimbostratus rankNimbostratus
      Please see enhancement to the code here below.
  • John

     

    I had to enhance your code slightly as the payload messages are quite small and therefore client_data event was not getting triggered again as the LTM has already buffered the rest of the data. I have therefore now added a check to see if we already have the data or not and if we do TCP::release. I cannot see a more affiant way of doing this, what do you think?

     

    when CLIENT_ACCEPTED {
        TCP::collect 2
        set header_collected "false"
        }
    }
    when CLIENT_DATA {
    
        if { not ($header_collected) } {
         Comment out one of the two binary scan lines below.
         use this form if lower order byte is last (big endian order)
        binary scan [TCP::payload] S message_length
         use this form if lower order byte is first (little endian order)
         binary scan [TCP::payload] s message_length
    
         as the message length collected does not include itself add 2 more onto the result
        set message_length [expr { $message_length + 2 }]
    
        if { $message_length > 0 } {
            set header_collected "true"
             collect data with the required length
            TCP::collect $message_length
             check if we already have the required data in the buffer if so 
             capture the payload in vars and release it
            if {[TCP::payload length] == $message_length } {
                 set vars with payload info for HSL logging later
                set my_req_msg_payload [TCP::payload]
                set my_req_msg_length [TCP::payload length]
                 release the data already collected
                TCP::release
            }
           }
        } else {
           if we are here the actual message is collected.
            set my_req_msg_payload [TCP::payload]
            set my_req_msg_length [TCP::payload length]
            TCP::release
        }
    }
  • John_Alam_45640's avatar
    John_Alam_45640
    Historic F5 Account

    Good catch.

    Your modification look great. Only a couple suggestions:

    1) don't add the 2 to the message_length variable because it is used for the TCP::collect later, instead add it within the comparison operation.

    2) Perform the second the TCP::collect in the else statement so that it can be avoided if not needed.

    Here is what it would look like:

    when CLIENT_ACCEPTED {
        TCP::collect 2
        set header_collected "false"
        }
    }
    when CLIENT_DATA {
    
        if { not ($header_collected) } {
         Comment out one of the two binary scan lines below.
         use this form if lower order byte is last (big endian order)
        binary scan [TCP::payload] S message_length
         use this form if lower order byte is first (little endian order)
         binary scan [TCP::payload] s message_length
    
         as the message length collected does not include itself add 2 more onto the result
        
    
        if { $message_length > 0 } {
            set header_collected "true"
    
             check if we already have the required data in the buffer if so 
             capture the payload in vars and release it
            if {[TCP::payload length] == [expr { $message_length + 2 }] } {
                 set vars with payload info for HSL logging later
                set my_req_msg_payload [TCP::payload]
                set my_req_msg_length [TCP::payload length]
                 release the data already collected
                TCP::release
            } else {
                 collect data with the required length
                TCP::collect $message_length            
            }
           }
        } else {
           if we are here the actual message is collected.
            set my_req_msg_payload [TCP::payload]
            set my_req_msg_length [TCP::payload length]
            TCP::release
        }
    }
    
    • beefy80's avatar
      beefy80
      Icon for Nimbostratus rankNimbostratus
      John, I have just tried your enhancement but this does not work for me until our software sends the request again then it works. I have reverted to the previous version of code that I included. I like the code on 11.4 not aware of the procedures function however I am on 11.3 at the moment but will keep that in mind for the future.
  • John_Alam_45640's avatar
    John_Alam_45640
    Historic F5 Account

    if you are on 11.4 or above, here is a slight variation in order to demonstrate iRule procedures "proc".

    proc save_n_let_it_go args {
             the args variable is ignored here 
            set my_req_msg_payload [TCP::payload]
            set my_req_msg_length [TCP::payload length]
             release the data already collected to be forwarded to destination.
            TCP::release
    } 
    
    when CLIENT_ACCEPTED {
        TCP::collect 2
        set header_collected "false"
        }
    }
    when CLIENT_DATA {
    
        if { not ($header_collected) } {
         Comment out one of the two binary scan lines below.
         use this form if lower order byte is last (big endian order)
        binary scan [TCP::payload] S message_length
         use this form if lower order byte is first (little endian order)
         binary scan [TCP::payload] s message_length
    
         as the message length collected does not include itself add 2 more onto the result
        
    
        if { $message_length > 0 } {
            set header_collected "true"
    
             check if we already have the required data in the buffer if so 
             capture the payload in vars and release it
            if {[TCP::payload length] == [expr { $message_length + 2 }] } {
                 set vars with payload info for HSL logging later
                call save_n_let_it_go
            } else {
                 collect data with the required length
                TCP::collect $message_length            
            }
           }
        } else {
           if we are here the actual message is collected.
          call save_n_let_it_go
        }
    }