Forum Discussion

JaZy's avatar
JaZy
Icon for Altostratus rankAltostratus
Sep 27, 2024

Keep encoding when request is handled by irule

I have custom irule to check data in payload. I noticed issue when non-latin characters are detected, for example Japan characters.

In this case seems that byte array is not read correctly (2 bytes vs 4 bytes) what just corrupt/change payload. I am almost sure that I hit follow: https://my.f5.com/manage/s/article/K22406348.

There is mentioned option to use binary scan but does not work for me. In generally I do not add anything to payload or modify it. I just compare some properties and block request if it match condition.

How I can keep origin payload as it is?

devops
security
  • Lucas_Thompson's avatar
    Lucas_Thompson
    Icon for Employee rankEmployee
    Sep 30, 2024

    Can you pare down your irule to the simplest operation that causes the problem and post it, along with a curl command to reproduce the results?  Chrome Browser Dev Tools now allows you to right-click and network request and select "Copy as cURL" to make this very easy.

     

    Dealing with encoding issues and iRules can sometimes be troublesome. Additionally, the behavior can change with dynamic conditions like response chunking and compression. The best way to handle these is with a good reproducible test case.

    • JaZy's avatar
      JaZy
      Icon for Altostratus rankAltostratus
      Oct 01, 2024

      During troubleshooting I did simples possible condition to replicate issue also for other endpoint. As I received report for specific endpoint where custom irule was written I covered another endpoint to replicate issue. And it's behave same. Below is part of irule.

      when HTTP_REQUEST {
catch {
    		    if { [HTTP::uri] starts_with "/api/test/endpoint" } {
			        log local0. "DEBUG: Edit URL detected"
			        set UrlEdit 1
    		    }
            }

    		if { [HTTP::header value Content-Length] > 0 } then {
    		    log local0. "DEBUG: myapp - collecting data for edit URL"
    		    HTTP::collect [HTTP::header Content-Length]
    		}
}
when HTTP_REQUEST_DATA {

  catch {
    if { $UrlEdit } {  
        #set editpayload [HTTP::payload]
        #binary scan $editpayload [HTTP::payload]
        #HTTP::payload replace 0 [HTTP::header "Content-Length"] $editpayload
        log local0. "DEBUG: myapp: edit page detected with payload: $editpayload"
        HTTP::release
    }
  }
}

      Part of the payload is:

      --data-raw '{"customer":{"name":"漢字__字汉__"

      There is no compression. Only small payload is forwarded.

      At the end in application is string with Chinese characters represent as 漢字字汉

      If we take first character 漢 in UTF-8 it is 0xE6 0xBC 0xA2 what is converted back as E6 (æ) + BC (¼) +  A2(¢). In summary 漢 = æ¼¢.

       

      Adding also part of the log from ltm. Characters were already converted:

      {"customer":{"name":"漢�__��__

  • JaZy's avatar
    JaZy
    Icon for Altostratus rankAltostratus
    Oct 16, 2024

    Is there any way how to avoid this and keep origin string at the end? I tried to add binary scan to more places but result is still same:

      catch {
    if { $UrlEdit } {
        set editpayload [HTTP::payload]
        #binary scan $editpayload [HTTP::payload]
        #HTTP::payload replace 0 [HTTP::header "Content-Length"] $editpayload
        log local0. "DEBUG: myapp: edit page detected with payload: $editpayload"
        binary scan c* $editpayload 
        binary scan c* throwawayvariable
        binary scan $editpayload @0
        HTTP::payload replace 0 $content_length $editpayload
        log local0. "DEBUG: myapp: payload after binary scan: $editpayload"
        HTTP::release
    }
  }
}