Forum Discussion

JEFF029's avatar
JEFF029
Icon for Nimbostratus rankNimbostratus
Jun 17, 2016

[HTTP::payload] UTF-8 encoding issue

Hi,

I have an issue with [HTTP::collect]/[HTTP::payload] in my Irule.

I want to collect HTTP payload from client POST, then send it on another virtual server :

            set Contenu [HTTP::payload]
    set conn_id [connect -timeout 100 -idle 30 -status conn_status DESIRP4_TM01_J_i01]
        set entete "POST /MetierTM01ws HTTP/1.1\r\nContent-Type: application/soap+xml;charset=UTF-8\r\nContent-Length: $content_length\r\nHost: X.X.X.X\r\nConnection: Keep-Alive\r\nUser-Agent: Apache-HttpClient/4.1.1 (java 1.5)\r\n\n"
        set Fin "\r"
        set Soapenvelope $Contenu$Fin
        unset Fin
        set data $entete$Soapenvelope
                    log local0. "04 Lancement connexion avec data : $data "
        set send_bytes [send -timeout 1000 -status send_status $conn_id $data]

The client request is encoded in UTF-8, but it seems when I collect http payload, the F5 considers the content is in ascii and re-encodes it in UTF-8. For exemple the é (e acute) character (it is c3+a9 in UTF-8), after processing the character becomes a four-byte (c3+83+c2+a9).

Has anyone ever implemented this? If so could you share your iRule?

Many thanks,

JEFF

5 Replies

  • Bonjour Jeff, have a look to wiki on HTTP::payload there is a binary scan example for the replace option, maybe it can give you idea to adapt. if the content is then handled to another VS, i would perhaps encode base64 the data pass it , and decode it on the second VS.

     

  • I am not sure if this is what you are seeking but I am hoping this may help you. I had to work on encoding issue with non-English characters as noted in this iRule example.

     

  • I think nobody.

     

    The main issue there is not defined charset of payload stream. They get bytes and convert it into string byte by byte into UTF. This byte array is allready encoded as UTF and then you reencode special byte as character from ASCII. C5 -> C3 85 C2 (UTF encoding of char Å)

     

  • I have used the same solution (by Mamut 78) to reencode my [HTTP::Payload] from UTF-8 to ISO-8889-1. It's working fine, but it's a bit time consuming (CPU Cycles). I was searching for another solution using STREAM Profiles. The first idea seems good, but the problem comes when I try to substitute UTF-8 characters into ISO-8889-1 characters.

    In UTF-8, lowcase acute A is encoded using "C3A1". In ISO-8889-1 just a single byte is needed ("E1"). I was planning an replacement expression like:

    STREAM::expression "@\u00c3\u00a1@\u00e1@"
    

    But this is not working at all :-(. F5 reads ISO-8889-1 \u00e1 directly as \u00c3\u00a1, so no replacement was taking place.

    I have tried to use enconding method, but I don't know where it should be placed...

    STREAM::encoding [ascii | utf-8]
    

    Anyone has experience using Stream Profiles? Any help?

    Also, I would like to catch payload after replacement, to retry query --> "HTTP::retry [HTTP::request][HTTP::payload]" Initially payload is wrong-formed, so I can't take it during HTTP_REQUEST event, and after that it seems all variables where released (Here in HTTP_REQUEST_SEND is not working).

    Any idea how could i take it?

    KR, Dario.

  • Hi Dario, please can you tell me how to gather the whole HTTP payload during HTTP_REQUEST_SEND event? Thanks very much Vaga