Forum Discussion

khawasli_50856's avatar
khawasli_50856
Icon for Nimbostratus rankNimbostratus
Mar 09, 2011

Problem with charset and rewriting HTML content

Hi,

 

I have been facing an issue where Arabic characters come out all funny (corrupted basically) after rewriting some URIs and javascript events in my HTML (not using a streaming profile). I have seen this issue reported in other posts but no solution yet.... so I was wondering if someone has been able to resolve this issue. Please note that when using a streaming profile, this problem does not occur but my regular expression is a bit complex where it is not supported by the streaming profile (positive lookaheads and case insensitive statements) so I have to use the usual HTTP::collect and rewrite with a regular expression.

 

 

I am using BIGIP LTM 10.2

 

 

You help is appreciated

 

Thank you,

 

  • Hi,

    I'd go back to using a stream profile and STREAM::expression iRule. While you can't use positive lookaheads, you can modify your stream expression to match more of the source string than you need to replace and then inspect the match and customize the replacement in the STREAM_MATCHED event using STREAM::match and STREAM::replace. Here are the related wiki pages for the commands. You can check the STREAM::expression page for an example of the rewriting in STREAM_MATCH:

    http://devcentral.f5.com/wiki/default.aspx/iRules/stream

    http://devcentral.f5.com/wiki/default.aspx/iRules/stream__expression

    This example shows how you can use STREAM::match in the STREAM_MATCHED event to check if the matched string meets some condition that can't easily be checked for using a single regex in STREAM::expression.
    
    when HTTP_REQUEST {
        Disable the stream filter for all requests
       STREAM::disable
    }
    when HTTP_RESPONSE {  
    
        Check if response type is text  
       if {[HTTP::header value Content-Type] contains "text"}{  
    
           Match an http://*example.com string and replace it with nothing yet
          STREAM::expression {&http://.*?example\.com&&}
    
           Enable the stream filter for this response only  
          STREAM::enable  
       }  
    }   
    when STREAM_MATCHED {  
    
        Check if the matched string meets some condition that can't easily be checked for using a single regex in STREAM::expression
       if {[STREAM::match] starts_with "host1"}{
    
           Replace http:// with https:// and do the replacement
          STREAM::replace "[string map {http:// https://} [STREAM::match]]"
          log local0. "[IP::client_addr]:[TCP::local_port]: matched: [STREAM::match], replaced with: [string map {http:// https://} [STREAM::match]]"  
       }
    }
    

    Aaron
  • Hi hoolio

     

    Thank you for the reply.... the streammatch event is a great idea thank you very much fior this great tip..... ut the issue I am facing is more generic, I have faed more than one issue due to the streaming profile limitation with regex.... one example is that I once has a string that looked something like

     

    Var somevariable. = "tessting"

     

     

    If I want to rewriteit with

     

    /var[ ]{5,}[a-z]+[ ]{5,}=[ ]+"[a-zA-Z]"/var somevariable = "testing"

     

    This will not match because the string I am trying to match is too long that the stream would not detect such length (I am probably wrong regarding this assumption but I have faced this more than once where I cannot match long strings where the same regex works with buffering technique)

     

     

    Another simple example is the case insensative :) a simple and silly as it may sound but it is such a headache that I cannottell regex to match without case sensativity.

     

     

    I think that the good old buffering techniqu must not be replaaced with streaming streaming is good in certain situations but in complex cases (like writing uri exnryption) we have to use bufferin what do you think hoolio?
  • By the way the code example I show above did not show the way I wrote it... I actually addded a lot of spaces between the words
  • Can you put your post in [ code ] [/ code ] blocks (without the spaces) to preserve the characters?

     

     

    Thanks, Aaron
  • Sorry about that ....... but pleae do not take my example litterly,what I was trygin to say that generally speking buffering cannot match long strings... the example I faced as far as I remember (I might be showing some mistakes) is rewriting this

    var.                   Varname.     =                   "somevalue" 

    And rewriting it to

    var varname="somevalue"

    But like I said before in complex irule like an irule that is use for identifying and encrypting uris treaming is not effecient.... so my issue is not specific to those example it is more general

    Thank you hoolio
  • Sorry about that ....... but pleae do not take my example litterly,what I was trygin to say that generally speking buffering cannot match long strings... the example I faced as far as I remember (I might be showing some mistakes) is rewriting this

    var.                   Varname.     =                   "somevalue" 

    And rewriting it to

    var varname="somevalue"

    But like I said before in complex irule like an irule that is use for identifying and encrypting uris treaming is not effecient.... so my issue is not specific to those example it is more general

    Thank you hoolio