Forum Discussion

Jeff_Goot_42327's avatar
Jeff_Goot_42327
Icon for Nimbostratus rankNimbostratus
Apr 26, 2012

STREAM::expression Not Replacing All Occurrences

I'm trying to track down an issue with an iRule that uses STREAM::expression. It's able to replace the occurrences of the string, but not all of the occurrences.

 

 

Here's the iRule:

 

 

 

when HTTP_REQUEST { ... }

 

 

 

when HTTP_RESPONSE {

 

STREAM::disable

 

if { [HTTP::header Content-Type] starts_with "text/" }{

 

STREAM::expression "@https://somedomain.com@@ @http://anotehrdomain.com@@"

 

STREAM::enable }

 

}

 

 

 

In my test that has 93 occurrences of https://somedomain.com in the response stream, 91 occurrences were replaced, but two were not. They're not the same occurrences all the time. Sometimes only 1 isn't replaced, sometimes 3. I've tried using regexp but same result.

 

 

 

I'd appreciate any information.

 

 

 

Thanks, Jeff

 

  • Hi Jeff,

     

     

    Can you post an anonymized copy of the original and modified responses (including the response headers)? You can try inlining the responses here in [ code ] [/ code ] blocks, attaching them as files to your post or uploading them to a site like pastebin.

     

     

    You might also try opening a case with F5 Support to get help troubleshooting this.

     

     

    Also, it shouldn't matter much, but you could make the find string in the STREAM::expression a little more precise by escaping the periods:

     

     

    STREAM::expression "@https://somedomain\.com@@ @http://anotehrdomain\.com@@"

     

     

    Aaron
  • I appear to be having exactly the same issue.

     

     

    Jeff: Did you find a solution?
  • I seem to be experiencing the same issue. Has anyone identified a solution? Here's a copy of my iRule. I can also provide the original and modified responses if need be.

    
    when HTTP_REQUEST {
    
        Explicitly disable the stream profile for each request so it doesn't stay 
        enabled for subsequent HTTP requests on the same TCP connection.
       STREAM::disable
    }
    when HTTP_RESPONSE {
    
        Apply stream profile against text or javascript responses from the application
       if { [HTTP::header value Content-Type] contains "text" || [HTTP::header value Content-Type] contains "javascript" }{
       
           Replace /webcenter/spaces/MortgageCenter/page/ with /MortgageCenter/page/
           Replace /webcenter/oracle/webcenter/siteresources/scopedMD/ with /MortgageCenter/siteresources/scopedMD/
           Replace /webcenter/faces/oracle/webcenter/page/scopedMD/ with /MortgageCenter/page/scopedMD/
           Replace /webcenter/faces/ with /MortgageCenter/faces/
           Replace /webcenter/adf/ with /MortgageCenter/adf/
           Replace /webcenter/afr/with /MortgageCenter/afr/
           Replace /webcenter/sessioninvalidation with /MortgageCenter/sessioninvalidation
           Replace /webcenter/oracle/webcenter/webcenterapp/view/templates/publichtml/ with /MortgageCenter/publichtml/
          STREAM::expression {@/webcenter/spaces/MortgageCenter/page/@/MortgageCenter/page/@ @/webcenter/oracle/webcenter/siteresources/scopedMD/@/MortgageCenter/siteresources/scopedMD/@ @/webcenter/faces/oracle/webcenter/page/scopedMD/@/MortgageCenter/page/scopedMD/@ @/webcenter/faces/@/MortgageCenter/faces/@ @/webcenter/adf/@/MortgageCenter/adf/@ @/webcenter/afr/@/MortgageCenter/afr/@ @/webcenter/sessioninvalidation@/MortgageCenter/sessioninvalidation@ @/webcenter/oracle/webcenter/webcenterapp/view/templates/publichtml/@/MortgageCenter/publichtml/@}
     
           Enable the stream profile
          STREAM::enable
       }
    
        If the response contains the Set-Cookie header, rewrite the context root
       if { [HTTP::header value Set-Cookie] contains "/webcenter" }{
          HTTP::header replace Set-Cookie [string map "/webcenter /" [HTTP::header value Set-Cookie]]
       }
    
        If the response is a redirect rewrite the Location header.
        Replace /webcenter/spaces/MortgageCenter/page/ with /MortgageCenter/page/
        Replace /webcenter/oracle/webcenter/siteresources/scopedMD/ with /MortgageCenter/siteresources/scopedMD/
        Replace /webcenter/faces/oracle/webcenter/page/scopedMD/ with /MortgageCenter/page/scopedMD/
        Replace /webcenter/faces/ with /MortgageCenter/faces/
        Replace /webcenter/adf/ with /MortgageCenter/adf/
        Replace /webcenter/afr/ with /MortgageCenter/afr/
        Replace /webcenter/sessioninvalidation with /MortgageCenter/sessioninvalidation
        Replace /webcenter/oracle/webcenter/webcenterapp/view/templates/publichtml/ with /MortgageCenter/publichtml/
       if { [HTTP::is_redirect] }{
         HTTP::header replace Location [string map "/webcenter/spaces/MortgageCenter/page/ /MortgageCenter/page/ /webcenter/oracle/webcenter/siteresources/scopedMD/ /MortgageCenter/siteresources/scopedMD/ /webcenter/faces/oracle/webcenter/page/scopedMD/ /MortgageCenter/page/scopedMD/ /webcenter/faces/ /MortgageCenter/faces/ /webcenter/adf/ /MortgageCenter/adf/ /webcenter/afr/ /MortgageCenter/afr/ /webcenter/sessioninvalidation /MortgageCenter/sessioninvalidation /webcenter/oracle/webcenter/webcenterapp/view/templates/publichtml/ /MortgageCenter/publichtml/" [HTTP::header value Location]] 
       }
    }
    
  • Hi Ken,

    Is the server sending a chunked or compressed response? Those are two reasons that the stream filter might not match (assuming the stream filter is set correctly).

    You can prevent chunking by setting the proxied request to HTTP 1.0 in HTTP_REQUEST:

    https://devcentral.f5.com/wiki/iRules.CreditCardScrubberUsingStream.ashx

    
           Don't allow response data to be chunked
          if { [HTTP::version] eq "1.1" } {
              if { [HTTP::header is_keepalive] } {
                 HTTP::header replace "Connection" "Keep-Alive"
              }
              HTTP::version "1.0"
           }
    

    And to prevent compression, you can remove the Accept-Encoding header from the proxied request in HTTP_REQUEST.

    HTTP::header remove Accept-Encoding

    Aaron
  • Thanks Aaron. Yes, the server is sending a chunked response, but there is no compression enabled. I'm using a modified HTTP Profile that uses the default HTTP Profile as it's parent, the only change in the new profile was to set Response Chunking to rechunk. If I use your suggestion to prevent chunking via the iRule with the new HTTP Profile the page load doesn't complete. If I disable chunking via the iRule and use the default HTTP Profile the page load completes but I'm still experiencing the issue where not all occurrences are bring replaced.
  • Can you set the HTTP version to 1.0 in the iRule and use a custom HTTP profile with response chunking set to rechunk?

     

     

    If that doesn't work, please open a case with F5 Support referencing this thread.

     

     

    Thanks, Aaron