Forum Discussion

Kevin_Basler's avatar
Kevin_Basler
Icon for Altostratus rankAltostratus
May 19, 2022

STREAM::expression doesn't seem to work

We've created an iRule that is supposed to change any occurance of the host internal.com to external.com in the payload of the response. We've added some logging steps to confirm that the STREAM::exp...
application delivery
  • Kevin_Basler's avatar
    Kevin_Basler
    May 24, 2022

    So this is the solution that we arrived at, thanks to you and your suggestion of using HTTP::header remove "Accept-Encoding". I've removed it and it fails just like before. Add it back in and everythign is working.

    when HTTP_REQUEST {
    HTTP::header remove "Accept-Encoding"
    }

    when HTTP_RESPONSE {
    if { [HTTP::header exists "Location"] }{
    set internal_host "https://internal.com"
    set external_host "https://external.com"
    if { [HTTP::header "Location"] contains $internal_host } {
    HTTP::header replace "Location" [string map "$internal_host $external_host" [HTTP::header "Location"]]
    }
    }
    STREAM::disable
    if { [HTTP::payload] contains "internal.com"} {
    log local0. "Before if, payload contains : $[HTTP::payload]"
    set oldString "internal.com"
    set newString "external.com"
    STREAM::expression @$oldString@$newString@@
    STREAM::enable
    log local0. "Before if, payload contains : $[HTTP::payload]"
    }
    }

    Something that is interesting is that I'm seeing readable payload both before and after using HTTP::header remove "Accept-Encoding" in the HTTP_REQUEST section. I can't explain it, but the important part is we have a working solution.

    Thanks for all of your help, Alex! Really appreciate it!

     

  • AlexBCT's avatar
    AlexBCT
    May 26, 2022

    Hi Kevin,

    Great to hear! Indeed interesting as to why this didn't work beforehand, but at least it's working now. Also thanks for posting to solution, I'm sure someone else will come across it and be happy with it as well. 😉 

AlexBCT's avatar
AlexBCT
Icon for Cumulonimbus rankCumulonimbus

Hi Kevin, 

Do you have a compression profile attached to the virtual server? Most web servers will compress HTTP responses by default, which means that when the STREAM profile/irule looks through the return payload, it tries to match the compressed content, which doesn't contain the strings internal.com. 

If you don't have it on the virtual servers, two things you can try; 

1. attach a HTTP compression profile on your virtual server, this will offload HTTP compression onto the F5 (and saving a bit of CPU on the web server as well), or; 

2. Using an iRule or Local Traffic Policy, remove the Accept-Encoding header from the request before sending it through to the web server. Be aware though that this will now serve up uncompressed pages to the client and may increase bandwidth usage. 

I've done a quick test on my lab system and this does make your iRule work. 

Hope this helps. 

Edit: Here's some more information about the impact of compression on the stream profile: https://support.f5.com/csp/article/K39394712

Kevin_Basler's avatar
Kevin_Basler
Icon for Altostratus rankAltostratus
May 23, 2022

Hey Alex, appreciate the quick reply. I'm just trying to understand this. 

If compression was the issue, would I still see the value of HTTP::payload both before and after the STEAM::expression? As I mentioned before, I'm currently seeing the value before and after the stream::expression has run. Unfortunately, it doesn't seem to be changed.

Thanks,

Kevin

  • AlexBCT's avatar
    AlexBCT
    Icon for Cumulonimbus rankCumulonimbus
    May 24, 2022

    Well, you would still see a payload, but wouldn't make any sense. (garbled text) If you did see readable text, then compression indeed wouldn't be the issue. 

    Can you tell some more about the payload that you are looking into? E.g. Is it HTML/JSON/JS/etc. and the amount of data per response? And just to confirm, the text "internal.com" is indeed in the payload of the HTTP response rather than the headers? 

     

     

    • Kevin_Basler's avatar
      Kevin_Basler
      Icon for Altostratus rankAltostratus
      May 24, 2022

      So this is the solution that we arrived at, thanks to you and your suggestion of using HTTP::header remove "Accept-Encoding". I've removed it and it fails just like before. Add it back in and everythign is working.

      when HTTP_REQUEST {
      HTTP::header remove "Accept-Encoding"
      }

      when HTTP_RESPONSE {
      if { [HTTP::header exists "Location"] }{
      set internal_host "https://internal.com"
      set external_host "https://external.com"
      if { [HTTP::header "Location"] contains $internal_host } {
      HTTP::header replace "Location" [string map "$internal_host $external_host" [HTTP::header "Location"]]
      }
      }
      STREAM::disable
      if { [HTTP::payload] contains "internal.com"} {
      log local0. "Before if, payload contains : $[HTTP::payload]"
      set oldString "internal.com"
      set newString "external.com"
      STREAM::expression @$oldString@$newString@@
      STREAM::enable
      log local0. "Before if, payload contains : $[HTTP::payload]"
      }
      }

      Something that is interesting is that I'm seeing readable payload both before and after using HTTP::header remove "Accept-Encoding" in the HTTP_REQUEST section. I can't explain it, but the important part is we have a working solution.

      Thanks for all of your help, Alex! Really appreciate it!

       

      • AlexBCT's avatar
        AlexBCT
        Icon for Cumulonimbus rankCumulonimbus
        May 26, 2022

        Hi Kevin,

        Great to hear! Indeed interesting as to why this didn't work beforehand, but at least it's working now. Also thanks for posting to solution, I'm sure someone else will come across it and be happy with it as well. 😉