Editing JSON HTTP Response

[Chris_Bates]

Hey guys!

I'm completely new to F5 and am currently trying to fix an issue that we are having with 'HTTP' to 'HTTPS' rewrites. 

There was another post that fixed somewhat the issue that we are facing linked here: Solved: iRULE regsub error - DevCentral

The iRule that this person used doesn't seem to be changing our HTTP response?...

Here is our current rewrite here: Current iRule rewrite - Pastebin.com 

If anyone could help that would be amazing!

Thanks 😀

[Kevin_Stewart]

You shouldn't need to do an HTTP::collect to handle response headers. Those can be seen and manipulated directly within the HTTP_RESPONSE event.

[Chris_Bates]

Okay, so would HTTP::collect be required in the HTTP_RESPONSE_DATA? or can they still be seen?

The response data stays the exact same:

{"result":{"pathTo3DJSON":"/PackWorks/tempfiles/DOC2865386461295932248/data.cbor"}}}}

with or without the iRule even though we are supposedly replacing the 4 closing brackets with 2...

[Kevin_Stewart]

I'm saying that if you need to change HTTP headers, you don't need to do an HTTP::collect and handle in HTTP_RESPONSE_DATA. The headers will be available in the HTTP_RESPONSE event. You would do an HTTP::collect if you're trying to replace text in the response payload (after the headers).

But I see maybe you're trying to replace headers AND payload. Again, headers are visible in HTTP_RESPONSE, so change those there. For the payload, without testing I'm assuming there's just something incorrect about your string replace logic. The following works for me to replace the ending "}}}}":

when HTTP_REQUEST {
    set doWrite 0
    if { [string tolower [HTTP::path]] eq "/website/generate3dview.wcr" } {
        set doWrite 1
    }
}
when HTTP_RESPONSE {
    if { ( [info exists doWrite] ) and ( $doWrite ) } {
        HTTP::collect [HTTP::header Content-Length]
        set clen [HTTP::header Content-Length]
        set doWrite 0
    }
}
when HTTP_RESPONSE_DATA {
    set payload [string map [list "\}\}\}\}" "\}\}"] [HTTP::payload]]
    HTTP::payload replace 0 $clen $payload
    HTTP::release
}

[Chris_Bates]

And to do the http to https and remove the port I can just do this?

when HTTP_RESPONSE_DATA {
    regsub "http://" [HTTP::payload] "https://" fixeddata
    regsub ":80" $fixedddata "" fixeddata1
    set payload [string map [list "\}\}\}\}" "\}\}"] [HTTP::payload]]  
    HTTP::payload replace 0 $clen $payload $fixeddata1 
    HTTP::release  
}

[Kevin_Stewart]

Is http:// and :80 in the payload or headers?

For example, you'd normally see this in Location headers (redirects) and in payload data referencing document objects. But if in payload, it's probably better to avoid regex and just do multiple string maps.

when HTTP_RESPONSE_DATA {
    set payload [string map [list "\}\}\}\}" "\}\}"] [HTTP::payload]]  
    set payload [string map [list "http://" "https://"] [HTTP::payload]]
    set payload [string map [list ":80" ""] [HTTP::payload]]
    HTTP::payload replace 0 $clen $payload $fixeddata1 
    HTTP::release  
}

What you're doing is grabbing the existing payload into a variable, making changes to that variable, and then completely replacing the HTTP::payload with the contents of that variable.

[Chris_Bates]

Just to confirm that this works for you to replace the brackets?

We've just implemented this change and it still doesn't want to replace them...

[Kevin_Stewart]

Try this:

when HTTP_RESPONSE_DATA {
    log local0. "payload before: [HTTP::payload]"
    set payload [string map [list "\}\}\}\}" "\}\}"] [HTTP::payload]]  
    set payload [string map [list "http://" "https://"] [HTTP::payload]]
    set payload [string map [list ":80" ""] [HTTP::payload]]
    log local0. "payload after: $payload"
    HTTP::payload replace 0 $clen $payload $fixeddata1 
    HTTP::release  
}

Tail the LTM log:

tail -f /var/log/ltm

And send a request. This should show you the payload before and after manipulation.

[Chris_Bates]

Okay... so this was interesting...

After checking the logs both the before and after payloads did not have the 4 brackets. So somewhere else these are being added...

[Kevin_Stewart]

Might there be another iRule and/or device in front of this handling HTTP? Or maybe some client side JavaScript that could be modifying the response?

[Chris_Bates]

Yeah, that's what I'm thinking... 

I've asked the networking team to take a look. Thanks for the help 🙂

[Leslie_Hubertus]

Now I'm curious - did they figure out the issue, @Chris_Bates?

[Chris_Bates]

Not as of yet. The networking team are adamant that there are no other rules that could affect this despite the logs showing that it's the most likely cause. 

We are arranging a meeting this week with F5 to get to the bottom of it hopefully...