Now my problem is there are around 1200 URIs or paths which need to be translated from one One URL+URI to another URL+URI where the catch is URL+URI must not change in user's browser.
What would be correct way to do it? With proflies or with iRule.
In profile I saw that there is an option where I can create a rule where it states is any URL+URI matches it can be send done with HTTP_Header_replace and then I can add one more rule with HTTP_URI_Replace.
I'd maybe argue that partitions aren't create separate logical BIG-IPs. It's more about building separate administrative domains on a single BIG-IP. Further, partitions can always access the /Common partition. To answer your question though, it depends on the level of isolation, but you should simply be able to point to the VIP in the oter partition,
virtual "/abc-app-dev/np-api-vip"
or
virtual "/Common/abc-app-dev/np-api-vip"
You don't need SNAT to communicate between VIPs on the same BIG-IP.
Thanks for your guidance. Here is what I figured out and it is working now. All happened due to your idea so thanks again.
IRule for same routing domain :-
when HTTP_REQUEST { log local0. "Incoming request: [HTTP::host]:[HTTP::uri]" if { [set match [class match -value -- "[HTTP::host]:[HTTP::uri]" starts_with datagroup_Name]] ne "" } { log local0. "Datagroup match: $match" ## Get first URI path in client request set FIRSTPATH [getfield [HTTP::path] "/" 2]
## Split match into host and path set matchlist [split $match ":"] set MATCHHOST [lindex $matchlist 0] set MATCHPATH [lindex $matchlist 1]
## Replace FIRSTPATH with MATCHPATH (from full HTTP::uri) set UPDATEDPATH [string map [list "/$FIRSTPATH" "$MATCHPATH"] [HTTP::uri]] log local0. "Updated host: $MATCHHOST" log local0. "Updated path: $UPDATEDPATH"
HTTP::host $MATCHHOST HTTP::uri $UPDATEDPATH
virtual "/path/virtual_server_name"
} }
For different route domain
when HTTP_REQUEST { log local0. "Incoming request: [HTTP::host]:[HTTP::uri]" if { [set match [class match -value -- "[HTTP::host]:[HTTP::uri]" starts_with datagroup_name]] ne "" } { log local0. "Datagroup match: $match" ## Get first URI path in client request set FIRSTPATH [getfield [HTTP::path] "/" 2]
## Split match into host and path set matchlist [split $match ":"] set MATCHHOST [lindex $matchlist 0] set MATCHPATH [lindex $matchlist 1]
## Replace FIRSTPATH with MATCHPATH (from full HTTP::uri) set UPDATEDPATH [string map [list "/$FIRSTPATH" "$MATCHPATH"] [HTTP::uri]] log local0. "Updated host: $MATCHHOST" log local0. "Updated path: $UPDATEDPATH"
In my opinion, with such a large amount of items you should use an iRule to perform this task, as it will be more beneficial for scalability and easier to manage.
How much will the URI paths change? Make sure to use [HTTP::uri] , [HTTP::path], [HTTP::query] parameters properly in order to isolate the parts of your old URL that you want to change/keep. For example you could save the full querystring in a variable so that you can add it back to the rewritten url.
If you have a large list of paths that all need to be rewritten on a single new path, you can also use data groups for even better scalability and maintenance.
This might give you a couple ideas
when HTTP_REQUEST {
set uri [HTTP::uri]; set path [HTTP::path]; set query [HTTP::query]
HTTP::header replace Host "pqr.com"
if { $path starts_with "/xyz" } {
set newpath [string map -nocase {"/xyz" "/mnp"} $path]
HTTP::uri "$newpath$query"
}
}
How to incoroprate the different paths with the use of datagroup in the iRule provided above. Also when the response would come back doesn't it need to be changed to original URL+URI/path as well?
I've tried to modify my syntax, haven't had the chance to test it in a lab so please be careful with this and don't run it in prod without testing.
Code below is supposed to replace anything that matches /aaa/ /bbb/ or /ccc/ at depth1 in your path (this is driven by "starts_with" condition) with static /mnp/
so /aaa/path/to/index.jsp will change into /mnp/path/to/index.jsp/
and i believe /bbb/bbb/file.txt should change to /mnp/bbb/file.txt
Please also integrate knowledge shared by @Kevin_Stewart in his comments which is very detailed and much more reliable than my attempt 🙂
Lastly, I don't think you will need to change anything in HTTP response.
when HTTP_REQUEST {
set uri [HTTP::uri]; set path [HTTP::path]; set query [HTTP::query]
HTTP::header replace Host "pqr.com"
if {[class match $path starts_with datagroup_rewriteURL]} {
set dgValue [class match -value $path eq datagroup_rewriteURL] ; #i believe this should return /xyz/ if you've set as a string entry in the data group
log local0. "replacing $dgValue with /mnp/"
set newpath [string map -nocase { $dgValue "/mnp/"} $path]
HTTP::uri "$newpath$query"
}
}
ltm object will be something like
ltm data-group internal datagroup_rewriteURL {
records {
/aaa/ {}
/bbb/ {}
/ccc/ {}
}
type string
}
It would depend on how statuc or variable the host/uri strings are. If somewhat static, you could simply do a class lookup to a datagroup using the host/uri combination as a single string match. Something like this:
when HTTP_REQUEST {
if { [set match [class match -value -- "[HTTP::host][HTTP::uri]" equals url_datagroup]] ne "" } {
... match is true if the match is made
... value will be the matched data group value to be used in replacing HTTP:host and HTTP::uri]
}
}
But curious, if abc and pqr are on separate BIG-IPs, AND you need the user's URL to not change in the browser, then one BIG-IP would have to be behind the other BIG-IP.
Thank you for your kind response. Here actually the existing applications of Akana is hosted on a F5 and customer wants to move from Akana to Mulesoft. Mulesoft being a cluster has virtual IP which is hosted on a different F5. These are actually API calls which presently being done on current F5 hosting the virtual server of Akana. Now customer wants to migrate these API calls from Akana to mulesoft with a phase wise migration where the primary need is till the migration is completly done URL+URI in user end should not change as it would drop the response.
customer does not have licesne of APM so they are trying to achieve it with LTM.
I was thinking in the following way which may not be effective at all
When HTTP_REQUEST { if {[string tolower [HTTP::host]] starts_with "https://abc.com"}{ if {[HTTP::path] eq "/xyz"}{ HTTP::header replace Host "https://pqr.com" HTTP::uri "/mnp"} } }
When HTTP_RESPONSE { if {[string tolower [HTTP::host]] starts_with "https://pqr.com"}{ if {[HTTP::path] eq "/mnp"}{ HTTP::header replace host "https://abc.com" HTTP::uri "/xyz"} } }
But If the above works at all I am struggling if there are more URI paths which need to be matched and switched with each other like "xyz1" to replace "mnp1" where the URL https://abc.com and https://pqr.com are static/constant.
From the above solution you have provided, would be kind enough to elaborate how would I be achieve the solution?
Okay, so let's say you have a string datagroup that looks like this:
ltm data-group internal url_datagroup {
records {
www.blah.com:/blah {
data www.test.com:/test
}
www.foo.com:/foo {
data www.bar.com:/bar
}
}
type string
}
I've intentionally added a colon ":" between the host and uri to make it easier to separate them later. So then you're matching on "[HTTP::host]:[HTTP::uri]" in the following iRule. If you have a match, you'll split the corresponding value into host and uri (split on the ":" to make a list object). You'll then, on separate lines, update the HTTP Host header and replace the HTTP URI.
when HTTP_REQUEST {
if { [set match [class match -value -- "[HTTP::host]:[HTTP::uri]" equals url_datagroup]] ne "" } {
log local0. "match = $match"
set matchlist [split $match ":"]
log local0. "match host = [lindex $matchlist 0]"
log local0. "match uri = [lindex $matchlist 1]"
HTTP::host [lindex $matchlist 0]
HTTP::uri [lindex $matchlist 1]
}
}
You can now just add all of these HOST:URI patterns to a single datagroup. Now as I was mentioning earlier, it also depends on how static or dynamic the URLs are. For example, if the URI is statically "/foo", then the above works fine. But if /foo is just the start of the URI (ex. /foo/blah?this=that...), then the above won't completely address the problem. It'll change the HTTP Host, but then you need to do a string map to replace the portion of the URI while (presumably) keeping the rest. Example:
/foo/blah?this=that
would be changed to:
/bar/blah?this=that
You're doing all of this in the HTTP request. There's no URI, path or URI values in an HTTP response.
Point is, if you need to keep the user's browser on http://abc.com, you need to keep that traffic on the abc.com VIP. You can change the HTTP Host and URI on the abc VIP and forward the traffic to the pqr VIP, but there's no scenario where the abc VIP would change these values if the user was able to go directly to the pqr VIP directly.
Okay, so small modification. This assumes the path to be replaced is the first path in the URI:
when HTTP_REQUEST {
if { [set match [class match -value -- "[HTTP::host]:[HTTP::uri]" starts_with url_datagroup]] ne "" } {
## Get first URI path in client request
set FIRSTPATH [getfield [HTTP::path] "/" 2]
## Split match into host and path
set matchlist [split $match ":"]
set MATCHHOST [lindex $matchlist 0]
set MATCHPATH [lindex $matchlist 1]
## Replace FIRSTPATH with MATCHPATH (from full HTTP::uri)
set UPDATEDPATH [string map [list "/$FIRSTPATH" "$MATCHPATH"] [HTTP::uri]]
log local0. $UPDATEDPATH
HTTP::host $MATCHHOST
HTTP::uri $UPDATEDPATH
}
}
I used the first iRule provided by you but seems it is not working. Here is how I used it.
Syntax API URL: https://{host}/{base-or-context-path}/{resource-path}. Here resource path as per customer could be anything so when we are rewriting they only care about to rewrite https://{host}/{base-or-context-path} and rest of the respurce path can be passed as a string to next VIP. So I used your guidance as following :-
URL at Akana :- https:/api-tint.blah.com/akana-mulesoft-routing-poc/{anything will return 200 OK}
But when a user is calling the Akana API with the help of the above list it should go to Mulesoft as we are rewriting the https://{host}/{base-or-context-path} but unfortuntaley it is landing on Akana itself not getting directed to Mulesoft. Interseting thing is iRule is taking the hit as we can see in statistics and also there is no error is showing while the iRule is getting the hit.
The [HTTP::host] command is going to return the hostname only (ex. api-tint.blah.com). It looks like you included the protocol scheme in the data group.
It should be:
records {
api-tint.blah.com:/akana-mulesoft-routing-poc {
data np.api.us.test.cnb:/akana-mulesoft-routing-poc
}
Not sure what went wrong as this time also I can there is hit in iRule but the rewrite is not happeing as the traffic still going to Akana instead of Mulesoft.
Okay, so when you say there's a hit in the iRule, you mean you see the log statements? So the let's say hypothetically that the request URL is: https://api-tint.blah.com/akana-mulesoft-routing-poc.
You should see three log statements:
match = np.api.us.test.cnb:/akana-mulesoft-routing-poc
match host = np.api.us.test.cnb
match uri = /akana-mulesoft-routing-poc
If you see these log entries, then you are indeed also internally changing the HTTP Host header and the URI path. But again, that just means you're changing the Host header and URI on the way to the assigned pool. This isn't doing any traffic routing or redirecting.
Here hit in iRule means when I check the statistics of the iRule I see there is hit count increasing as user test it. About the logs I have not checked it in CLI level in /var/log/ltm but when checked in GUI log section under Local Traffic and searched with the VIP there was no log.
Also as you are saying that host header and URI is getting changed on the way while the traffic is traversing to the exisisting pool, if this is one possibility how to check that or achieve the desired goal. As I stated earlier "api-tint.blah.com" is on a VIP which is in backend holding Akana servers and np.api.us.test.cnb is on another VIP (these two VIPs are in same partition of F5) where in the backend Mulesoft cluster is there.
If you've used the "equals" version of the iRule, then the HOST/URI needs to match exactly, so it should generate a log message when you attempt to access this:
There's a version of the code above that uses a "starts_with" evaluation if the second example is expected. But for coompleteness here it is again with a few minor tweaks:
when HTTP_REQUEST {
log local0. "Incoming request: [HTTP::host]:[HTTP::uri]"
if { [set match [class match -value -- "[HTTP::host]:[HTTP::uri]" starts_with url_datagroup]] ne "" } {
log local0. "Datagroup match: $match"
## Get first URI path in client request
set FIRSTPATH [getfield [HTTP::path] "/" 2]
## Split match into host and path
set matchlist [split $match ":"]
set MATCHHOST [lindex $matchlist 0]
set MATCHPATH [lindex $matchlist 1]
## Replace FIRSTPATH with MATCHPATH (from full HTTP::uri)
set UPDATEDPATH [string map [list "/$FIRSTPATH" "$MATCHPATH"] [HTTP::uri]]
log local0. "Updated host: $MATCHHOST"
log local0. "Updated path: $UPDATEDPATH"
HTTP::host $MATCHHOST
HTTP::uri $UPDATEDPATH
}
}
Put this iRule on the VIP and tail the LTM log while you make a few requests from the client.
tail -f /var/log/ltm
At the very least you should see a single log entry per request showing the "Incoming request: " value. If there's a match against the data group, you'll see the other log entries. If you do not see any other log entries per request, then either the data group does not contain the correct values according to the actual/expected request, or the request isn't as you've described.
Now the rewrite is happening from api-tint.blah.com/akana-mulesoft-routing-poc to np.api.us.test.cnb/akana-mulesoft-routing-poc with the help of the iRule but the traffic instead of going to np.api.us.test.cnb/akana-mulesoft-routing-poc (this URL is mapped to VIP in same partition of F5)traffic is hitting to a pool member of URL - api-tint.blah.com/akana-mulesoft-routing-poc.
I believe one of your original requirements was that the Host and URL in the browser didn't change. If these applications are sitting on separate client-facing VIPs, that's a rather hard problem to solve. If you want client traffic going to api-tint.blah.com to land on the np.api.us.test.cnb VIP you could do one of two things:
HTTP redirect the client to the np.api.us.test.cnb VIP. However this WOULD change the Host and URL in the browser.
Pool to the np.api.us.test.cnb VIP from the api-tint.blah.com VIP, where np.api.us.test.cnb is effectively behind the api-tint.blah.com VIP. If it's separate BIG-IPs, then that should be pretty easy. If it's the same BIG-IP, you can employ VIP targeting in an frontend iRule to forward traffic to the second VIP.
Assuming it's the same BIG-IP, the VIP target could look like this, ammending the previous iRule:
when HTTP_REQUEST {
log local0. "Incoming request: [HTTP::host]:[HTTP::uri]"
if { [set match [class match -value -- "[HTTP::host]:[HTTP::uri]" starts_with url_datagroup]] ne "" } {
log local0. "Datagroup match: $match"
## Get first URI path in client request
set FIRSTPATH [getfield [HTTP::path] "/" 2]
## Split match into host and path
set matchlist [split $match ":"]
set MATCHHOST [lindex $matchlist 0]
set MATCHPATH [lindex $matchlist 1]
## Replace FIRSTPATH with MATCHPATH (from full HTTP::uri)
set UPDATEDPATH [string map [list "/$FIRSTPATH" "$MATCHPATH"] [HTTP::uri]]
log local0. "Updated host: $MATCHHOST"
log local0. "Updated path: $UPDATEDPATH"
HTTP::host $MATCHHOST
HTTP::uri $UPDATEDPATH
virtual "/Common/np-api-vip"
}
}
Just to confirm one more thing if these are VIPs are into two separate partition (Two different BIG IPs logically) this same "virtual" command should work as command "virtual" is referring to another VIP here to get land the traffic in the referred VIP.
I'd maybe argue that partitions aren't create separate logical BIG-IPs. It's more about building separate administrative domains on a single BIG-IP. Further, partitions can always access the /Common partition. To answer your question though, it depends on the level of isolation, but you should simply be able to point to the VIP in the oter partition,
virtual "/abc-app-dev/np-api-vip"
or
virtual "/Common/abc-app-dev/np-api-vip"
You don't need SNAT to communicate between VIPs on the same BIG-IP.
when HTTP_REQUEST { log local0. "Incoming request: [HTTP::host]:[HTTP::uri]" if { [set match [class match -value -- "[HTTP::host]:[HTTP::uri]" starts_with url_datagroup]] ne "" } { log local0. "Datagroup match: $match" ## Get first URI path in client request set FIRSTPATH [getfield [HTTP::path] "/" 2]
## Split match into host and path set matchlist [split $match ":"] set MATCHHOST [lindex $matchlist 0] set MATCHPATH [lindex $matchlist 1]
## Replace FIRSTPATH with MATCHPATH (from full HTTP::uri) set UPDATEDPATH [string map [list "/$FIRSTPATH" "$MATCHPATH"] [HTTP::uri]] log local0. "Updated host: $MATCHHOST" log local0. "Updated path: $UPDATEDPATH"
While cheking the tcp dump found that VIP of api-tint.blah.com is resetting the connection and error showing as Route domain not match. We tried to with disabling strict isolation on VIP of api-tint.blah.com but result is same(although this time we did not captired the logs)
I know it would be a silly question but do I need to add virtual server IP of np.api.us.test.cnb as a pool member to Virtual server api-tint.blah.com or the front end iRule will forward the traffic to target virtual server np.api.us.test.cnb with command "virtual"?
Still no. I probably confused the issue adding a note about route domains. Normally to pool to a VIP across route domains, you have to physically leave the box and return. Without doing that, you can't simply pool to another VIP. That's what VIP targeting is for.
Thanks for your guidance. Here is what I figured out and it is working now. All happened due to your idea so thanks again.
IRule for same routing domain :-
when HTTP_REQUEST { log local0. "Incoming request: [HTTP::host]:[HTTP::uri]" if { [set match [class match -value -- "[HTTP::host]:[HTTP::uri]" starts_with datagroup_Name]] ne "" } { log local0. "Datagroup match: $match" ## Get first URI path in client request set FIRSTPATH [getfield [HTTP::path] "/" 2]
## Split match into host and path set matchlist [split $match ":"] set MATCHHOST [lindex $matchlist 0] set MATCHPATH [lindex $matchlist 1]
## Replace FIRSTPATH with MATCHPATH (from full HTTP::uri) set UPDATEDPATH [string map [list "/$FIRSTPATH" "$MATCHPATH"] [HTTP::uri]] log local0. "Updated host: $MATCHHOST" log local0. "Updated path: $UPDATEDPATH"
HTTP::host $MATCHHOST HTTP::uri $UPDATEDPATH
virtual "/path/virtual_server_name"
} }
For different route domain
when HTTP_REQUEST { log local0. "Incoming request: [HTTP::host]:[HTTP::uri]" if { [set match [class match -value -- "[HTTP::host]:[HTTP::uri]" starts_with datagroup_name]] ne "" } { log local0. "Datagroup match: $match" ## Get first URI path in client request set FIRSTPATH [getfield [HTTP::path] "/" 2]
## Split match into host and path set matchlist [split $match ":"] set MATCHHOST [lindex $matchlist 0] set MATCHPATH [lindex $matchlist 1]
## Replace FIRSTPATH with MATCHPATH (from full HTTP::uri) set UPDATEDPATH [string map [list "/$FIRSTPATH" "$MATCHPATH"] [HTTP::uri]] log local0. "Updated host: $MATCHHOST" log local0. "Updated path: $UPDATEDPATH"
