Select a pool member based on URI and remove the URI info

[VAF]

We have one virtual server with multiple pool members. Currently, the site is functioning with an irule that inserts a header and selects a pool member round robin. We would like to select a specific pool member based on the uri. The goal is for a user to be able to append a server name as the URI and the correct pool member is then selected. If the web site is mysite.com, the user can append mysite.com/server1 to select the pool member associated with that server. The URI would then be stripped from URL so it reaches the backend server as mysite.com.

Note: An access policy is being used for this virtual server/site.

I initially started with using when HTTP_REQUEST, but found that would not work correctly with an access policy and the APM in use. I then used the ACCESS_ACL_ALLOWED. It now lands on the correct pool member page, but the page stays stuck at "Loading". The header info is inserted within this event, but I am not sure if it should be within another event? The current implementation is shown below.

when ACCESS_ACL_ALLOWED {

  switch -glob [string tolower [HTTP::path]] {

    "/server1" {

               HTTP::path "/"

                 pool test_pool member 111.111.111.111:8000

                 }

    "/server2" {

                HTTP::path "/"

                 pool test_pool member 222.222.222.222:8000

                 }

   }

set username [string tolower [ACCESS::session data get session.ldap.last.attr.employeeID]]

HTTP::header insert X-Remote-User $username

}

[SanjayP]

HTTP_REQUEST event should work here. You can use HTTP_REQEUST_SEND event to insert HTTP headers to backend. What error do you get with HTTP_REQUEST event?

[Nicolas_Martin-]

Hi!

Does your application require persistence profile (once your user if forwarded to a node, all request goes to the same node) if yes what is your persitence config ?

Regards,

[VAF]

Thanks for the quick replies.

I first started out with using when HTTP_REQUEST instead of when ACCESS_ACL_ALLOWED

When any combination of HTTP_REQUEST is used, the web page returns a "Can't reach this page" with a connection reset. Since then, I have tried putting the code in the following events: HTTP_REQUEST_SEND and HTTP_REQUEST_SEND with the clientside code added. These all produced the same "can't reach this page connection reset" . I also tried using HTTP_REQUEST and putting the header inserts in HTTP_REQUEST_SEND with the same result.

Apparently, the APM's own internal redirects create a problem and the pool member selection and/or changing the URI to "/" gets lost.

When I used the ACCESS_ACL_ALLOWED, it actually drops on a page, but never completely loads, it just stays stuck on "loading" but there are no errors, so that appears to be closer as a solution.

Nicolas,

This is a connection to Splunk servers. Per the Splunk requirement, we use a cookie persistence profile, which is needed for the connection to the Splunk servers to work.

[Nicolas_Martin-]

Hi!

Thanks for providing this information.

My guess is you have issue with persistence : Remember by default the persistence is attached to the TCP session and persistence data are not parsed for each HTTP request. Nowaday a lot of browser is keeping it's TCP session alive and reuse it to pass multiple HTTP get request. This mean that if you come with a new persistence cookie in a TCP connection wich already get tied to a node, your will be send to the old persist node until you get a new TCP connection (by fully close and restart your browser for example)

Issue number 2 is when you target a Node in your Irule you probably don't set a persist cooki for this node and the next request will determine your persist node.

You can check if all request goes to the good node by using the "developers tools" of your browser and check for your persist cookie you should see that the reply for request to example.com/server1 does not contains a set-cookie for your persist cookie.

To solve this you can try the following :

• Create an irule to force persist-cookie evaluation for each request (so if your tcp session is tied to server3 and you go to /server1 the following request will be forward according to the new cookie. The content of the rule is quite simple :

when HTTP_REQUEST {
        catch {LB::detach}
}

• Next you need to handle /server1 and /server2 uri... This can be done easily with policy. Policy are much easier than irule and well optimised => anything that can be done with policy should be done with policy rather than irule. See the following screenshot for the policy config :

Just change :

• Node IP/port with your value
• CookieName with your actual persist cookie name
• Value : you can use this tool https://www.nooby.fr/pages/f5-cookie-encode-decode.php to convert your node IP:port to actual cooki value.

I did lab this set up with a dummy APM policy (just a per session permit all) an it is working fine on Big-ip v15.1.3

Regards,

[VAF]

Nicolas,

Thanks, for the response and the effort you have put in with the policy. Our team is new to policies, which is why we were trying to implement this with irules. Also, our background is network WAN/LAN, so our background with web sites, cookies, etc., is limited

I am creating the policy, but the cookie part is where I have a question.

For this virtual server, we are using the basic Cookie persistence profile. To understand the cookie part, I set the virtual server back to where it is working ( round robin pool members, base url of mysite.com, irule that inserts user info). In the browser developer tools, under cookies, I see lots of entries. One of them is MRHSession. Is that the cookie name/value I would be using? There are numerous entries there, BIGIPServersplunkpool, LastMRHSession, etc.

Is there a place in the configuration of the virtual server/profiles, etc where I would see the cookie name being used, or can I just use a name of my choosing and then insert the value associated with the IP/port obtained at the F5 Cookie encode/decode site listed above in that line of the policy/rule action?

[VAF]

Nicolas. I found the default cookie name and created this policy. (IP details removed). I tried to match your policy, with only selecting one of the servers at this time.

I also added the catch {LB::detach} to the HTTP_REQUEST that includes the header insert. That is the only irule applied at this time. Since we are picking a node in the policy, I removed the default pool from the virtual server configuration.

when HTTP_REQUEST {

 catch {LB::detach}

set username [string tolower [ACCESS::session data get session.ldap.last.attr.employeeID]]

HTTP::header insert X-Remote-User $username

}

It's back to the same "Can't reach page error connection reset" that was the original problem when putting the irule coding in anything except the when ACCESS_ACL_ALLOWED event.

I'm going to add some more logging and perhaps tcpdumps to see what is happening, but at this time it doesn't work with the policy.

[VAF]

policy:

[Nicolas_Martin-]

Hi you have to keep the default pool configuration otherwise any traffic not directed to /server1 will have no* direct node affection nor any pool to pick a member.

Edit : just checked on my lab : removing the default pool lead to "connection reset" => you definitively have to set your default pool.

[Nicolas_Martin-]

Could you also confirm the BigIP version you are running and maybe more information regarding to your APM policy ?

[Nicolas_Martin-]

And one more thing to test is to remove the APM profile just to check if this feature is involved in the issue.

[VAF]

version is 15.1.2.1. The Access policy is shown below. It is used on several splunk related sites. We check for a client certificate, run an ldap query to get user info, show a banner. This access policy is working.

 

I have been using some logging to see what is going on. I do see the logging from the local traffic policy, so, the policy appears to be working, since that is the correct URI entered.

 [/Common/splunk_admin_select_pool_member/match coisvrpp008 in URI]: selected coisvrpp008,

So it lands on a page but it looks to just be round robining the servers in the pool. There are two of them. Sometimes it lands on 008 and sometimes it lands on 009 which is the other server in the pool. I think I would say that this is an improvement because it does the following:

• 1. strips off the URI because it lands correctly on a server and displays all the page info and I can click on menus and links on the page and move around the site.
• 2. collects the information from my client certificate and inserts the HTTP header because it lands on a page that only authorized users are allowed. The access policy shows the "unauthorized user" banner if I don't select the correct certificate, so the access policy/ldap query works.

[Nicolas_Martin-]

Well, as far as I understand now you have set the default pool you can somehow reach the application. However you still have issues with persistence as in the same session you move from node 008 and 009.

Next test : (make sure your persist profile "default cookie" is still set to this virtual server just in case)

Fully close your browser => Open web browser => switch to private mode => enable developers tools and start recording "network" => go to your splunk.local/server1

Now inspect your persist cookie for each request :

• First request to application.lan/server :
  • request should not contain the cooki
  • response must set the persit cooki
  • 
• Folloing request must insert the persist cooki :
  • 

Make sure the cookie value correspond to your node_ip:port using https://www.nooby.fr/pages/f5-cookie-encode-decode.php

Regards,

[VAF]

Nicolas,

I will check what you've listed above,, but I did remove the access policy and it is still doing the round robin to either server. It still lands on a page, I'm just not logged in since it doesn't have my credentials. I just checked and the Default persistence profile is still set to cookie. I will verify the cookie name, values, etc.

[VAF]

 

I checked & verified the cookie name and the value. I also thought that maybe the settings for the cookie may need to be changed. Does this look right?

[VAF]

Nicolas,

Again. I much appreciate the effort you have put into this.

Here is what I have done based on your suggestions.

I did open up the developer tools in the browser. Private browser setting is not available on any of our browsers in our work environment. However. I believe I could see all the info you described.

First, I set the cookie profile for this virtual server to not do encryption in order to check/verify the value setting. I then went back to the working environment (no local traffic policy, irule only doing header insert, landing on round robin backend server). I could view the cookie as you described. It is set in the response and then the next time, it is sent in the request. I verified that this is the value that I use when I apply the local traffic policy. All that matches and looks good (name, values)

Then, I implemented the policy and the irule with the catch LB. Again I could see the correct cookie being set and then sent in the subsequent request. However, there is still the same issue with the load balancer selecting a backend server in the same round robin fashion as before.

I can send you screen shots, but it all looked good and matched what you showed.

This is a production application that is on a time line. Although the idea of having one virtual server with a user being able to add a uri to select a specific backend would have been the preferred solution, this does not appear to be doable on the F5 LTM, possibly because of using an access policy? We know that we can get this to work with having one virtual server per backend server. Although this means 10+ virtual servers, a solution we really didn't want, a working solution is better than a non-working solution, especially because this would have been in place weeks ago.

If you have any thoughts on how this could be accomplished without having one virtual server per backend server, that would be appreciated. I probably haven't considered other possible (working) solutions.

[VAF]

Nicolas,

I finally noticed the behavior of this application. I now have three backend servers in the pool. I have three rules in the policy, one for each backend server. Here is what happens:

Open web browser, enter the url with one of the server names, let's say 008. the F5 appears to round robin this and lands on any of the three backend servers. However, if I keep the browser open and enter the url of any of the three backend servers it will then go to the correct one from that point on.

So basically this:

Open browser - select mysite.com/008 - F5 LTM load balances to any of three in pool

With browser still open enter mysite.com/009 - click Refresh, correct backend server loads

Enter mysite.com/012 - refresh - correct backend server

and so forth

So, after initial landing which is round robining from pool, it then works if one does NOT close the browser and enters one of the three urls.

Also, after initial check for credentials, it doesn't ask again after first landing on round robined backend server, so jumping to the different servers is quick.

[Nicolas_Martin-]

Hi !

Nice to see you going forward !

Even if it looks like some stuff may still need to be fix.

However when you says that the first time you go to mysite.com/008 you do not land on the correct node but you do after refreshing the page make me thinking that maybe when you think you get round -robined to a wrong node it is indeed your browser using previous data from it's local cache.

Regards,

[VAF]

Nicolas.

Any ideas on the solution for the caching issue. Note. Each time I access the site it is after browser has been closed and then restarted, so a completely fresh opening of a browser (any browser). What else could be done to stop that from occurring? Is there anything else that could be affecting this? http profile, some setting?