Forum Discussion

ukitsysadmin_95's avatar
ukitsysadmin_95
Icon for Nimbostratus rankNimbostratus
Jan 17, 2011

Old URL 404, compare URI with external data group file

Hello all,

 

 

I was wondering if someone could help me or point me in the correct direction.

 

 

 

I have about 40,000 URL's and growing which need redirecting,

 

 

 

here is an example of a few

 

 

 

Original URL

 

http://www.xyz.com/event/123/man-in-band

 

to

 

http://www.xyz.com/event/98473/man-in-band

 

 

 

or

 

 

 

Original URL

 

http://www.xyz.com/venue/2347/band-in-park

 

to

 

http://www.xyz.com/venue/9383783/band-in-park

 

 

 

I've been told that the original URL will no longer exist and will produce a 404 status

 

 

 

Is it possible to say something like

 

 

 

when HTTP_RESPONSE {

 

if { [HTTP::status] contains "404"} {

 

 

 

 

extract end of URI

 

e.g. "man-in-band"

 

 

 

then search / reference an external data group file, which will hold the 40,000 URL and redirect to URL which has matching end of URI

 

e.g. "man-in-band"

 

 

 

Also another quick question, where is the best place to store the external data group file and in what format is best for performance ?

 

 

 

 

 

I hope this makes sense,

 

 

 

regards

 

 

 

 

 

  • You can use some string commands to get the string after the last forward slash:

     

     

    % set uri /abc/def/ghi/jkl/mno

     

    /abc/def/ghi/jkl/mno

     

     

    % string range $uri [expr {[string last / $uri] + 1}] end

     

    mno

     

     

    So you could parse the last field of the HTTP::path in HTTP_REQUEST, save it to a variable (set uri_token [string range [HTTP::path] [expr {[string last / [HTTP::path]] + 1}] end]) and then check in HTTP_RESPONSE for a 404 status. You could then do a lookup against a string datagroup to get the updated URI. You could either retry the request using HTTP::retry to hide the change or send a 301 redirect and have the client see the change and make a new request.

     

     

    Aaron
  • Hi Aaron

     

     

    Thanks for the reply, we understand your reply but not sure how to finish the last part of this.

     

     

     

    when HTTP_REQUEST {

     

    setting a variable for the URI

     

    set uri [HTTP::uri]

     

    setting a variable for the external string data group

     

    set v3 $::v3redirects

     

    finds the text after the last / in the URL

     

    string range $uri [expr {[string last / $uri] + 1}] end

     

    }

     

    if 404 and URI matches our external data group

     

    if HTTP_RESPONSE { [HTTP::status] == 404} and { [matchclass $uri equals $v3] }

     

     

     

     

    I think the above is almost correct, what we are stuck ont is if there is a $uri match extract the whole URL from the data group into another variable

     

     

     

    Once we have a variable we can then do a 301 redirect to.

     

     

     

    Another question, what happens if there is no match ? will you get a 404 or should we handle it with an else statement to our home page for example ?

     

     

     

    thanks in advance

     

     

     

    Jim

     

     

     

     

     

     

     

     

     

  • Colin_Walker_12's avatar
    Colin_Walker_12
    Historic F5 Account
    Aaron's got the idea for sure, and gave you the expression you'll need to get the end of the URI. All you need to do is perform the check on the response code and do the class lookup.

     

     

    So what you're looking for is something like:

     

     

    
    when HTTP_REQUEST {
      set uri_token [string range [HTTP::path] [expr {[string last / [HTTP::path]] + 1}] end]
    }
    
    when HTTP_RESPONSE {
      if {[HTTP::status] == 404} {
        set red_uri [class match -value $uri_token equals class_name]
        HTTP::redirect $red_uri
      }
    }
    

     

     

    This would have a class called "class_name" with a format like:

     

     

    class_name {

     

    "man-in-band" := "http://www.xyz.com/event/98473/man-in-band",

     

    "band-in-park" := "http://www.xyz.com/venue/9383783/band-in-park"

     

    }

     

     

    Of course, if you wanted to re-use the host name or other parts of the URL so you don't have to specifically set the entire URL to redirect to in the class, you could do that, but this should show you how the idea works, anyway.

     

     

    Colin

     

  • Hi Colin,

     

     

    thank you for the response,

     

     

     

    we are currently running version 9.4.7, does "class match" work with this version ?

     

     

     

    also if class match doesn't work, I assume match class would work but would the format of the class_name you defined still work ?

     

     

     

    class_name {

     

    "man-in-band" := "http://www.xyz.com/event/98473/man-in-band",

     

    "band-in-park" := "http://www.xyz.com/venue/9383783/band-in-park"

     

    }

     

     

     

    Kind Regards

     

     

     

    Jim

     

     

     

     

     

     

     

     

     

  • Colin_Walker_12's avatar
    Colin_Walker_12
    Historic F5 Account
    If you're on 9.4.7 you'll want to use the findclass command.

    Your class would look like:

    class_name {

    "man-in-band http://www.xyz.com/event/98473/man-in-band",

    "band-in-park http://www.xyz.com/venue/9383783/band-in-park"

    }

    And your logic would change slightly to:

    
    when HTTP_REQUEST {
      set uri_token [string range [HTTP::path] [expr {[string last / [HTTP::path]] + 1}] end]
    }
    
    when HTTP_RESPONSE {
      if {[HTTP::status] == 404} {
        set red_uri [findclass $uri_token class_name " "]
        HTTP::redirect $red_uri
      }
    }
    

    Colin
  • You'd probably also want to check that:

     

     

    uri_token isn't null before doing the class lookup ($uri_token ne "")

     

     

    the result from the class lookup (findclass or class match) isn't null ($red_uri ne "")

     

     

    $uri_token could be null for URIs ending in /. $red_uri could be null if there isn't a corresponding datagroup entry for the $uri_token value.

     

     

    Aaron
  • Maybe something like this for 9.4.4 - 9.4.x:

    
    when HTTP_REQUEST {
    
       Save the path to a variable in case we see a 404 from the server
      set path [HTTP::path]
    }
    
    when HTTP_RESPONSE {
      if {[HTTP::status] == 404} {
    
         Parse the string after the last forward slash
        set uri_token [string range $path [expr {[string last / $path] + 1}] end]
    
         If that string wasn't null, look it up in the datagroup
        if {$uri_token ne ""}{
    
          set red_uri [findclass $uri_token class_name " "]
    
           Use the second field in the matching datagroup line to redirect the client
          if {$red_uri ne ""}{
            HTTP::redirect $red_uri
          }
        }
      }
    }
    

    Aaron