Forum Discussion

Oeg87_310913's avatar
Oeg87_310913
Icon for Nimbostratus rankNimbostratus
Mar 06, 2017

Google bot Unreachable on 301 redirect iRules

Hello, premise: I'm not managing directly the F5, I'm just the web designer of the company, so I'm pretty new with this kind of stuff;

 

I’m migrating all European websites (ccTLDs) of the company I work for to a global one (.com), and I have some issues with the google boot and 301 redirects.

 

For example, what I'm doing:

 

  • Example.fr > example.com/fr
  • Example.co.uk > example.com/en-gb
  • …etc..

My system eng. suggested me to implement the redirect with F5 iRules. This means that the actual flow is:

 

Local domain name DNS changed to the F5 IP, inside F5 I put the iRules to redirect the pages to the new global website:

 

i.e. example.fr > F5 > script for the redirect 301 > example.com/fr

 

The iRules code I used is the following:

 

when HTTP_REQUEST {
            if { ( [HTTP::host] eq "www.kerrdental.co.uk") or ( [HTTP::host] eq  "kerrdental.co.uk" ) } {  
                           if { ( [HTTP::uri] contains "/AccessoriesForFillingMaterials/AmalgamAccessories/AmalgamGuns" ) } {
                                              HTTP::respond 301 Location "https://www.kerrdental.com/en-uk/dental-restoration-products/amalgam-guns-accessories"
                           }
                           elseif { ( [HTTP::uri] contains "/AccessoriesForFillingMaterials/CompositeRestorativeSystems/CompoRoller" ) } {
                                              HTTP::respond 301 Location "https://www.kerrdental.com/en-uk/dental-restoration-products/comporoller-accessories"
                           }
            }

}

 

Now, the problem I have concerns the “Change of address” feature in Google Search Console (Webmaster tool):

 

Search Console notify me that “Cannot confirm that 301-redirects work properly, we couldn’t crawl your site. Check that Googlebot can crawl the root of your site using the Fetch as Google tool” Then, when I go under “do Fetch as Google” the system returns me the status “Unreachable”.

 

When I use RexSwain on the HTTP version of the old URL, I get a 503.

 

Receiving Header:
HTTP/1.1·503·Service·Unavailable(CR)(LF)
Content-Type:·text/html;·charset=UTF-8(CR)(LF)
Content-Length:·931(CR)(LF)
Connection:·close(CR)(LF)
P3P:·CP="CAO·PSA·OUR"(CR)(LF)
Expires:·Thu,·01·Jan·1970·00:00:00·GMT(CR)(LF)
Cache-Control:·no-store,·no-cache,·must-revalidate,·post-check=0,·pre-check=0(CR)(LF)
Pragma:·no-cache(CR)(LF)
(CR)(LF)

Can someone help me to understand why google boot cannot find anything?

 

Thank you in advance,

 

Best regards

 

  • Based on the header provided, it looks like a server error and not F5 error.

    Try doing this:

    curl -IL http://www.kerrdental.co.uk/AccessoriesForFillingMaterials/AmalgamAccessories/AmalgamGuns

    You should get a 301 redirect as the 1st response and then try to follow what the subsequent responses are in order to identify the problem.

  • Found the problem: the root cause was the firewall of F5