Forum Discussion

insert12_38638's avatar
insert12_38638
Icon for Nimbostratus rankNimbostratus
May 13, 2013

robots.txt allow

Hi Bhatthman,

 

 

Sorry , let me re-write my question once again which you understand correctly. Actual Question was can we block or allow robots.txt on GTM or firepass ?

 

Some users from a specific ISP not able to access a website abc because of an issue with “robots.txt. ( website abc is load balanced using GTM )

 

 

Thanks

 

 

application delivery
dev
devops
iRules

1 Reply

  • The_Bhattman's avatar
    The_Bhattman
    Icon for Nimbostratus rankNimbostratus
    May 14, 2013
    HI Insert12,

     

    It's a bit hard to understand your question correctly. However, If I am interpretting your question, I don't believe the GTM is where you would place allowances for robots.txt It would happen most likely at the places where they are blocking by using Robots.txt. That is most likely the place where you would make the allowances.

     

     

    I hope this helps

     

    The Bhattman