Forum Discussion
Vishal_96707
Nimbostratus
NimbostratusUrgent Help - Sharepoint search crawl failing
We have sharepoint 2003 and sharepoint 2007 environments setup on F5. We are migrating from sharepoint 2003 to SP 2007. We had configured iRule which was capturing "404" and "File Not Foud" msgs from 2003 environments and redirecting to 2007 environments. We are not having issues with the sharepoint search crawl. The sharepoint search crawl fails with the iRules however everything works fine without iRules. Can someone help me? We are in the process of getting new environment in production soon.
The below logic is combined in one iRule.
when HTTP_REQUEST {
set MYHOST [HTTP::host]
set MYURL [HTTP::uri]
}
when HTTP_RESPONSE {
if { [HTTP::status] ends_with "404" }
{HTTP::redirect "https://intranet.bmc.com/_layouts/redirects/Default.aspx?URL=$MYHOST$MYURL"}}
when HTTP_REQUEST {
set MYHOST [HTTP::host]
set MYURL [HTTP::uri]
}
when HTTP_RESPONSE {
if { [HTTP::payload] contains "File Not Found" }
{HTTP::redirect "https://intranet.bmc.com/_layouts/redirects/Default.aspx?URL=$MYHOST$MYURL"}}
Thanks in advance,
hoolioCirrostratus
What are the symptoms of the issue? Do you think the problem is just with the URIs that are being redirected? If so, do you know whether the spider will follow 302 redirects? Maybe the spider only follows 301 redirects. If that's the case, you could change HTTP::redirect... to something like:
HTTP::respond 301 Location "https://intranet.bmc.com/_layouts/redirects/Default.aspx?URL=$MYHOST$MYURL"
If the problem isn't with just the File Not Found redirects, can you provide more details?
Aaron
Vishal_96707http://sptest.bmc.com/SMTIndia
Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled. (The item was deleted because it was either not found or the crawler was denied access to it.)
Here is what we are getting on the index server. If you remove the iRule everything woks fine. This is the only data that i have. As per the sharepoint team the account used for crawling has got Admin rights. If you hit the same url manually and use same credentials it works fine. Let me know if you need more info.- Vishal_96707Symptoms,
- SP 2007 index server is crawling SP2003 Portal server
- I have checked using tcpdump and there was only one place where the redirect was done by F5 using iRule. All the 401 Unauthorized messages are returned by IIS server. All these messages start only when you apply the iRule
- If you remove the iRules and crawl it works fine.
- As per the app team, they start seeing access denied messages frequently and the crawl fails - hoolioDoes the issue occur on every request or just the ones that trigger the iRule redirect? Are you redirecting the spider to a site which it doesn't have access to?
Aaron - Vishal_96707Some more tests..
I tried to log iRules however it is not matching with any of the index server error messages.
Index Server
http://sptest.bmc.com/CCB/Lists/Objectives
Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled. (The item was deleted because it was either not found or the crawler was denied access to it.)
LTM
: Found HTTP Status 404 and the uri is sptest.bmc.com /default_files/filelist.xml Redirecting
Feb 27 06:28:21 tmm tmm[1084]: Rule File_Not_Found-404-Redirect : Found HTTP Status 404 and the uri is sptest.bmc.com /Project%20Status%20for%20Sharepoint%20new_files/filelist.xml Redirecting
Feb 27 06:28:21 tmm tmm[1084]: Rule File_Not_Found-404-Redirect : Found HTTP Status 404 and the uri is sptest.bmc.com /default_files/filelist.xml Redirecting
Feb 27 06:28:23 tmm tmm[1084]: Rule File_Not_Found-404-Redirect : Found HTTP Status 404 and the uri is sptest.bmc.com /default_files/filelist.xml Redirecting
Feb 27 06:28:26 tmm tmm[1084]: Rule File_Not_Found-404-Redirect : Found HTTP Status 404 and the uri is sptest.bmc.com /default_files/filelist.xml Redirecting
Feb 27 06:28:27 tmm tmm[1084]: Rule File_Not_Found-404-Redirect : Found HTTP Status 404 and the uri is sptest.bmc.com /ESM/_vti_bin/sitedata.asmx Redirecting
There is something with the iRules since once we remove the iRule it starts working. Even dont get 401 Unauthorized messages - Vishal_96707The issue has been reolved by bypassing index server hitting redirection logic in iRule.