← Forum Post Reply →

• ← Group Forum | Forum Topic Directory

  Crawling External Website for indexing (2 posts)

• Andy said 1 year, 10 months ago:

  I have been trying to add an external site to crawl on our SharePoint 2010 site for a few days now and keep recieving the same error with nothing getting crawled. Error in Crawl Logs says…

  "Item not crawled due to one of the following reasons: Preventive crawl rule; Specified content source hops/depth exceeded; URL has query string parameter; Required protocol handler not found; Preventive robots directive."

  To set it up I went to CENTRAL ADMIN > SEARCH SERVICE APPLICATION > CONTENT SOURCES and added the URL of the site I wanted to crawl. This is actually the first time I have set something like this up so not even sure I am doing it correctly.

  I thank you in advance for any help with this problem!

  Also the external site does not have any permissions set up. Let me know if you need anymore info. Thanks again!

• Kevin Pine said 1 year, 10 months ago:

  Is your server Windows 2008 R2? Apparently there is an issue with R2 that is causing SP not to be able to crawl external sites properly. There are some workarounds and an active thread monitoring Microsoft's response to the problem here:

  http://social.technet.microsoft.com/Forums/en/sharepoint2010setup/thread/9ae514a1-3d19-4d85-b458-6be9743e1d7b