Comments on: Robots.txt Misconfiguration = SEO Death http://www.theorganicseo.com/seo-tips-techniques/robotstxt-misconfiguration-seo-death.html Just another WordPress weblog Fri, 12 Mar 2010 19:16:54 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Kevin http://www.theorganicseo.com/seo-tips-techniques/robotstxt-misconfiguration-seo-death.html/comment-page-1#comment-763 Kevin Wed, 02 Sep 2009 14:58:59 +0000 http://www.theorganicseo.com/?p=540#comment-763 I once offered to help a friend of mine with the site for his business. He'd done his best to do some things on his own, and despite his efforts, he just wasn't seeing any changes in his search results. So I made some changes on his site, got him listed in some great topical directories, executed a pretty nice link building campaign -- and his site vanished entirely from the SERPs. I couldn't believe it. After checking virtually every element on the site, I finally thought I'd check the robots.txt (he had implemented this himself while I was working on the site after misunderstanding a tutorial). Upon discovery, I facepalmed a few hundred times, but thankfully the rankings did come back (and eventually improve) in pretty short order. I once offered to help a friend of mine with the site for his business. He’d done his best to do some things on his own, and despite his efforts, he just wasn’t seeing any changes in his search results.

So I made some changes on his site, got him listed in some great topical directories, executed a pretty nice link building campaign — and his site vanished entirely from the SERPs. I couldn’t believe it.

After checking virtually every element on the site, I finally thought I’d check the robots.txt (he had implemented this himself while I was working on the site after misunderstanding a tutorial). Upon discovery, I facepalmed a few hundred times, but thankfully the rankings did come back (and eventually improve) in pretty short order.

]]> By: John http://www.theorganicseo.com/seo-tips-techniques/robotstxt-misconfiguration-seo-death.html/comment-page-1#comment-755 John Fri, 21 Aug 2009 01:45:55 +0000 http://www.theorganicseo.com/?p=540#comment-755 Well it works with Google anyway. There are so many spiders that completely ignore robots.txt, it's really annoying. You stats should come back pretty quickly though in my experience. Well it works with Google anyway. There are so many spiders that completely ignore robots.txt, it’s really annoying. You stats should come back pretty quickly though in my experience.

]]> By: David http://www.theorganicseo.com/seo-tips-techniques/robotstxt-misconfiguration-seo-death.html/comment-page-1#comment-753 David Thu, 20 Aug 2009 17:07:57 +0000 http://www.theorganicseo.com/?p=540#comment-753 I know a "professional SEO company" that did this for 4 months on a high-end site :S I know a “professional SEO company” that did this for 4 months on a high-end site :S

]]>