DNForum - Domain Sales, Domain Forum, Domain Appraisals, Domain Registrars - View Single Post

**meetmydomains** · 07-18-2007, 09:00 PM

I was prepping to say the same thing...

Been around web dev for awhile, the only time I use a robot.txt file is when I want to STOP the crawl of specific directories or pages.

A good public sitemap is all you need!

Mark

07-18-2007, 09:00 PM	#5 (permalink)
meetmydomains Platinum Lifetime Member Name: Mark Last Online: 09-22-2008 10:40 AM iTrader: (2) Join Date: Jul 2007 Posts: 45 DNF$: 0 Location: North Carolina Country:	Re: Google sitemaps I was prepping to say the same thing... Been around web dev for awhile, the only time I use a robot.txt file is when I want to STOP the crawl of specific directories or pages. A good public sitemap is all you need! Mark __________________ Start Building Niche Stores Today!