Google sitemaps

[nikniknik]

A few weeks ago it has been released that by using the robots.txt you can avoid manually submitting your sitemaps to google and yahoo. Do you think it works? that there is really no reason to keep submitting it manually?

[smirkley]

I utilize both, robots.txt as well as sitemap submission at google.

I havent experienced any obvious difference in either yet, as long as the googlebot finds the sitemap, I am happy.

[hum]

You have to your robots.txt for MSN live. There is no option to submit manually.

Best wishes,
------------
HumDomains

[hyped]

guys, a robots.txt file defines where the search engine should not crawl.

For google, you can either:
1) create an account and submit a sitemap
2) create an account and submit your site to be crawled
3) wait for google to find your site via a link from another site

[meetmydomains]

I was prepping to say the same thing...

Been around web dev for awhile, the only time I use a robot.txt file is when I want to STOP the crawl of specific directories or pages.

A good public sitemap is all you need!

Mark

[webd]

You can use robots.txt to let search engines know about your sitemaps. See http://www.sitemaps.org/protocol.php#informing for information.

I don't think that XML sitemaps are necessary. It's more for search engines' benefit than yours...

[whitebark]

Yeah it seems some missed the press release. Yahoo, Google, Microsoft and i believe the smaller players agreed upon a single website directive.

Make your xml sitemap and tell the search engines where it is via your robots text file like so:

Sitemap: http://www.yourdomainurl.com/sitemap.xml

For those that like full control you can still use google's webmaster tools as it will alert you to any problems etc.

[POLiSH]

Use your XML sitemap to submit to google. - Rember to resubmit every 30 days. I like to rename my xml file everytime I do an upload, keeping it looking fresh.

Use the Robots.txt to setup restrictions to folders/directorys/files that you do no want listed.