I utilize both, robots.txt as well as sitemap submission at google.
I havent experienced any obvious difference in either yet, as long as the googlebot finds the sitemap, I am happy.
If you are new to domains and looking to buy, sell and learn about domains then you have come to the right place. DNForum is the largest domain name community on the internet and continues to grow every day. There are over 105,000 domainers on DNForum doing everything from buying domains, selling domains, learning about domains and discussing domains. Take a minute and Register.
Register Today on DNForum IT'S FREE!A few weeks ago it has been released that by using the robots.txt you can avoid manually submitting your sitemaps to google and yahoo. Do you think it works? that there is really no reason to keep submitting it manually?
Submit your Article and links at a multilingual articles directory. Articles are automatically being translated and distributed all over the world. ArticleZap.com
I utilize both, robots.txt as well as sitemap submission at google.
I havent experienced any obvious difference in either yet, as long as the googlebot finds the sitemap, I am happy.
You have to your robots.txt for MSN live. There is no option to submit manually.
Best wishes,
------------
HumDomains
HumDomains - Cheapest Domain Registration | Cheap Domain Transfer & Private Registration
GoDaddy/Wild West Domains Reseller Plans : Cheapest and Best Price Offer ($120 for Pro)
I was prepping to say the same thing...
Been around web dev for awhile, the only time I use a robot.txt file is when I want to STOP the crawl of specific directories or pages.
A good public sitemap is all you need!
Mark
You can use robots.txt to let search engines know about your sitemaps. See http://www.sitemaps.org/protocol.php#informing for information.
I don't think that XML sitemaps are necessary. It's more for search engines' benefit than yours...
Domains Names and Development, SEO -- Travel domains
JerusalemGuidebook.com - EgyptGuidebook.com
AmsterdamGuidebook.com - and many more...
Yeah it seems some missed the press release. Yahoo, Google, Microsoft and i believe the smaller players agreed upon a single website directive.
Make your xml sitemap and tell the search engines where it is via your robots text file like so:
Sitemap: http://www.yourdomainurl.com/sitemap.xml
For those that like full control you can still use google's webmaster tools as it will alert you to any problems etc.
Zombie Movie Bong of the Dead - Get it on DVD or via Digital Download Today! ~ "This is a sure winner." - Tommy Chong
Use your XML sitemap to submit to google. - Rember to resubmit every 30 days. I like to rename my xml file everytime I do an upload, keeping it looking fresh.
Use the Robots.txt to setup restrictions to folders/directorys/files that you do no want listed.
BigCommerce. All-in-one, easy to use, SEO, marketing tools and more. Try it free!
Get 1st page rankings with Click SEO Link Wheelers
Bookmarks