Error. Page cannot be displayed. Please contact your service provider for more details. (17)


Ideal robots.txt and Sitemap for WordPress | Creation Robot
Creation Robot random header image

Ideal robots.txt and Sitemap for WordPress

February 6th, 2008 · 1 Comment

Looking for a good robots.txt file for WordPress? Well here’s a good example, culled from a few different guides around the web and rolled into one file:

example_robots.txt

The crawl delay prevents bots using too much of your site bandwidth by over crawling.

example_sitemap.xml.txt

You can auto generate your basic Sitemap here. Auto generate a 500 page Sitemap here.

Category: Blog Tech · CreationRobot

1 response so far ↓

  • 1 Zhdanov // Nov 5, 2008 at 8:46 PM

    Fun. Add to your bookmarks. And how long it took to write articles?

Leave a Comment