Ideal robots.txt and Sitemap for WordPress | Creation Robot
Creation Robot random header image

Ideal robots.txt and Sitemap for WordPress

February 6th, 2008 · 1 Comment

Looking for a good robots.txt file for WordPress? Well here’s a good example, culled from a few different guides around the web and rolled into one file:

example_robots.txt

The crawl delay prevents bots using too much of your site bandwidth by over crawling.

example_sitemap.xml.txt

You can auto generate your basic Sitemap here. Auto generate a 500 page Sitemap here.

  • Digg
  • del.icio.us
  • Facebook
  • Live
  • Spurl
  • StumbleUpon
  • Reddit
  • Technorati

Category: Blog Tech · CreationRobot

Related Posts:

Sitemap
...
Switching from MT to WordPress
...
WordPress 2.0 is Released
...
Moving to WordPress: Complete
...
It’s the all new, all singing, all dancing WordPress 2.0 Theme Competition
...
Is it time for a new look?
...

1 response so far ↓

  • 1 Zhdanov // Nov 5, 2008 at 8:46 PM

    Fun. Add to your bookmarks. And how long it took to write articles?

Leave a Comment