Robots.txt Generator

Create a robots.txt file to control how search engines crawl your website.

Configuration

Default Access

Sitemap URL (optional)

Crawl Delay (optional)

Seconds between requests (not all bots support this)

/admin/ /wp-admin/ /cgi-bin/ /private/ /tmp/ /search/ *.pdf files URLs with query strings

Meta Tag Generator

Sitemap Generator

The robots.txt file tells search engine crawlers which pages they can or cannot request from your site. Place it in your website's root directory.