Robots.txt Generator

Create robots.txt files for search engine crawlers

Valid robots.txt• 3 lines

User-Agents:1

Rules:2

Configuration

Quick Presets

User-Agent Block 1

User-Agent

Rules

Crawl-delay (seconds)

Sitemap URL

Host (optional, for Yandex)

robots.txt3 lines

User-agent: *
Allow: /
Disallow: /admin/

About Robots.txt Generator

Generate robots.txt with allow/disallow rules, sitemap references, and crawl delays. Presets for common configurations.