# Exclude Files From All Robots: User-agent: * Disallow: / # Allow only these User-agent: googlebot Disallow: Crawl-delay: 10 User-agent: msnbot Disallow: Crawl-delay: 10 User-agent: slurp Disallow: Crawl-delay: 10 # End robots.txt file