Robots.txt Tester

Test your robots.txt directives to see if specific paths are accessible to search engine crawlers.