- ASP(3477)
- ASP.NET(1519)
- Ajax(293)
- C and C++(487)
- CFML(272)
- CGI and Perl(4376)
- Flash(2133)
- HTML5(1)
- Java(1807)
- JavaScript(2961)
- PHP(16078)
- Python(128)
- Remotely Hosted(2495)
- Ruby on Rails(82)
- Tools and Utilities(4427)
- Web Hosting(24)
- XML(269)
- 1,251 categories
- 40,829 scripts
Categories
Updated: 03/06/2008
robots.checker(tm)
- This is a free, open-source, PHP script that will check for a robots.txt file. If it finds one, it reads it and flags any errors. The valid lines are delivered as links so you can click on them to see what the server delivers for those URLs. You can run it free, or download the code and run it on your own server. It is an easy way to check if the robots.txt file is valid and current.
- Categories
- Platforms
- Linux
- Licenses
- GPL
- Author