About the bot project

The Bot Project is about webmasters working together to learn more about search engine bot activity.

A central database is updated each time robots.txt is accessed. The details of the bot are then recorded and available to the webmaster.

The summarised data is then published on this website for all webmasters to get a better understanding of search engine bot activity.

Data such as regularlity, dangerous bots, new bots and other useful information is published dynamically.

This project is not about making money or harvesting email addresses - it's about the webmaster commuity working together.

All data collated with be available publically with site specific information only available to the relevant webmaster.

All of the code that this project uses will be made available and contributors are actively sought in the following areas :

PHP
Java
.NET
Forum moderators

The future

It would be great if we could do more than just monitor the inital visit and actually track (and report) on a bots journey through your site but we're not overly keen on adding yet more Javascript to your webpages.

The future of The Bot Project will be in the hands of the community.

Disclaimers and thanks

User information is NOT shared with third party organisations.

The Bot Project was founded by Dan Frost and Matt Burke. You can contact us by clicking here.

This website uses Drupal and the template was designed by Luka Cvrk who creates a number of excellent Drupal Themes.

The shield logo is courtesy of the excellent Photshop Tutorials over at PSDTuts.com.

The statistics / bot reporting centre will be available as a free module on Drupal.org soon.