Frequently Asked Questions
- Isn't this a "so what?" project?
- Who are you people?
- What happens if there's a problem with the bot project - will my robots.txt still be delivered.
- Will this code have any impact on my ranking?
- How do I know you're not going to sell my email address, URLs, etc to the devil?
- Aren't there other sites doing something like this?
- Will my data be safe?
Dunno. We like to know who or what is on our sites and when they're doing it.
We all analyse user behaviour to ensure that they get the best experience and there are applications out there that does that same for bots but we think that if we put together a bigger picture of bot acivity then who knows what'll come out of it.
At the very least, you'll know which bots have visited your site and when they did it. That's pretty cool but to have that info displayed on a larger scale could well be really... errr.... cool.
At the moment, 11th March 2008, "we" are Dan Frost and Matt Burke.
Matt is a developer with a huge range of skills (PHP, .NET, IIS, Apache, Photoshop, 3d stuff, etc, etc) and is always willing to help where he can which means he's a busy boy.
Dan is the one that talks too much (IE the person who is rattling this stuff off now). I'm a techy but haven't really coded for years as people like Matt are much better than me at it.
Once we get the ball rolling we'll be actively seeking developers, moderators, etc to help with the general direction of the project.
Yes. We do a flush before we do any updating so your robots.txt will be displayed, then the bot project updating code is executed.
The code is as small and "light" as possible so there really shouldn't be a problem.
No. The search engines won't be aware that your robots.txt is being tracked so if, for some bizarre reason, they all decided that the bot project was dangerous to them and wanted to penalise everyone involved (think paid-for links), they simply couldn't... unless they just penalised everyone which would kinda level things out again...
We wouldn't dream of passing your information onto Google... (joke, joke).
Seriously, we just aren't in that business. We wouldn't know where to start with it. I can't imagine the information is worth enough for us to spoil this whole project anyway.
It won't happen.
As far as we can tell, not really. There's a couple of sites that publish info about bots but I don't think there's anything that aims to publish the bigger picture.
Nor do we think there's anything that's community driven that is just about monitoring bots.
As safe as possible, yes. You have the option to publish your data on a site by site basis but other than that, your data is only available as part of the summary data.
