The Other Search Engines

**J.K.** · 12-27-2007, 01:25 PM

How do you sort out the robot page views vs. the live people page views?
We get hits on our site, but how many are robots? Is there any way to tell?

Is there a good free web stats package out there that may sort this out?

Thanks...

**CSS4Life** · 06:28 PM

Quote:

Originally Posted by J.K.

Is there a good free web stats package out there that may sort this out?

There's Google Analytics or Awstats. I've used the latter one myself because it comes with cPanel, but a lot of webmasters recommend Google Analytics for it's in depth information about website hits.

Hope this answers your question

**chrishirst** · 12-27-2007, 03:51 PM

and FunnelWeb so you can download your logs and analyse them.

**J.K.** · 12-27-2007, 04:25 PM

Hey..thanks for the tips..
I signed up for Google Analytics and downloaded FunnelWeb..
we'll see if that sorts out the robots!

**chrishirst** · 12-27-2007, 04:37 PM

You will NEVER see a reference to a bot visit in GoAn, it uses javascript to gather the data and 99% of bots are NOT javascript enabled.

**J.K.** · 12-27-2007, 07:22 PM

So you're saying GoAn ignores the bots? Then all the results are from live people? That's exactly what I want!

**chrishirst** · 12-27-2007, 07:25 PM

Seeing what bots are hitting your site is an important thing to be knowing about.

Not all of them are good ones, and you need to know the bad ones to block them.

**J.K.** · 12-27-2007, 07:58 PM

That is a good point..I don't have a robots.txt file now. How do you know which bots are bad? Is there a site that keeps track of them? Obviously I don't want to ban all bots..

**vangogh** · 12-27-2007, 08:11 PM

Bob unfortunately most of the bad bots aren't going to identify themselves. There are lists of the ones people have found. Usually what I do is when I see something I suspect I want to block I'll search on it's name and see what it's all about. Sometimes it is a bot I want to block and other times it's something harmless.

With all the different analytics packages it's important to remember that none are perfect. Google Analytics won't be showing bot activity, but they may also be under reporting traffic since there are some people who might still be surfing with javascript disabled. It's usually best to have several analytics packages and best to look at the numbers relatively instead of absolutely.