How can I find out the behavior of the GoogleBots and which pages they crawl? Also, how often do they crawl an average site?

I'm not sure as to how often they crawl or how you can find out, but a quick google search will help.
As far as i know, the Pageranks are updated every 2-4 months by google.

You can see the spiders in different analytics. I use statcounter and there you can see every bot crawling your site.

Don't you use google analytics. Try google analytics you can watch each & every thing what you want to see

i also use statcounter.. and i frequently see google bots crawling my site.. it also indicate the page/s..

Use statcounter and google analyse.

what's an "average" site??

Originally Posted by bigcat1967

How can I find out the behavior of the GoogleBots and which pages they crawl? Also, how often do they crawl an average site?

It all depends on what kind of site, how big the site is and how much content is on the site.

Also, the person may or may not have a modified GoogleBot crawl speed set in their Google Webmaster Tools.

Can you trace it from Google analytic?
How about the tool to check google bot's last visit

no you can't, bot's do not trigger javascript.

And you can do exactly the same as that spectacularly useless "tool" by checking the cache date on your own pages at Google.

Use extreme tracking this is free and most effective

Originally Posted by chrishirst

no you can't, bot's do not trigger javascript.

And you can do exactly the same as that spectacularly useless "tool" by checking the cache date on your own pages at Google.

That reminds me of the two million page rank checkers out there.

Originally Posted by bigcat1967

How can I find out the behavior of the GoogleBots and which pages they crawl? Also, how often do they crawl an average site?

If the site is well optimized and update content daily Google crawl daily ... Can see many forums like this crawling daily.. check by Google tool bar.

Google made some minor crawling almost every day then major crawling once in a month.

Download google toolbar and see catche pages.

Use google analytics is the best for tracking

Originally Posted by bigcat1967

How can I find out the behavior of the GoogleBots and which pages they crawl? Also, how often do they crawl an average site?

The typical site probably gets crawled once or twice a week by Google. That's
because the typical site does not send update notifications very often, does not
add new links very often, and usually just plain sucks. LOL


You can do better if you learn the inside.

Nobody here can tell you the exact behavior of the Googlebot. If they could they'd be getting rich beyond the dreams of avarice and Google would be frantically changing it. The exact algo is one of the most valuable corporate secrets in the world.

But we have worked out some of the basics, and you're in the right place to glean some of those answers. Mind you there's way to many for one post.

As for how long an average site get's crawled... Chrishirst is trying to get you to see that there are no average sites. Every site is different and more importantly so is the quantity and quality of their incoming links.

As a rule:

1. The more incoming links you have, the more often your site will get crawled. That's how the crawlers find your site, and hey, since they're there anyway, why not crawl a link or two before following a link out?

2. The more links that go into the page that linked to you, the more often it gets crawled. The more often it gets crawled the more often it will follow your link out and you'll get crawled again.

3. If the page linking to you has a lot of Outbound Links (OBLs) it's less likely the crawler will pick your link, so a high PR page with a lot of OBLs may be less valuable than a lower PR site with only 1 OBL.

Confused yet? Cool. And I'm seriously oversimplifying.

The easy version is this. The more work you do, the more links you get, the more you get crawled.