SEO Tycoon

**BamaStangGuy** · 06-05-2007, 12:57 AM

I have never had this many Googlebots on my forum at once:

Code:

Yahoo! Slurp (119), Google (36), MSNBot (2), Voyager (3), BecomeBot (1), Teoma/Ask Jeeves (1), Google AdSense (1), Gigabot (1)

I have maybe 1 or 2 on at a time but now it has exploded. Anyone else seeing this??? I am hoping something good follows.

**skyhawk133** · 06-05-2007, 10:31 AM

Do you have a robots.txt file blocking your redundant URLs?

I uploaded a new robots.txt file a little over a month ago, I had 308,000 pages indexed (according to site

. Today, I still have 308,000 pages indexed, but now Webmaster Central shows over 450,000 URLs blocked by robots.txt.

Mostly different thread modes, and the like.

**BamaStangGuy** · 06-05-2007, 04:01 PM

yea my robots.txt is ridiculously long lol

I've had up to 20,000 pages blocked by robots.txt in mine before. Photopost is a huge duplicate content hog.

**Hobbit** · 06-09-2007, 12:50 AM

Most users ever online was 420, that's for my forum and it was mostly bots. It was insane. I think it's normal.

**imported_myhomehost** · 06-10-2007, 04:13 AM

I had the same , but i heard it's ok

**Mark07** · 06-14-2007, 09:09 AM

Quote:

Originally Posted by Hobbit

Most users ever online was 420, that's for my forum and it was mostly bots. It was insane. I think it's normal.

Ya you are right, it is normal and it happens many times so there's nothing to bother about. Actually it could be a good thing that multiple bots are there to crawl the site helping to index new pages faster.