Not all robots but some. This one in particular has just made my umm... list.
I had been watching my forum on and off all night tonight while working on other things. I noticed a 'guest' user browsing around several pages of the forum and also hitting the private messages which is something guests cannot do.
I left for a while and returned to find them still there and back in private messages again after being gone about 3 hours.
So I logged into my control panel to view the stats. It is a darn bot and it keeps trying to access the private messages page but it can't. Not without logging in and no bot can do that.
I have now blocked it from accessing my forum because it would be using up too much bandwidth. The darn thing has been there all night trying to get somewhere it cannot go!!
Here is the info my logs showed me about it:
Host: 64.140.49.68
Http Code : 302
Date: Jan 17 07:17:23
Http Version: HTTP/1.0 Size in Bytes: 0
Referer: - Agent: TurnitinBot/1.5 http://www.turnitin.com/robot/crawlerinfo.html
That link told me why they were crawling my site and how I could block them which was awfully nice but I am still upset that they have been crawling around my site all night trying to gain access to places they cannot.
Anyone who has a forum should check for this robot as it is really not a good one if it repeatedly tries to visit a page it isn't allowed to. |
