Quote:
Originally Posted by Worldwebuser
3. changing code to divert bots (such as changing the cookie settings)
|
I don't believe it's the cookie settings that kill most of the bots. The majority of the bots use web automation where they are using an actual ie browser object and literally simulating as if the user was clicking on the HTML page elements. So, any cookies and hidden form elements are being sent. That's the easiest approach by far, but it is also the least performant by a large margin.
Myspace can wreak havoc by changing the id's of their form elements, changing urls, and making subtle changes to their underlying HTML code to kill the bots' regular expressions.
The problem for them is that their code is obviously a spaghetti coded mess of crap. It's old coldfusion code mangled to run on .net via a third party vm if i'm not mistaken. Unless they are complete buffoons, they know exactly how to stop the bots. But, they are afraid to change code since a change in one place would probably break code in other places. If myspace were to rewrite their entire app from scratch, they could really put a hurting on the bots. As it stands, they can't even keep their servers running smoothly on a daily basis.
I think it's humerous that whenever there is a server problem they put a message up like Tom is there working on the problem......."this feature is down for the moment. We're fixing it now. Thanks - Tom". That is just ridiculous.
|