Travel Agent suggested http://www.botsense.com/bots.html in another thread bringing up mod_rewrite under Apache. I had a look, and they recommend a bit of code to add to your .htaccess file to ban a lot of spiders, either by their user agent string, or IP address. These aren't GoogleBot; one is called "EmailSiphon."
So I'm curious whether other people do this, and how much strain about 50 lines of RewriteCond sending out "not authorized" messages would put on the server? I'm not that worried about the image miners ( although maybe I should be, I think the © watermark helps some ), but wouldn't mind cutting down the spam I get.
Any thoughts? |
