|
Of course. If you built your own spider / bot from the ground up, how would you distinguish between a forum, blog, static web site, and any other type? It's all just html you get from requesting URLs.
And like DaveMo said, it's a good thing. I work in IT, and when I search for how to fix some obscure problem, I get plenty of forums, which are usually more useful than the actual documentation.
|