I want to remove the header and footer from my site when MY SPIDER comes through to index the content (not other spiders, just my own). Since the header and footer have alot of text, I think cutting that out will reduce the size of my search database, make indexing faster, and speed up searches.

So, here is my question. If I do something like this, will it slow down my site?

$agent=getenv("HTTP_user_agent");
if agent != "mybot";
include (header);
else
echo "";

I realize the synax is not correct, but is the concept sound? I want to make sure that this will not slow down the site for real visitors, or other bots.

My search script has an option to "ignore" certain content inside special tags, but it still puts that content in the database. As the database grows, I think that extra unused content will slow down the serving of results.

Any opinions on this?

Frank

I think the answer's no. Tons of site does that, even this forum (vBulletin) have tons of conditional statements. I believe it will do no harm.

No, it won't. What will slow down your site (at least the searcg page) is not adding this. You don't want half of your db being filled with junk data, eh?

The text will not effect your site.
Nowadays, people prefer CSS than other scripts