There's a certain section of code on most of the pages of my site that randomizes a "featured section" once a day. I have a google custom search for searching for content on the site.

I want that "featured section" to not be indexed by search engines, so that if I search for something that happens to be commonly in the featured section, the search results don't turn up every page on the site.

Look into PHP, It's a hypertext pre-processor, meaning it codes all the html server side hiding a good amount of the code

You may want to refrain yourself from doing that invisionary.
Google TOS specify that the site must not behave differently for users than for search engine.
If they see it, you may be off their directory.

If it's so much of a problem, you can disallow the search engines to index this file, with either the use of meta tags, or by adding the fie to the robots.txt

That "file" is included into the page via PHP, and as I said, is on most of the pages of my site.

The problem is that when I use a search engine (Google) on just my site, if I were to search for something that happens to be commonly in the featured section, which changes every day, then the results would include most of the pages on my website.

That's bad.

I know that the idea of a randomized "featured" section isn't a new concept, so there must be some way around this problem.

From what I understand about robots.txt you can't exclude things PHP-included into a non-excluded page, right?

From what I understand about robots.txt you can't exclude things PHP-included into a non-excluded page, right?

Right, because for the search engine, datas are datas.
It cannot makes the difference between an "hard" written text, and a extrat of text pulled outside another page from php.
Both looks the same.

But i won't fear that much. When I'm searching solutions for linux related problems on google, I often see results that looks like a reply from an email.
They are just a part of a precedous email, included in a reply, but Google is smart enough to make the link with the page that seems to be the original one on that same host, and provide both as results, with a small padding between both.

Wait, I'm getting an exemple:

... Like that...
Totally out of context, but you see what I'm talking about.

So, I would remain confident in the search engines ability to make their way and find the right page as a source from your site.

No, you cannot exclude php-included content.

But if you're really concerned about it being indexed, you could have the content loaded via a Javascript. This wouldn't work for all users (approx. 10% still seem to have it disabled), but the ones who do have it will still see it. SEs and non-JS users will still see a blank.

Personally, though, I don't see this as a big issue. I have sections of sites that load random content via ASP includes (same idea as yours, just a different language) and it doesn't do a thing to hurt the search engine rankings of those clients.

If you're really concerned about this issue, I'd suggest building your own search engine for the site. It will have the look and feel of the rest of your site, and you can customize it to include (or not include) whatever you like.

Hello Invisionary,

I have a very similar problem. I have pages with Tips of the Day for each day of the week with tip archives that list all previous titles (as links to the actual entries) from all previous weeks of a given day at the bottom of each page. Site search engine results are heavily burdened by hits from all the titles listed in these archives. These archives are php includes.

The only thing I can think of to do is to put the archives on their own page, so instead of a named anchor linking to archives from top of page to bottom--the tip archive link will take the visitor to the different page where all the tip archives titles/links. THEN each of the archive pages can include meta tags that will block the search engine 'bots from seeing that page. I suppose for your situation, this would translate to an icon or text that links to the Daily Feature--instead of posting on the same page as the other content.

I'm still studying this problem, but so far this is what I've come up with.

susataun