Hi folks, I have a quick question.

Has anyone noticed that the newest version of FP inserts a "robot" and a "googlebot" command in the header of EVERY page on your site if you open your site with it?

What I want to know is can that actually HURT your SEO with Google, and are you more experienced webmasters recommending that those commands be removed? After all I have had a robots text file on my site for years and I have seen HUGE crawling activity from the googlebot until recently.

I have noticed a HUGE drop in googlebot activity on my site since I started editing my pages with the newest version of FP.

As a quick note, I do not use FP sync, I only use FP as my editor, and I upload my pages manually with another ftp program in binery. FP extensions aren't even enabled on my host as I choose not to use it for that. Its just my editor.

Thanks for any help anyone can provide on this.

Frank

depends on the content of the meta element as to whether it will cause problems or not.

"noindex, nofollow" will definitely be a problem, "index,follow" won't.

The only time to use a Robots Meta Tag is if you want to restrict either with nofollow or noindex or both. Otherwise, there is no point in using it, because the default is follow, index.

The Googlebot meta tag is only recognized by Google and offers a chance to give special instructions to Google. I would have to research the additional options this tag offers in addition to nofollow, noindex, but unless you have a need for special handling by Google, there is no need to have it.

Sounds like your Robots.txt has been doing the job just fine.

I might add, speaking of Google, I have seen Google use the text of my Description tag verbatim on several occasions to place a description in my index listing. It has to fit in with the page content, of course, and the search terms used relative to page content. And this is on pages that contain few pictures or other non-textual content.

Larry

Thanks guys.

All my robots file says is this..

User-agent: *
Disallow: /webstat
Disallow: /webstat_old

Didn't think there was any need for them to crawl my stats archive.

I just wasn't sure if these header commands were needed at all or if they were hurting my site in any way crawler wise.

The actual header inserts that the new FP adds look like this...

<meta name="robots" content="follow,all">
<meta name="googlebot" content="follow,all">

I have one other question if anyone would care to comment....

I have quite a few blogs and myspace pages using pics on my site, and they are not saving their own copy and keeping it on thier site, they are just using the linked pic from my site and displaying it on their blog or personal page. Basically they are stealing bw from me...

I am not in traffic trouble because of it, but I don't like the potential for issues. Since ALL of the pics used on my site are in the traditionl "images" directory, what permissions, if any, can I set on the images directory that only items from inside my site can use those pics, and outside links will be stopped? Is there a way to do this?

Again, thx for any input.

Frank

<meta name="robots" content="follow,all">
<meta name="googlebot" content="follow,all">

Of no use at all http://www.robotstxt.org/meta.html

Stopping hotlinking will depend on what server technology you are using.