Hi Guys

I noticed a while back this page http://www.flypark.co.uk/testimonials.html was no longer cached, I thought nothing of it at the time, but I just noticed webmaster tools is reporting the page is being restricted by the robots.txt file. I checked the file and there is no restriction, even stranger I deleted the robots file from the server and fetched as googlebot and it still returned it being restricted by robots.txt even when I had deleted the file.

I posted this on Google forum but got no response

The page may not have a cached link, but it seems to be listed in Google's index. I'm not sure why it's being reported as being restricted by robots.txt. I see a small robots.txt file on the site, but nothing in there should block that page.

if you want it crawled and indexed get some links pointing to that URI

I know the page is indexed, but its not indexed properly. The page even has a page rank 4 but for some reason fetch as googlebot is saying it is restricted by robots. I am worried this may happen to my main landing pages.

It has been like this for a while.

I am going to put a fresh post on Google forum and see if I can get an employee to take a look.

The "fetch as Googlebot" doesn't.

If the page is indexed and is showing a SGB "value" then it obviously ISN'T blocked by robots.txt and Webmaster Tools is getting it wrong (and not for the first time either).

Just ignore things that are obviously wrong, put common sense first and rely on "tools" last, if at all.

Ok I will put it doen to an error but the fact the page isn't cached or showing the correct title and meta data and hasn't done so for some time does indicate there is a problem.

hasn't done so for some time does indicate there is a problem.

Not really, it simply indicates that the page does not have enough links pointing to it and therefore not enough REAL PageRank to warrant regular crawling and indexing.
It is NOT a problem it is a SYMPTOM of poor promotional efforts.

We will have to disagree here Chris, the page is well linked throughout the site, why would it have sgb 4 if not? I just saved the page under a different filename uploaded, fetched as GBot and it worked.

experiment
testimonials.html restricted by robots.txt
testimonials1.html restricted by robots.txt
form23.html passed no problems

How do you explain this?

Ignore what the SGB shows that data could be from six months ago and is from a different DC or server cluster that the WMT data is from as are pages from the SERPs.

It doesn't really matter how "well linked" the URI is throughout the site if the only real PR that enters the network is via a single node, that of the "home" URL.

I figured out what it was,

I had a folder called /test disallowed in the robots. Because I didn't have it stated like this /test/ it was restricting all urls in the test folder and urls that began with "test".

A lesson learned here

That makes sense. I'm glad you figured it out. Something so simple that it gets overlooked is usually the problem.