Originally Posted by chrishirst

For now, ignore what the "webmaster tools" tells you, IF your site logs show there is a problem ie: UAs getting 404's and 403's instead of 200's, then look for a cause.

If GA shows REAL visitors, compare these against your logs.

Your site is new, it takes time . The days of putting a site on-line and it being successful in weeks are long gone.

what do you mean by UA (unknown agent? and if so, do you mean bots?)

and GA - Google Ads?

Just making sure.

UA = User Agent. Everything that requests a page from a site is a user agent and all should leave an identifying string in your logs.


GA = Google Analytics aka GoAn

Originally Posted by chrishirst

UA = User Agent. Everything that requests a page from a site is a user agent and all should leave an identifying string in your logs.


GA = Google Analytics aka GoAn

Ok, thanks just to clairfy. GoAn has a little less hits than what my stats are, by a quick look.

How do you suggest i judget this? Pick a day and see what my Cpanel stats in weblizer say, vs Funnelweb of my own downloading vs, GoAn? Let me know...

And thanks for the clarification for user agent.


(quote)UAs getting 404's and 403's instead of 200's, then look for a cause.<quote>

In refernce to this, where would you recomend i go to determine this? Will funnel web give me that direct info, or do I just need to go and see waht errors the site gives - and it will say there?


I have added back the sitemap to google, becuase whether its there or not I will still get errors, best to give it the best chance.

I am still debating with the robots.txt - I'll leave it off for a bit as its not hurting anything that I can tell at the moment without it.

Thanks for the help

why did you removed your robots.txt? put it back! everything fine . . .

Ok, thanks just to clairfy. GoAn has a little less hits than what my stats are, by a quick look.

Yep. Non-javascript UAs (bots etc) wont be seen by GoAn.

In refernce to this, where would you recomend i go to determine this? Will funnel web give me that direct info, or do I just need to go and see waht errors the site gives - and it will say there?

Funnel Web, in the diagnostics report will show that.

Originally Posted by chrishirst

Yep. Non-javascript UAs (bots etc) wont be seen by GoAn.


Funnel Web, in the diagnostics report will show that.

For my diagnostics, I have mostely 404's

favicon.ico being the main one, tho it shows up in firefox for me fine.
robots.txt is next - I added it back to the server tonight

owssvr.dll - someone's lookin for this, I know i don't have it lol.
cltreq.asp - dunno where on earth this can be coming from, i know its not mine nor would it be linked on the site.

The rest were a few picture urls i got wrong, which i have since corrected.

I have 3 forbidden hits, and I can't even see where they were or what they are.

And my robots info looks like this
Robot Hits % Bytes % Errors
1 GoogleBot 770 64.5% 9.6MB 41.7% 9 2 Hotbot & AOL & GoTo & MSN & Iwon 285 23.9% 11.8MB 51.1% 0 3 Alexa 85 7.12% 1.3MB 5.80% 42 4 Crawler.de 52 4.36% 261.4kB 1.11% 23 5 EmailSiphon 1 <1% 49.1kB <1% 0


Its not even picking up on yahoo..why is that? i know for fact yahoo slurp is on my page multiple times daily from the latest visitors in the cpanel stats.

*sighs*

Its not even picking up on yahoo..why is that?

You probably need to update the UserAgent list on FunnelWeb, there is a link on the download page to an updated file.

owssvr.dll & cltreq.asp Script kiddie bots looking for security holes.

I am not impressed with Google at all for this.

The last time my site has been cached, is Dec 3rd, and I have far more than 64 pages on my site.

My ranking in yahoo, honestly is near spotless. I did my most common searches to compare them and google's on the ****ty end.

But at the same time WHEN googlebot manages to get on my site, I see a surge in the rankings.

Webmaster tools is still erroring out on me. Honestly I wish i knew the answer to this one because I know it should be indexing this site more regularly.

The last few visits to my site from google,

http://www.xfilesnews.com/ robots.txt unreachable [?] Dec 14, 2007http://www.xfilesnews.com/index.php?op...ask=category&sectionid=1&id=1&Itemi d=32 robots.txt unreachable [?] Dec 12, 2007http://www.xfilesnews.com/index.php?...id=2&Itemid=34 robots.txt unreachable [?] Dec 12, 2007http://www.xfilesnews.com/index.php?...d=20&Itemid=32 Network unreachable [?] Dec 14, 2007


Ok, editing this the third time...

I found this at google, its different than what was there 2 weeks ago.

http://www.google.com/support/webmas...er=35662&hl=en
http://www.google.com/support/webmas...y?answer=35653

My sitemap is at http://www.xfilesnews.com/sitemap.xml

Does anyone here have the knowledge to tell me if this sitemap passes these standards?

Originally Posted by Thinkey

I am not impressed with Google at all for this.

The last time my site has been cached, is Dec 3rd, and I have far more than 64 pages on my site.

My ranking in yahoo, honestly is near spotless. I did my most common searches to compare them and google's on the ****ty end.

But at the same time WHEN googlebot manages to get on my site, I see a surge in the rankings.

Webmaster tools is still erroring out on me. Honestly I wish i knew the answer to this one because I know it should be indexing this site more regularly.

The last few visits to my site from google,

http://www.xfilesnews.com/ robots.txt unreachable [?] Dec 14, 2007http://www.xfilesnews.com/index.php?op...ask=category&sectionid=1&id=1&Itemi d=32 robots.txt unreachable [?] Dec 12, 2007http://www.xfilesnews.com/index.php?...id=2&Itemid=34 robots.txt unreachable [?] Dec 12, 2007http://www.xfilesnews.com/index.php?...d=20&Itemid=32 Network unreachable [?] Dec 14, 2007


Ok, editing this the third time...

I found this at google, its different than what was there 2 weeks ago.

http://www.google.com/support/webmas...er=35662&hl=en
http://www.google.com/support/webmas...y?answer=35653

My sitemap is at http://www.xfilesnews.com/sitemap.xml

Does anyone here have the knowledge to tell me if this sitemap passes these standards?

Well my webhost finally listened and whitelisted all the googbot IP address (it WAS that after all). My site is now being cached every 2-3 days by google with no hassle.

Hopefully it keeps up!

Originally Posted by Thinkey

Well my webhost finally listened and whitelisted all the googbot IP address (it WAS that after all). My site is now being cached every 2-3 days by google with no hassle.

Hopefully it keeps up!

So far so good, 6th on google for my main keyword, and somehow I went from nothing to PR3 since this problem has been fixed.

Proves two things: unreachable problems chances are is something blocked on the webhost thing

- it IS possible to build a new successful site in under 3 months if you put the correct work into it.

I think i'm gonna start another site now, using the same strategies.

[quote]it IS possible to build a new successful site in under 3 months if you put the correct work into it.[quote]

Holly I hope a lot of people read this one statement of yours and take it to heart. And congrats on the recent success.