I submitted a sitemap to Google several days ago.

I imagined I would have to wait for Google to next crawl my website before any of the pages would be indexed, but how do I know when Google has visited?

See in the attachment that the 'Downloaded' field has changed to 3rd September, does this mean that Google has visited my website??

However there are still only 2 url's in the web index, see the attached image.

Could it be that for some reason Google hasn't indexed my pages? Or do I simply have to wait a bit longer?

Thanks

Its totally normal. Google does not index all link is sitemap of most websites. Most webmasters have this problem and it takes time that Google index all of your pages. It maybe takes even some months. But don't worry. Google indexes other pages of your website continuously but indexing sitemap links, takes many time. Don't worry at all for this problem.

Having a xml sitemap is no guarantee of the pages getting crawled.

but how do I know when Google has visited?

Check the site logs.

Originally Posted by chrishirst

Having a xml sitemap is no guarantee of the pages getting crawled.

Check the site logs.

Thanks, How would I go about doing that? I tried Googling that kind of thing the other week but couldn't find anything

Check your hosting, you probably have some log analysers available or you could use FunnelWeb

Hmmm I've looked at my access logs through cPanel, and can see several lines such as this:

66.249.65.103 - - [07/Sep/2010:14:20:32 +0100] "GET /company/some-page HTTP/1.0" 200 3396 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

If that is irrelevant then I may give FunnelWeb a try.

Does this mean the Google bot visited today? The page isn't showing in Google searches and Webmaster tools still says that only 1 page exists in their index.

I've also looked on http://websitegrader.com/ and it says that Google last crawled my site on August 31st. Which sort of gives me a bit of confidence, however I've sometimes noticed in the past that this website can give slightly inaccurate information

Check the crawl stats section of Google webmaster tools. Click "Diagnostics", then "Crawl stats".

I can now see that Google has indexed 12 pages on my website (Website Grader is now telling me that Google last crawled my site on the 6th September).

That still leaves over 200 submitted URLs that have not been indexed.

Is it more a case of sitting and waiting for Google to index all (or hopefully most) of my pages? I assumed Google would only have to crawl it once, or does it have to crawl my site several times before most of the pages are indexed?

I understand that Google doesn't guarantee that all (or any) pages will be indexed, but I don't see what the problem would be?? After all, Webmaster Tools states that there are no content issues with my website.

Google has indicated that although addition of XML sitemaps to the websites could be recommended but they would not guarantee all the URLs would get indexed fast. If your website pages are neatly connected to each other you do not even need an XML file because Googlebot is already scanning your web parts. Addition of fresh contents as well as links in the course of time could help a lot.

Originally Posted by bermuda

Google has indicated that although addition of XML sitemaps to the websites could be recommended but they would not guarantee all the URLs would get indexed fast. If your website pages are neatly connected to each other you do not even need an XML file because Googlebot is already scanning your web parts. Addition of fresh contents as well as links in the course of time could help a lot.

Thanks, but my pages are organised quite neatly. I cant understand why only 12 have been indexed, unless of course the GoogleBot takes quite some time to index every page on a new site.

Here is a link to my site if that helps:
http://bit.ly/dw7W3s

Thanks

Does this mean the Google bot visited today?

Yes it means that Google visited that page and retrieved the contents.

I cant understand why only 12 have been indexed, unless of course the GoogleBot takes quite some time to index every page on a new site.

Because it takes time for a site to be deemed worthy of "deep crawling".
It does NOT happen overnight, may take a month, may take six months, but one day your promotional work will trigger a high speed deep crawl, and you'll have more "googlebots" on the site than you know what to do with, and every page they can access will be crawled with sometimes less than a minute between requests.

Originally Posted by chrishirst

Yes it means that Google visited that page and retrieved the contents.

Because it takes time for a site to be deemed worthy of "deep crawling".
It does NOT happen overnight, may take a month, may take six months, but one day your promotional work will trigger a high speed deep crawl, and you'll have more "googlebots" on the site than you know what to do with, and every page they can access will be crawled with sometimes less than a minute between requests.

Ahhh thanks, that's cleared it up

Yes, not all the pages from sitemap will be indexed at once. It will take time. My tip.. update your site content frequently and regularly.. According to my experience if you do so then google will visit your site more often and your site pages will be indexed fast.

Have you tried checking error on your URL? Try to check it on your webmaster accounts maybe some links are not valid.

Originally Posted by pealo86

I submitted a sitemap to Google several days ago.

I imagined I would have to wait for Google to next crawl my website before any of the pages would be indexed, but how do I know when Google has visited?

See in the attachment that the 'Downloaded' field has changed to 3rd September, does this mean that Google has visited my website??

However there are still only 2 url's in the web index, see the attached image.

Could it be that for some reason Google hasn't indexed my pages? Or do I simply have to wait a bit longer?

Thanks