Our website have a dynamic sitemap that we submit to Google

its a dynamic file it is generated daily and we enter the current date of lastmod and changefreq is daily. However its not a sitemap on our site for Google to find and crawl naturally our sitemap is a submission to Google to crawl, it is not on our site, no link to it from our site and it is just a file for Google and yahoo to use.

Each of the sitemap linked from the index sitemap has different numbers of url and all of them do not get indexed completely
for example; one sitemap has 2,432 urls, but only 2,165 got indexed sometimes 101 out of 105 or 591 out of 726 it's really random and no pattern. But for some reason, it crawls 100% on some but about 75% on most.

My question, How can my dynamic sitemap achieve 100% crawl rate from Google?
Any help would be much appreciated, thanks

I think no one know why, just google knows.

Originally Posted by vnsmith

Our website have a dynamic sitemap that we submit to Google

its a dynamic file it is generated daily and we enter the current date of lastmod and changefreq is daily. However its not a sitemap on our site for Google to find and crawl naturally our sitemap is a submission to Google to crawl, it is not on our site, no link to it from our site and it is just a file for Google and yahoo to use.

Each of the sitemap linked from the index sitemap has different numbers of url and all of them do not get indexed completely
for example; one sitemap has 2,432 urls, but only 2,165 got indexed sometimes 101 out of 105 or 591 out of 726 it's really random and no pattern. But for some reason, it crawls 100% on some but about 75% on most.

My question, How can my dynamic sitemap achieve 100% crawl rate from Google?
Any help would be much appreciated, thanks

Interesting. I don't think there's a problem with the way the sitemap is generated. It's not really dynamic. A dynamic sitemap would be generated on the fly as it was read.

Anyhow, first question: do you have Google's Webmaster Tools installed? If not, do so. It may give you an indication as to what the problem is. Maybe duplicate content? Not found URLs? Install them so you can eliminate those.

Secondly, have you checked the sitemap through thoroughly to make sure there are no errors? It sounds like there might be errors in it, or errors with the scripts that generates the sitemap.

Google usually index original and unique pages according to its algorithm but not all pages which you submited.

Originally Posted by edgray

Interesting. I don't think there's a problem with the way the sitemap is generated. It's not really dynamic. A dynamic sitemap would be generated on the fly as it was read.

Anyhow, first question: do you have Google's Webmaster Tools installed? If not, do so. It may give you an indication as to what the problem is. Maybe duplicate content? Not found URLs? Install them so you can eliminate those.

Secondly, have you checked the sitemap through thoroughly to make sure there are no errors? It sounds like there might be errors in it, or errors with the scripts that generates the sitemap.

Thanks for the reply.. I'll do that and post back here as soon as I got the result..
Thanks a lot for the help..