Hi

This is my first post in the Google Forum, so please accept my appologies if I am making no sense

The question I would like to ask is: Can GoogleBot successfully index dynamic URLs?

The reason I ask is because I have recently started a website related to Diabetes, and I am using the PhpNuke engine as the backbone to my site as a 'quick start' option. I plan on modifying the script highly to fit the needs of my site once membership grows.

I have so far managed to get my site indexed into Google within 3 days, but I am just wondering if ALL my pages that will be added later will be indexed also?

By a dynamic URL i mean something like the one used for my web links page at http://www.diabetesinformation.co.uk...name=Web_Links where the ?name=Web_Links is the dynamic part of the URL and the actual page I require being indexed.

I would be very interested to hear your thoughts on this

Kind Regards,

Craig Wistow

Almost never.

Google usually comes to a halt once it reaches the "?" tag. Not always, but most of the time. It is best to (in php, coldufsion, asp, etc) look up Search Engine friendly urls and implement a code that can change ?, =, and & tags to "/" so that search engines think it is a regular page.

Goodluck wiht that.

Does anyone know of such a script?

Or am I better off writing one myself?

Kind Regards,

Craig

Why does everyone have this misconseption that google will not crawl dynamic urls.

Here is a recent search I performed Google?

Originally Posted by Google

Your pages are dynamically generated. We are able to index dynamically generated pages. However, because our web crawler can easily overwhelm and crash sites serving dynamic content, we limit the amount of dynamic pages we index.

http://www.google.com/webmasters/2.html#A1

The thing is that Google may not index all of your dynamic pages. What it really looks for is the use of a session id in the url. A session id confuses google bot and tells it that each time it comes its getting a new page (as the url now contains a different session id).

Google comes to my site on a daily basis. They have indexed over 5000 of my dynamic pages. I am not sure, but this is my guess. Make your entry page into an html or shtml page, and include your php scripts using the include tag. Also, put a <meta name="REVISIT-AFTER" content="1"> tag on the main page, and preferably on all headers. For some reason, google likes that.

Good luck.

Frank

There is a script called googletap which uses mod rewrite in a .htaccess file to change the long dynamic nuke style URLS to /file.html style links. I got it from http://www.nukecops.com

Originally Posted by DKasler

There is a script called googletap which uses mod rewrite in a .htaccess file to change the long dynamic nuke style URLS to /file.html style links. I got it from http://www.nukecops.com

My site was slowly spidered by google, when I used mod_rewrite to make SE friendly URL's, it was spidered much faster.

Originally Posted by neorunner

Also, put a <meta name="REVISIT-AFTER" content="1"> tag on the main page, and preferably on all headers. For some reason, google likes that.

In another SEO Forum there has been discussions about whether or not this tag actually works. I personally beleive that it probebly doesn't due to Google's frequent crawling anyway and have stoped using the tag completly. After looking through the logs I have noticed that there has been no drop in the frequency of which Google visits.

What I tell clients are if you want Google to visit every day give it something new to find each day and used to come everyday.

In more recent times some people have been reporting (not noticed anything my self) Google has slowed somewhat in visiting sites, which has led to much speculation about another Florida type up-date on the way!

How are people here finding the frequency of Googlebot?

I have found that Googlebot has been slow and infrequent lately. Yahoo and MSN have been crawling me faster or more often which never happened to me before.