I am looking for a way to have a certain page on my website crawled at a preset intervals. Basically, I need these pages spidered so they can be cached by a script before they expire.

Does anyone know of a way to do this at preset intervals? It can be a program that I run on my own pc, or something I host, or remotely hosted. Just as long as I can set the amount of time between crawls.

Thanks in advance for your help.

Frank

if you meen crawled by spiders like google then i dont think there is as you get crawled when they want to

No, I just mean any spider going through the page and "hitting" all of the links on the page. That way it triggers the cacheing script, and it keeps a copy of the news article before it expires.

bump