Hey guys,

I'm looking for a script or other type of software program that allows me to crawl my ecommerce website in order to generate a CSV or XML datafeed with all my products.

Unfortunately my company is using a pretty ol JSP based software and there is not exporting function for the database.

So I thought I can include certain comment-tags into my website and have a software extract all the product info such as price, description, category etc.

Does anyone know any good software tools? Or is there another way to do it?

Thanks a lot for your help!!

Kai

Are your products stored in a database? It might be easier to write a quick script to pull the products out of the db

Originally Posted by Anacrusis

Are your products stored in a database? It might be easier to write a quick script to pull the products out of the db

Yeah, i know it sounds much easier, but we are getting an encrypted datafeed from several external suppliers. i think spidering the website is less complicated :-( any suggestions?

THANKS
kai

Maybe this is longwinded...

But have a tag like this:

<! prodInfo="thiswillbeafixedlength" !>

Then in your crawling page find the line that equals

<! prodInfo="??????????????????????" !>

and your gonna have to find a way in which you can tell your script that '?' equals any character.



Maybe thats just outta me bum :P

Hi,

we are still looking for someone that can help us, crawl our pages. Anyone know a good tool/software or company that can help?

THanks,

kai

I think spidering the website is easier

If your server supports ASP, you could crawl the site using ASPTear. I haven't tried the commercial component yet, but the free one's not bad (although it can run a bit heavy sometimes.)