Hello,
I've seen posts all over these forums, some so long ago, it was called 'EarnersForums'. They all say that spiders are easy to make, that all they do it grab the contents of a web page, and then stick it in a database, which is then searched.
I, however, have no idea on how to make one.
I've heard of Sphider, and it looks pretty good, but I don't need something that complicated to do what I need. What I need it to do is simply look through a flat file database (let's call it database.php), and take all the entries in it (separated by a '|') and index them. I'll work out keywords after. For now, I just need the title.
The problem is though, where this content in database.php shows up is a page that uses URL parameters to know what chunk of content ot display. Example:
http://www.example.com/page.php?content=2
Which would display my second content 'chunk'.
I need the spider to index that page, with that URL, parameters and all, because if the parameters aren't there, then the link would lead to the home page of page.php, not page.php?content=2.
So far, I have found no tutorials, no hints, nothing. In case you don't know, I'm the kind of person that has to make everything for themselves. I don't like to download other's work. I make it myself, as a part of a learning experience
, I guess.
I'm not asking for a spider to be written for me in a forum post (and I wasn't asking about the CMS thing, either
), but some simple framework, in english, not code.
Thank you, I hope this isn't too big of a task to ask for
-PG
that's where it can get complicated.
