hey,

now i had a thought which frankyl only i could have.

i made thread about googlewhacking a while ago, and now i want to know how i could go abotu making a googlewhack bot.

i think i know the kind of processes bur have no clue how to do it.

firstly i would need to get EVERY single word and a link to that word from Dictionary.com
save it in a Database.

then have the script run and google two words at a time form this database of words. doing all of them in all combinations.

and then saving all the combinations (and a link to the search page) of the combinations which get ONE result ONLY into another table

now i have no clue how to go about getting the info and that from both google and dictionary.com

so how would i create my googlewhacking bot?

Dan

You should read this first
http://www.google.com/support/webmas...n&answer=66357

The making of a crawler bot is very simple. Here's one in vBScript

function GetPageCode(strURL)
 Response.Buffer = True
  Dim objXMLHTTP, xml
  Set xml = Server.CreateObject("Microsoft.XMLHTTP")
  
 	xml.Open "GET", strURL, False
	xml.setRequestHeader "User-Agent", strUserAgent 
	xml.Send
	if xml.status = 200 then
	GetPageCode = xml.responseText
	else
	GetPageCode = cstr(xml.status) & " (" & xml.StatusText & ")"
	end if
  Set xml = Nothing
end function

returns the source code if the response is 200 or the response code and message for any other response.
(From my "spider" code class, used when looking for cloaking etc)

Scraping something like dictionary.com for a word list can't be done as they don't appear to have one So buying a word database would be the best option and it would be cheaper than the "gone over the monthly bandwidth limit" costs

erm **** where could i get a word list for free?

the thing is this would be the general Bot.

im not sure if you know the whole idea and that of googlewhacking,

so if some how i get a list of words,

would could i then get it to query every combination against google, and the if the response is 1 save to a DB

if i could like get it to like

$word1 $word2 $linktogoogleresult

like that i could save it to a table, its just i have no idea how to interact in this way to another website and essentially another database.

Thanks,

Google won't allow you to send automated queries like you want to, especially on the scale you're talking about.

surely there must be a way

Of course there is; there are plenty. Chris gave you one.

You could go the don't-get-your-site-banned route and use the Google API to perform the queries, using AJAX, so you'll have to be a little more clever to persist the data into your relational backend.

You can probably get an open source word list from a spell checker. Firefox for example; you'd have to figure out where and how the built-in dictionary is stored, but if you can code the system you're describing, you should be able to grab the code and find their word list.

thats the problem! i dont know properly, which would be why im asking.