I need to crawl information from other sites, dynamically.

d = new Document();

throws this exception:

Error: uncaught exception: [Exception... "Object cannot be created in this context"  code: "9" nsresult: "0x80530009 (NS_ERROR_DOM_NOT_SUPPORTED_ERR)"  location: "file:///home/aatiis/path/to/file/example.html Line: 9"]

Of course, I could use iframes, but then the browser loads the complete page, with scripts, styles, images, plugins, etc... I only need the data in that html. How do I get it using as little bandwidth as possible?

Can you just confirm that nothing underhand is taking place here - i.e. stealing someone else's content or something?

Actually I'm glad you asked. I'd like to use someone else's search results. It's not Google, so there's no API available. If I put on my page that I use eg. Yahoo's or Google's search results, is it ok? Or need I ask the mentioned company? I thought the search results are not copyrighted, as there are other "parasite" search engines on the internet. If I'm wrong, please show me where could I read more about this matter?

Thank you for warning me anyway.

If you are planning on earning anything from this site or getting a large number of visitors, it would be a very bad idea not to ask the company themselves.

I'm not a Javascript developer so I can't answer your original question, but I'm sure there would be someone here who could.

Hope this helps,
Dan

Thank you for your help, Dan. Right now I'm trying to solve the problem whit a cute little javascript library called jQuery. I'm not planning to earn big bucks, just coding around... But if my site ever gets more than 10 visitors a week, I'll consult the companies. Basically, I think it's a bad idea to crawl data from search engines, strip out their ads and place my own ads on the site. It sounds like stealing for me. Anyway, If I manage to do something useful, I'll consult someone before publishing anything.

I'm fairly sure that this (opening files on remote servers) is not allowed in JavaScript due to odd security reasons... You could load a JavaScript file from a remote server through HTML (e.g., <script src="http://example.com/js.js...). This is how Google's AJAX search API works- their JavaScript is hosted on their server, so they are able to use the AJAX on their own servers.

The only ways I know of to load files like this is through tags like <script> or <iframe> like you mentioned. You would have much more flexibility using a server-side language like PHP, or perhaps PHP in conjunction with AJAX.