<meta name="robots" content="all"> does this tag tell the robot to spider the entire website and also what exactly goes into the robot.txt file


Thanks

Mark

Hi Mark,

Using a meta tag for robots and robots.txt file is one and the same thing - you either use the one or the other. You can check Brett Tabke's tutorial about robots.txt file at http://www.searchengineworld.com/rob...s_tutorial.htm or search Google for more samples.

Cheers,
Maggie

They are certainly NOT "one and the same thing" other than they are both exclusion protocols. Both perform similar but different functions, the meta is only for the page it is on while robots.txt is effective for the whole site.

content="all" is the default behaviour for bots so will have no effect. The only time the "robots" meta is required is when you need to tell the bots not to do something.

more info at http://www.robotstxt.org/wc/exclusion.html

Thanks Chris and Maggie