|
They aren't talking about robots.txt, they are talking about meta tags.
Like WilliamC says, the default is that robots do index pages and do follow links, making the "index,follow" redundant.
The "revisit-after" bit is to tell robots not to come back for a specific period of time e.g. don't crawl my page more than once every 2 weeks. It isn't an instruction to make them come back more often.
|