Hi,

What is the proper way of listing multiple "User-agent: " in a robots.txt file?

a)

User-agent: *
Disallow: /
User-agent: Slurp, Googlebot
Disallow: /private

In robots.txt example a (above): Are all engines blocked from seeing everything; including Slurp, and Googlebot? Since the wildcase says every browser are disallowed from accessing anything than that includes Slurp, and Googlebot as well? And is it correct to list multiple crawlers with comma separation?

In robots.tzt example b (below): Are all engines blocked from seeing everything; including Slurp, and Googlebot? Since the user-agent (line separated) says Googlebot, and Slurp can access everything but /private, but then again the wildcard says all engines are blocked from seeing anything. And is it correct to list multiple crawlers with line separation?

b)

User-agent: Slurp
User-agent: Googlebot
Disallow: /private
User-agent: *
Disallow: /

Please advice.

individually

you can group Disallow: entries for each User-agent: entry

User-agent: Googlebot
Disallow: /folder/
Disallow: /folder/filename.ext

But NOT group the User-agent: entries.

IF you want to allow certain bots, then ban all other compliant bots. Your wildcard block should be the last entry. The file should be read and applied top to bottom and the least restrictive entry should be applied by the bot.

Why can I not group different user-agents?

because you can't! The protocol doesn't allow for it.

If you want to rewrite the protocol and get everyone to follow the new spec, fine. But for now you will have to follow the existing protocol specifications.