|
I believe I shoukd not have it a validator- the above tool is an analyzer. When I take out the first line the User-agent returns an error. The (/) at the end passes the analyzer.
I took out both the final (/) and the first line.
so now:
User-agent: *
This line doesn't follow a correct syntax. The correct syntax is: <field>:<optionalspace><value><optionalspace> , where "field" can be "User-agent" or "Disallow". Please refer to Robots Exclusion Standard page for more informations.
Regardless of what is in the first line, this is the same error message. However, without another first line the rest of the script brings back errors - no valid User-agent
|