|
My Error Logs show quite a few 404 entries.
The referer is usually google.gr, google.ee google.com.br etc.
Please keep in mind, this is not a case of broken links, renamed or removed files. The files in question never existed on the sites (as far as I know).
Also, the sites and the directories that the indexes point to have all robots.txt files with the following:
User-agent: *
Disallow: /
The sites are for private use and are not indexed by SEs. I am aware that bots can ignore the robots.txt files.
What concerns me is that the file names usually are something along the lines:
....serial-free.html
....CD-key-changer.html
...something-sex.html and so on.
Does anyone have any ideas about what's is going on? How do these end up in google index?
Thank you for your time.
|
