hello everyone....

i have one problem in my blog.... that i don't received organic traffic, andl also have all the content Crawl errors Restricted.

and i select from the setting of blog.... yes for visibility in search engine.

That is my file.... there's any error

User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: /search
Allow: /

Sitemap:http://motoreelettrico.blogspot.com/...rderby=updated

Is there really an independent robots.txt file for blogger? I was on the idea that there was none since Google hosts it and it'll only depend on the settings you've set for the search engine visibility, which looking at your case is enabled.

I believed the problem has nothing to do with the file but on whether or not, people are searching for the keywords you're targetting, hence directing them to your blog and getting you organic traffic.

Try and do a "site:" search for your blog in Google's SE, if it's there then there are no crawling errors, since your site is already indexed.

i can believe in that you say.... but i have all URLs restricted by robots.txt and i don't know why

Delete it or put an empty file there

I am Having the same issue in Blogger with one of my other sites.

I have figured it is due to the fact the labels at the bottom of the posts have their own HTML like for instance SITE://BLAHBLAH/LABEL1 and that page is not getting crawled.

I guess I shouldn't worry if the main posts are getting indexed as their title and HTML is getting crawled but still I don't like critical errors on my blog.

Strange I have a bunch of blogs and one has more errors than the other. Why would this be?

i have one problem in my blog.... that i don't received organic traffic,

Maybe it doesn't deserve any! Pages don't get SE traffic just for being there.

the posts have their own HTML

means what exactly??

BLOGGER give each blog post it's own little HTML addy unique to each post. POOF post and send it into cyberspace it's got it's own unique web address.

Users have options of adding labels. Repetitive labels then get assigned their own page of the blog, which is where my errors are occurring. We shall call these label pages.

Now the strange part is out of 5 blogs with label pages, two blogs are showing the robots.txt crawl error . One blog has errors and robots.txt on each and every label page. The other blog has robots.txt on one label page out of 100. Strange.

So does this mean the others' labels direct pages haven't gotten blocked yet? It's just a matter of time? Or that the robots are thinking these pages are duplicates of the original post ie: spam or what?

I want to know how those robots think. Also why did they stop calling them spiders.

Web document addresses are URLs for the HTTP URI schema.

HTML is the language that defines the structure and layout of a web document.

Each post does NOT have its own HTML document it is the SAME "HTML" that is accessible by multiple URLs. (Addresses).

The analogy of "robots" to spiders is simply because most species of spider create a "web" to inhabit or catch their prey in, and as SE bots "live" on the World Wide Web the name stuck.

The name of "crawler" is also incorrect as SE bots do not "crawl" from point to point they are instructed by other software agents to go to a particular location and retrieve the data.


The correct term for a software agent that retrieves data is a "user agent", as it is controlled by a user and in the case of SEs the "user" is the data retrieval scheduler (crawl queue in SE parlance)

Technically they are "robots" in the truest meaning of robot as they are independant automatons reacting to a programmed sequence of triggers and instructions.

Download it from where you have installed the theme or software.