When i see my page on Google webmaster tool, i see some crawler error on my blog. It seems to say that some links on my blog cannot be crawled by Google because that links was restricted by robot.txt.
It was confusing me. I ask my self, do i have make a robot.txt to my page? I have try tested robot.txt on Google webmaster tool, and successes. On that time i didn't check my robot.txt. Later i know that robot.txt contain such protection to my page, a stop sign to crawler in order to not crawled 'things' on my page that described by robot.txt.
That was make me feel sorry, i feel that i have make a mistake and i don't know what to do to fix it. Even that i try to change that robot.txt Google won't save it. It means that i can't do nothing about it. The only way to edit, remove or deleted it is i must have access, full access to my page, it means that i must owned the domain or self hosting. Impossible things because i hosted it on blogger.
But...after searching informations about it. I find that Google automatically insert robot.txt to blogger page, to my page especially to search links in order to avoid any duplicated links on that page. And that will not caused bad things on my page. Nothing to worry about... i feel so!
No comments:
Post a Comment