The video clearly shows that Google sometimes lists your blocked urls, but shows no description, meaning that they have not “crawled” those urls but still list them as they found them, thereby abiding by robots.txt. Sometimes they extract the description from the ODP, and not by actually crawling the url. So many times these Google search results are just an ‘uncrawled url reference’
Happy Blogging ! http://nitinmaximumhit.blogspot.com "We brand our clients to stand out from the competition."
Nitin Chauhan | nitin@maximumhit.com | www.maximumhit.com
Posts
No comments:
Post a Comment