1,105,371 Community Members

Role of Robot.txt in seo

Member Avatar
ilmkidunya
Light Poster
43 posts since Dec 2011
Reputation Points: -3 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

What is robot.txt file and what role it play in the seo. and how can we create it.

Member Avatar
Tiggerito
Junior Poster in Training
58 posts since Jul 2010
Reputation Points: 0 [?]
Q&As Helped to Solve: 5 [?]
Skill Endorsements: 0 [?]
 
0
 

The file lets you request robots not to index certain files.

With relation to SEO, it helps you control which pages get indexed. So you can have pages that have no SEObenefit excluded from indexing. Also, restricting the number of pages that can be indexed increases the chance that your valuable pages will get indexed more frequently.

There are 1,000s of articles on the web to explain how to write one.

Member Avatar
claydcousta
Junior Poster in Training
64 posts since Jun 2011
Reputation Points: -3 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

Robots.txt file is used to tell the search engines which pages to be indexed by and which pages not to index. Robots.txt is file is used to block the admin pages in the site

Member Avatar
ilmkidunya
Light Poster
43 posts since Dec 2011
Reputation Points: -3 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

Thanks for your value able information, just one more question,

how can i create the text file is there any software for that or manually creates the file thanks

jhonden
Newbie Poster
17 posts since Sep 2011
Reputation Points: -15 [?]
Q&As Helped to Solve: 4 [?]
Skill Endorsements: 0 [?]
Banned
 
0
 

Robot.txt helps you to cover or not to show any files to search engine. It hides it from search engine.

Braylen
Newbie Poster
22 posts since Dec 2011
Reputation Points: -9 [?]
Q&As Helped to Solve: 1 [?]
Skill Endorsements: 0 [?]
Banned
 
0
 

Robots.txt file is a set of instructions that tell search engine robots which pages of your site to be crawled and indexed. In most cases your site is consist of many files or folders i.e. admin folders, cgi-bin, image folder, which are not relevant to the search engines. Robots.txt helps tell spiders what is useful and public for sharing in the search engine indexes and what is not. Robots.txt file is to improve site indexation by telling search engine crawler to only index your content pages and to ignore other pages (i.e. monthly archives, categories folders or your admin files) that you do not want them to appear on the search index.

jonywags
Newbie Poster
18 posts since Dec 2011
Reputation Points: -16 [?]
Q&As Helped to Solve: 1 [?]
Skill Endorsements: 0 [?]
Banned
 
0
 

Robots.txt file is used to tell the search engines which pages to be indexed by and not to index. Means this is the file where we can allow to searching engine what you have to follow or what not.

Create Robot txt file:

Member Avatar
ilmkidunya
Light Poster
43 posts since Dec 2011
Reputation Points: -3 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

Thank you jonywags is solves my problem....

vinodkunwar
Newbie Poster
23 posts since Dec 2011
Reputation Points: -12 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
Banned
 
0
 

Hi friends, robot.txt file basically used to tell search engine which page crawl and which are not.................Thanks,

Member Avatar
trishia
Light Poster
25 posts since Dec 2011
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

What is robot.txt file and what role it play in the seo. and how can we create it.

They retrieved link pages to your website, they are the ones telling the google spiders what to visit and what are not.

Member Avatar
dailyearner
Junior Poster
141 posts since Apr 2009
Reputation Points: -16 [?]
Q&As Helped to Solve: 5 [?]
Skill Endorsements: 0 [?]
 
0
 

it instruct search engines robot to crawl your website

Member Avatar
pankajrocks
Newbie Poster
22 posts since Dec 2011
Reputation Points: -18 [?]
Q&As Helped to Solve: 3 [?]
Skill Endorsements: 0 [?]
 
0
 

Robots.txt file is used to block any your of a websites which you don't want to get index by Search Engines.

Member Avatar
ADRIANEra
Junior Poster in Training
56 posts since Nov 2011
Reputation Points: -15 [?]
Q&As Helped to Solve: 5 [?]
Skill Endorsements: 2 [?]
 
0
 

robots are performing the link building.mostly doing the off site

You
This article has been dead for over three months: Start a new discussion instead
Post:
Start New Discussion
View similar articles that have also been tagged: