954,606 Members — Technology Publication meets Social Media
Username:
Password:
Lost login information?
Have something to say? Contribute New Article Reply to this Article

Robot.txt Suggestion

hi experts, using a robot.txt on a site. Will it help to increase the web traffic? or what are your thoughts and suggestion on this?

Thank you.

cguan_77
Nearly a Posting Virtuoso
1,317 posts since Apr 2007
Reputation Points: 19
Solved Threads: 115
 

I thought that a robots.txt file was just to prevent search engines from crawling through folders they shouldn't. I don't think it will increase web traffic. I could be wrong though but that's my understanding of it.

Borzoi
Posting Whiz
324 posts since Oct 2006
Reputation Points: 21
Solved Threads: 30
 

Having a neat robot.txt file, may help a bit with SEO. As Borzoi said, it allows/disallow robots to crawl certain files or folders on your website. If you use the robot.txt file cleverly, you can prevent robots from crawling duplicate content on your site, and this will help with site SEO.

I agree, it will not neccesarily mean more visitors, but will improve your site visibility on search engines if coded correctly, resulting in maybe more visitors.

Kraai
Senior Poster
3,981 posts since Feb 2008
Reputation Points: 76
Solved Threads: 87
 

Thanks :)

cguan_77
Nearly a Posting Virtuoso
1,317 posts since Apr 2007
Reputation Points: 19
Solved Threads: 115
 

This question has already been solved

Post: Markdown Syntax: Formatting Help
You
View similar articles that have also been tagged: