hi experts, using a robot.txt on a site. Will it help to increase the web traffic? or what are your thoughts and suggestion on this?
Thank you.
cguan_77
8
Nearly a Posting Virtuoso
Recommended Answers
Jump to PostI thought that a robots.txt file was just to prevent search engines from crawling through folders they shouldn't. I don't think it will increase web traffic. I could be wrong though but that's my understanding of it.
All 3 Replies
Borzoi
24
Posting Whiz
I thought that a robots.txt file was just to prevent search engines from crawling through folders they shouldn't. I don't think it will increase web traffic. I could be wrong though but that's my understanding of it.
Kraai
56
Senior Poster
Featured Poster
Having a neat robot.txt file, may help a bit with SEO. As Borzoi said, it allows/disallow robots to crawl certain files or folders on your website. If you use the robot.txt file cleverly, you can prevent robots from crawling duplicate content on your site, and this will help with site SEO.
I agree, it will not neccesarily mean more visitors, but will improve your site visibility on search engines if coded correctly, resulting in maybe more visitors.
cguan_77
8
Nearly a Posting Virtuoso
Thanks :)
Be a part of the DaniWeb community
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.