What's the significance of Robots.txt file in a website?
Jump to Post
To tell bots (such as Googlebot) and other spiders and crawlers what parts of your site to avoid.
All 5 Replies
Be a part of the DaniWeb community
We're a friendly, industry-focused community of 1.20 million developers, IT pros, digital marketers, and technology enthusiasts learning and sharing knowledge.