What is Robot. Txt?
What is the use of it?

Recommended Answers

All 4 Replies

More details about Robot.Txt can be found at here moz(dot)com/learn/seo/robotstxt

Robots.txt can be used to specify which webpages can be crawled by specific search engines.Also it can specify whether a webpage can be crawled by search engine or not.

Note the Difference

robot.txt is a an exercise in futility

robots.txt is a file containing control directives to search robots

summer semester has started,
the beginner html course questions have been asked again.

be worried,
if the coursework texts have got the name of such a basic element wrong,
the rest of the course will be total crap

Be a part of the DaniWeb community

We're a friendly, industry-focused community of 1.20 million developers, IT pros, digital marketers, and technology enthusiasts learning and sharing knowledge.