What is Robot. Txt?
What is the use of it?
More details about Robot.Txt can be found at here moz(dot)com/learn/seo/robotstxt
Robots.txt can be used to specify which webpages can be crawled by specific search engines.Also it can specify whether a webpage can be crawled by search engine or not.
almostbob
866
Note the Difference
robot.txt is a an exercise in futility
robots.txt is a file containing control directives to search robots
summer semester has started,
the beginner html course questions have been asked again.
be worried,
if the coursework texts have got the name of such a basic element wrong,
the rest of the course will be total crap
Be a part of the DaniWeb community
We're a friendly, industry-focused community of 1.18 million developers, IT pros, digital marketers, and technology enthusiasts learning and sharing knowledge.