Member Avatar

What is Robot. Txt?
What is the use of it?

  • 5 Contributors
  • 4 Replies
  • 191 Views
  • 1 Day Discussion Span
  • Latest Post Latest Post by almostbob

Recommended Answers

All 4 Replies

Member Avatar

More details about Robot.Txt can be found at here moz(dot)com/learn/seo/robotstxt

Member Avatar

Robots.txt can be used to specify which webpages can be crawled by specific search engines.Also it can specify whether a webpage can be crawled by search engine or not.

Member Avatar

Note the Difference

robot.txt is a an exercise in futility

robots.txt is a file containing control directives to search robots

summer semester has started,
the beginner html course questions have been asked again.

be worried,
if the coursework texts have got the name of such a basic element wrong,
the rest of the course will be total crap

Edited by almostbob
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, learning, and sharing knowledge.