Member Avatar

What is Robot. Txt?
What is the use of it?

  • 5 Contributors
  • forum 4 Replies
  • 185 Views
  • 1 Day Discussion Span
  • comment Latest Post Latest Post by almostbob

Recommended Answers

Answered by pritaeas 2,030 in a post from

http://www.daniweb.com/internet-marketing/search-engine-optimization/threads/454394/robot.txt

Jump to Post

All 4 Replies

Member Avatar

More details about Robot.Txt can be found at here moz(dot)com/learn/seo/robotstxt

Member Avatar

Robots.txt can be used to specify which webpages can be crawled by specific search engines.Also it can specify whether a webpage can be crawled by search engine or not.

Member Avatar

Note the Difference

robot.txt is a an exercise in futility

robots.txt is a file containing control directives to search robots

summer semester has started,
the beginner html course questions have been asked again.

be worried,
if the coursework texts have got the name of such a basic element wrong,
the rest of the course will be total crap

Edited by almostbob
Be a part of the DaniWeb community

We're a friendly, industry-focused community of 1.20 million developers, IT pros, digital marketers, and technology enthusiasts learning and sharing knowledge.

Sign Up — It's Free!