Member Avatar for thetechguy

How would I go about copying a robots.txt file from a webserver
and then displaying that information?

  • 2 Contributors
  • 1 Reply
  • 95 Views
  • 12 Hours Discussion Span
  • Latest Post Latest Post by pzuurveen
Member Avatar for pzuurveen

robots.txt usaly is not protected
just use www.thesite.com/robots.txtin your brouwser

Edited by pzuurveen because: n/a
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.