This is the Robots.txt of my website, user-agents determine the bot, i.e., if its google or bing or whatever. Allow tabs tells which part of the domain to be indexed by the SEs. Disallo part orders the bot to not crawl the mail directory.
[voice=porky]that's all folks[/voice] gotta make a joke sometime
robots.txt is just a text file, notepad will do it, the content is simple there are dozens of short tuts on what it needs to contain to accomplish the litany it is capable of
and being a text file is real easy to reset if something goes awry