942,962 Members | Top Members by Rank

Internet Marketing > Search Engine Optimization > robots.txt file syntax
Ad:
Permalink
Aug 28th, 2010
0

robots.txt file syntax

Expand Post »
hi,
i have a robots.txt file with this content:
User-agent
Allow: /
Allow: /sitemap
Disallow: /gerente

is it written in right order?
Similar Threads
Reputation Points: 10
Solved Threads: 1
Junior Poster in Training
fumnimda is offline Offline
60 posts
since Oct 2009
Permalink
Aug 28th, 2010
0
Re: robots.txt file syntax
Hi,

I think these 2 lines would be not useful Allow: / Allow: /sitemap for any purpose since they are allowed by default. Also, might I suggest the contents on robotstxt.org/orig.html#examples for you to read.

all the best,
Reputation Points: 19
Solved Threads: 14
Posting Pro in Training
AirForceOne is offline Offline
457 posts
since Jun 2009
Permalink
Aug 29th, 2010
0
Re: robots.txt file syntax
My robots.txt file looks like this:

Sitemap: http://www.daniweb.com/sitemap.xml

User-agent: Googlebot
Disallow: ...........
It sits in the root directory. I think that Allow is explicit.
Last edited by cscgal; Aug 29th, 2010 at 9:56 pm.
Administrator
Staff Writer
Reputation Points: 1422
Solved Threads: 162
The Queen of DaniWeb
cscgal is offline Offline
13,645 posts
since Feb 2002

This thread is solved

Either the thread starter or a moderator has marked this thread as solved. You can most likely trust the responses and answers given. There is most likely no reason for any further responses to be posted here. If you have a related question, please start a new thread in this forum instead.

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in Search Engine Optimization Forum Timeline: Effective Tips to Improve Website Traffic
Next Thread in Search Engine Optimization Forum Timeline: SEO for blogs, better to use plug-ins for Wordpress or to do the SEO manually





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC