How can I exclude the search engine bots on a Coldfusion website?

Does anyone know if it's possible to exclude the search engine bots by using a robots.txt file on a Coldfusion website just like an HTML site?
Thanks,
Heidi

mheidi

Newbie Poster

13 posts since Dec 2008

Reputation Points: 10

Solved Threads: 0

2 Years Ago

0

From the definition:The robots.txt file can be used to stop a search engine spider from crawling all or part of your site. Create a robots.txt file in the main folder of your site which will direct the search engine spiders what they may and may not search. Spiders generally look for this file before doing anything.this seems to imply that it will. I know that it works for databases and such so I can see no reason that Coldfusion websites would be any different.

GrimJack

Posting Maven

2,920 posts since Feb 2004

Reputation Points: 1,448

Solved Threads: 22

2 Years Ago

0

Thanks so much for your response. I just wasn't sure if the robots.txt worked for databases.

mheidi

Newbie Poster

13 posts since Dec 2008

Reputation Points: 10

Solved Threads: 0

2 Years Ago

0

Does anyone know if it's possible to exclude ... on a Coldfusion website just like an HTML site?...

It might help to clarify that Coldfusion generated pages ARE html regardless of the ".cfm" extension. Coldfusion is simply a tool for dynamically generating the HTML (or XML or even PDFs and images, for that matter). CF on the server creates the content and then sends it out to the browser which interprets it in the same way that it interprets any other data sent from a server. In that respect CF is like PHP, ASP, ROR, etc. They all create HTML on the server.

The robots.txt file is one way to do it. You can also include a META tag in the HEAD of your page.

<html>
<head>
<title>...</title>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
</head>
<body>...</body>
</html>

gklandes

Newbie Poster

10 posts since Jul 2009

Reputation Points: 10

Solved Threads: 0

2 Years Ago

0

Just visit google webmaster tools & generate robots.txt for the same

nherbal

Newbie Poster

7 posts since Jul 2009

Reputation Points: 10

Solved Threads: 0

2 Years Ago

0

Google WM Tools is a good but a quick fix. Eventually you will want to understand how they work and how to use them more effectively. You can get a good grounding in them at ...
http://www.robotstxt.org/

gklandes

Newbie Poster

10 posts since Jul 2009

Reputation Points: 10

Solved Threads: 0

How can I exclude the search engine bots on a Coldfusion website?

This question has already been solved