User Name Password Register
DaniWeb IT Discussion Community
All
What is DaniWeb IT Discussion Community?
You're currently browsing the Linux Servers and Apache section within the Tech Talk category of DaniWeb, a massive community of 391,592 software developers, web developers, Internet marketers, and tech gurus who are all enthusiastic about making contacts, networking, and learning from each other. In fact, there are 2,717 IT professionals currently interacting right now! Registration is free, only takes a minute and lets you enjoy all of the interactive features of the site.
Please support our Linux Servers and Apache advertiser:
Views: 2420 | Replies: 2
Reply
Join Date: Jul 2005
Posts: 1
Reputation: design1 is an unknown quantity at this point 
Rep Power: 0
Solved Threads: 0
design1 design1 is offline Offline
Newbie Poster

.htacess Is Not Working

  #1  
Jul 12th, 2005
My Robots.txt Was Not Blocking Out Sites From Any Search Engine. A Friend Suggested That I Should Create An .htaccess File

During A Web Search Using Google, Yahoo, Excite, Msn, Altavista, Hotbot, Infoseek, Lycos, Overture, Ask Jeeves, Teoma, Or Other Search Engines, I "do Not" Want These Sites To Be Listed. Can You Help Me To Improve My Search So These Cannot Show?

The Administrators Will Not Remove The Postings.
Attached Files
File Type: zip HTACCESS.zip (1.6 KB, 11 views)
AddThis Social Bookmark Button
Reply With Quote  
Join Date: Aug 2005
Posts: 32
Reputation: jusCk is an unknown quantity at this point 
Rep Power: 3
Solved Threads: 0
jusCk's Avatar
jusCk jusCk is offline Offline
Light Poster

Re: .htacess Is Not Working

  #2  
Sep 4th, 2005
if You're wanting to totally Block ALL Robots, from MY understanding of the file, Youshould only need these two lines.

user-agent: *
disallow: /

otherwise, Note the changes I made in Your htaccess file
post back, lemme know how things go for ya
Attached Files
File Type: zip Updated htaccess.zip (1.6 KB, 5 views)
Originally Posted by Ck
Ye Men of Honour, Knowledge, Wisdom, and Truth:
Be Dead To This World
Reply With Quote  
Join Date: Sep 2005
Posts: 9
Reputation: bsmith is an unknown quantity at this point 
Rep Power: 0
Solved Threads: 0
bsmith bsmith is offline Offline
Newbie Poster

Re: .htacess Is Not Working

  #3  
Sep 21st, 2005
If you are using a global header you may try using this meta tag as well:
<META NAME="ROBOTS" CONTENT="INDEX,NOFOLLOW">
But really you really only need to use that when you have files in a directory that you don't want to put in your robots.txt file, and want to have a tidy robots file.
not sure what using .htaccess gains you, unless you are willing to play around with rules based on user agents, which may end up blocking alot of traffic you don't really want to and also would take alot of work to keep up with all the different robot agents out there. Not trivial.
if your friend meant to do an .htaccess to password protect it that's a different matter and kind of a different issue, but what jusCk said in his post about the robots.txt should really be sufficient unless you're running into bots with bad etiquette.
Reply With Quote  
Reply

Only community members can participate in forum threads. You must register or log in to contribute.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

 

DaniWeb Linux Servers and Apache Marketplace
Thread Tools Display Modes

Similar Threads
Other Threads in the Linux Servers and Apache Forum

All times are GMT -4. The time now is 11:26 pm.
Forum system based on vBulletin Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
©2003 - 2008 DaniWeb® LLC