Hi,
I have the following entry in my Awstats file:
Unknown robot (identified by 'bot*')
How can I block this bot? I tried the following separately but none of them seems to be catching it:
RewriteCond %{HTTP_USER_AGENT} ^bot*
RewriteCond %{HTTP_USER_AGENT} bot\*
RewriteCond %{HTTP_USER_AGENT} bot[*]
Thanks for any ideas.
There's a module to block bad bots. You might want to try something like this:
Options All -Indexes
RewriteEngine on
<IfModule mod_setenvif.c>
SetEnvIfNoCase User-Agent "^bot.*" bad_bot
<Limit GET POST PUT>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>
</IfModule>
Here's some info and a huge list of bad bots:
http://www.allthingsdemocrat.com/htaccess-security-and-the-largest-block-bots-list-on-the-web/
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.