R
R
Roman2016-04-18 03:03:41
Apache HTTP Server
Roman, 2016-04-18 03:03:41

How do I prevent a search bot from accessing a website?

Good afternoon!
Tell me how to write a rule at the .htaccess (Apache) level to block access to a particular bot?
The problem is that various bots have got into the habit, which give a strong load on the site, for example:
AhrefsBot/5.1 - that's just evil!
bingbot/2.0
MJ12bot/v1.4.5
etc.
After analyzing the log file, I saw that 90% of the site traffic is not the bots I need, but the load is large. Tomorrow the hosting provider will trample out of the yard.
Please help, thank you!
--------------------
Solution:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} .*bingbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*ahrefsbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*mj12bot.* [NC]
RewriteRule .* - [R=403,L]

Thanks Alexey

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alexey, 2016-04-18
@RGBPlus

stackoverflow.com/questions/29212810/how-to-fight-...

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (bingbot) [NC]
RewriteRule .* - [R=403,L]

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question