Answer the question
In order to leave comments, you need to log in
How do I prevent a search bot from accessing a website?
Good afternoon!
Tell me how to write a rule at the .htaccess (Apache) level to block access to a particular bot?
The problem is that various bots have got into the habit, which give a strong load on the site, for example:
AhrefsBot/5.1 - that's just evil!
bingbot/2.0
MJ12bot/v1.4.5
etc.
After analyzing the log file, I saw that 90% of the site traffic is not the bots I need, but the load is large. Tomorrow the hosting provider will trample out of the yard.
Please help, thank you!
--------------------
Solution:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} .*bingbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*ahrefsbot.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*mj12bot.* [NC]
RewriteRule .* - [R=403,L]
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question