How to prevent robots.txt from being viewed by other than search engines?

I

Igor2016-11-29 02:52:50

htaccess

Igor, 2016-11-29 02:52:50

It is necessary to prohibit viewing robots.txt, in .htaccess register deny from all for it. But for certain search engines, access is allowed. Detect bots by HTTP_USER_AGENT, for example %{HTTP_USER_AGENT} !^yandex.*
How to create such a rule in .htaccess ? - forbidden for everyone (403), except for specific bots.

Reply

Answer the question

In order to leave comments, you need to log in

1 answer(s)

M

Moskus, 2016-11-29
@Moskus

SetEnvIfNoCase User-Agent .*google.* search_robot
SetEnvIfNoCase User-Agent .*yandex.* search_robot
SetEnvIfNoCase User-Agent .*bot.* search_robot

Order Deny,Allow
Deny from All
Allow from env=search_robot