Answer the question
In order to leave comments, you need to log in
.htaccess settings - how to disable indexing from all bots?
Hello, tell me how to fill in .htaccess correctly?
The task is to prevent the bot from seeing my site
if you fill it out like this:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (Googlebot) [NC]
RewriteRule .* - [R=403,L]
Everything works, you go under Google bot, then it does not see the site, but sees an error 403, but if you make a large list like
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (Googlebot) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (google) [NC]
RewriteRule . * - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (HPWebPrintSmart) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (HPWeb PrintSmart) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (HP WebPrintSmart) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (HP Web PrintSmart) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (AskJeeves) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (Ask Jeeves) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (HTTrack) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (Indy Library) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (Mediapartners) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (RPT-HTTPClient) [NC]
RewriteRule .* - [R=403,L]
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (ListChecker) [NC]
RewriteRule .* - [R=403,L]
or more, then error 500 crashes and the site does not work at all,
how to list all the bots correctly?
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question