How to block all bots?

Question

Nginx

You need to configure Nginx to block any clients that write to the user-agent that they are bots (for example, search engines, social networks, and so on).

Answer 1

Add a map to the http section, something like:

map $http_user_agent $isbot {
        default 0;
        ~*Googlebot 1;
        ~*bingbot 1;
        # и так далее
}

And in the server section:

if ($isbot) {
        return 403;
}

PS
Uncovering the possibilities of map in nginx
About using regexp in map nginx

Answer 2

Easily searched on stackoverflow. It works, of course, only for those who present themselves honestly - and there are usually no problems from such people. What problem are you solving?

Answer 3

Normal robots follow the rules specified in robots.txt