T
T
triggerfinger2018-01-27 20:14:44
Nginx
triggerfinger, 2018-01-27 20:14:44

How to block majestic and ahrefs bots in nginx or robots?

How to do this effectively (I read that they can ignore robots)? And is it okay to block it at the nginx level (not robots.txt)? What is needed for this, user agent? Where to find him?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
T
Twelfth Doctor, 2018-01-27
@triggerfinger

Yes, you are right, this really requires the User-Agent of the bots.
For nginx configuration:

if ($http_user_agent ~* (AhrefsBot|majestic) ) {
    return 403;
}

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question