A
A
Andrey2020-05-03 12:43:43
Nginx
Andrey, 2020-05-03 12:43:43

How to change pages on the fly that are given to bots that make too many requests?

Now too active bots and parsers are cut off through limit_req_zone $binary_remote_addr zone=one:10m rate=2r/s;
I want to make sure that the pages are given to them, but with distorted information. Let's say replacing text in a certain block with another.
Or another way - can Nginx add a specific HTTP header to requests from such bots? Then I will change the output at the web application level.

Answer the question

In order to leave comments, you need to log in

4 answer(s)
D
dodo512, 2020-05-03
@AndreyMorozov

limit_req_status + error_page

limit_req_status 403;
error_page 403 = /file.php?bot;

K
ky0, 2020-05-03
@ky0

Or another way - can Nginx add a specific HTTP header to requests from such bots? Then I will change the output at the web application level.

Certainly. Send bots to another location, which, when proxying inside, will add a header or even redirect requests to a different url or port. Directive proxy_set_header.

A
Andrey_Dolg, 2020-05-03
@Andrey_Dolg

And what's the point of processing requests from bots? Or is it so to speak from the category of annoy?

A
Alexey Sundukov, 2020-05-03
@alekciy

rate=2r/s

Taking into account the fact that you do not have an API service, but a website, the specified value is too small. On average, a legitimate visitor, not a bot, can generate up to 6-12 RPS on the site.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question