N
N
newaitix2018-07-24 15:57:41
Nginx
newaitix, 2018-07-24 15:57:41

Protection from bad bots / ddos ​​protection?

About 50 bots come to the site (online store) every two hours and make from 2000 to 3000 requests within 5-10 minutes.
Here is an example of what it looks like

94.141.177.70	94	141	177	70	15/Jul/2018 06:36:21	/category/shuttle-receivers/	301	522
54.37.74.196	54	37	74	196	15/Jul/2018 06:36:21	/category/pioneer-native-receivers/	301	529
54.37.74.196	54	37	74	196	15/Jul/2018 06:36:21	/category/native-receivers/	200	26882
94.141.177.70	94	141	177	70	15/Jul/2018 06:36:21	/category/receivers/	200	23986
80.211.160.11	80	211	160	11	15/Jul/2018 06:36:22	/category/pioneer-marine-audio-speakers/	200	15484
45.115.176.251	45	115	176	251	15/Jul/2018 06:36:22	/category/pioneer-media-receivers-stations/	200	20535
54.37.74.196	54	37	74	196	15/Jul/2018 06:36:23	/product/marine-speakers-pioneer-ts-mr1600/	200	15811
94.141.177.70	94	141	177	70	15/Jul/2018 06:36:23	/product/cd-usb-receiver-jvc-kw-r500eyd/	200	18492
80.211.160.11	80	211	160	11	15/Jul/2018 06:36:23	/product/native-reciever-gazer-cm182-re5-for-honda-civic-2012/	200	22725
94.141.177.70	94	141	177	70	15/Jul/2018 06:36:23	/product/pioneer-deh-s3000bt/	200	17935
54.37.74.196	54	37	74	196	15/Jul/2018 06:36:23	/product/marine-speakers-pioneer-ts-mr1640/	200	15987
120.138.102.45	120	138	102	45	15/Jul/2018 06:36:23	/category/pioneer-car-audio-speakers/	200	21485
80.211.160.11	80	211	160	11	15/Jul/2018 06:36:23	/product/native-reciever-gazer-cm6008-1k5-android-for-volkswagen-skoda-seat-2008-2017/	200	21173
45.115.176.251	45	115	176	251	15/Jul/2018 06:36:23	/product/pioneer-mvh-x580bt/	200	17638
54.37.74.196	54	37	74	196	15/Jul/2018 06:36:23	/product/marine-speakers-pioneer-ts-mr2040/	200	15913
94.141.177.70	94	141	177	70	15/Jul/2018 06:36:23	/product/sony-cdx-g1200u/	200	18199

Pay attention to ip 94.141.177.70 in 3 seconds, he made 3 requests to different pages. The logical question is why?
The average person came in, looked at the pictures, read a couple of lines, and went to another page, which would likely take more than 3 seconds.
There is a way to identify these bots, they all appear as Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:55.0) Gecko/20100101 Firefox/55.0
There are 3000 unique ips in 2 days.
90,000 requests come to the site per day with such an identification string. At the same time, some incomprehensible attempts are made to do something incomprehensible like /os.aspx?name=ogFoot, although I do not have such a file and never have.
And the server during these 5-10 simply goes crazy load grows to the maximum. It is not clear who is doing what, or they want to sink the site or try to cut prices or other information.
Advise what to do in this situation?
OS: Ubuntu
Web Server: Nginx

Answer the question

In order to leave comments, you need to log in

2 answer(s)
D
Developer, 2018-07-24
@samodum

They look for vulnerabilities by sorting through pages and requests to them according to a template that is stored in their dictionary. For example, the same CMS has the same entry points to the admin panel, to search pages, and so on.
Usually this is done through special software, which contains a set of such predefined parameters.
My site is also being attacked. But for me it can withstand high loads, so it won’t even notice 100 thousand extra requests per day.
Improve the quality of the site so that it can handle this kind of traffic. writing filters for such requests is unrealistic.
A temporary solution is to cut by User-Agent, but it is likely that a large percentage of users with exactly the same user agent will fall off.
Or put something like CloudFlare

K
ky0, 2018-07-24
@ky0

This is a search for vulnerabilities, not ddos ​​- if the web server is configured normally, this should not lead to any negative consequences. Diagnose what is loading the server and fix it, or increase performance.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question