D
D
Denis2019-02-25 13:48:09
System administration
Denis, 2019-02-25 13:48:09

Servers lie down from empty HEAD requests how to overcome?

Dear hello!
Servers crash from empty HEAD requests, user agents are different, there is no referrer.
These are not bots, I understand when search bots come in .. but here it’s just different browsers ..
Here is a piece from the log

116.231.90.3 - - [25/Feb/2019:12:43:10 +0200] "HEAD /watch/ami-ki-tomake-khub-birokto-korchi-cover-song-bengali-new-song/_NVvYMXhtzg HTTP/1.0" 200 - "-" "Mozilla/5.0 (compatible; MSIE 10.0; Macintosh; Intel Mac OS X 10_7_3; Trident/6.0)"
62.99.178.46 - - [25/Feb/2019:12:43:11 +0200] "HEAD /watch/obrabotka-poni-zakat-podarok-dlya-emon-pie-ili-sanshi-chan-chitayte-opisanie/EWrJAjVmLAk HTTP/1.0" 200 - "-" "Mozilla/5.0 (Windows NT 6.1; rv:12.0) Gecko/20120403211507 Firefox/14.0.1"
82.135.249.196 - - [25/Feb/2019:12:43:11 +0200] "HEAD /watch/sleep-well/ZbVeFEHOaJQ HTTP/1.0" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-TW; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4 ( .NET CLR 3.5.30729)"
169.255.127.137 - - [25/Feb/2019:12:43:11 +0200] "HEAD /watch/ppap-game-high-score-pineapple-pen-apple-pen-game/Do-aGlfTNIE HTTP/1.0" 200 - "-" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 6.1; chromeframe/12.0.742.100; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C)"
116.231.90.3 - - [25/Feb/2019:12:43:11 +0200] "HEAD /watch/mick-martin-the-blues-rockers-i-believe-in-you/V_kwDPUjBkU HTTP/1.0" 200 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:21.0) Gecko/20100101 Firefox/21.0"
177.52.249.128 - - [25/Feb/2019:12:43:11 +0200] "HEAD /watch/kak-legko-sdat-oge-po-literature-na-maksimum-moy-opit/zsUklqcR-yk HTTP/1.0" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.14 (KHTML, like Gecko) Chrome/10.0.602.0 Safari/534.14"
82.135.249.196 - - [25/Feb/2019:12:43:12 +0200] "HEAD /watch/shtamboviy-krizhovnik-pakspravila-ukhoda-i-plodonoshenie/iuXdbqvcLxw HTTP/1.0" 200 - "-" "Opera/9.80 (X11; Linux i686; U; en-GB) Presto/2.2.15 Version/10.00"
157.55.39.82 - - [25/Feb/2019:12:43:13 +0200] "GET /user/UCoLrcjPV5PbUrUyXq5mjc_A HTTP/1.0" 302 20 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
169.255.127.137 - - [25/Feb/2019:12:43:12 +0200] "HEAD /watch/uzi-sosudov-golovnogo-mozga/bs_0Vd95K2k HTTP/1.0" 200 - "-" "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.2.15) Gecko/20110330 CentOS/3.6-1.el5.centos Firefox/3.6.15"
177.52.249.128 - - [25/Feb/2019:12:43:13 +0200] "HEAD /watch/worst-kool-aid-ever-ragnas-day-1/QzT5cLyDWMM HTTP/1.0" 200 - "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.134 Safari/534.16"
116.231.90.3 - - [25/Feb/2019:12:43:13 +0200] "HEAD /watch/the-saints-stranded/Q-GueNOKolo HTTP/1.0" 200 - "-" "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.14 (KHTML, like Gecko) Chrome/24.0.1292.0 Safari/537.14"
82.135.249.196 - - [25/Feb/2019:12:43:13 +0200] "HEAD /watch/strizhka-graduirovannoe-bob-kare-strizhka-kare-na-nozhke/x6pfsJ7Hc3E HTTP/1.0" 200 - "-" "Mozilla/5.0 (X11; U; CrOS i686 0.9.128; en-US) AppleWebKit/534.10 (KHTML, like Gecko) Chrome/8.0.552.341 Safari/534.10"
169.255.127.137 - - [25/Feb/2019:12:43:14 +0200] "HEAD /watch/ak-scumfam-round-here-music-video-freekdot/lDERDYxjHLI HTTP/1.0" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.460.0 Safari/534.3"
It all started a week ago ..
Requests exclusively go from morning to evening, at night the servers take a break from this ..
At first there were requests from 1 ip to all 4 servers, everything calmed down and after half a day it started again but with 4 different ip also banned them and again after a while on a new one.
Maybe generally prohibit HEAD requests, but allow them only to search engines?
I don’t understand how to do it ..
HELP help ..

Answer the question

In order to leave comments, you need to log in

1 answer(s)
B
bkosun, 2019-02-25
@bkosun

Use fail2ban to analyze logs and block unwanted IPs automatically.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question