F
F
fStrange2019-12-25 19:10:29
Google
fStrange, 2019-12-25 19:10:29

Google is bombing the site with left requests, how to fix it?

For more than a month now, the Google bot has been bombarding a group of sites with the same type of requests.
Here is an example from the Apache log.

66.249.64.174 - - [23/Dec/2019:08:25:08 +0200] "GET /109367389/sudationreflex.jp HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:25:02 +0200] "GET /archegone-109244901_mammarymammary/df9d4eo.com.co HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:24:57 +0200] "GET /supervene-109309288_incipience.fyi HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:24:51 +0200] "GET /fishpaste/109008726_synchronicitysynchronicity415a3aj.software HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:24:45 +0200] "GET /tawney_111772799-corregidor.art HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:24:39 +0200] "GET /109320230-planningladylady.fyi HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:24:33 +0200] "GET /f74292bzebeck_109137128_adenovirus.jp HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:24:28 +0200] "GET /stormfulunswornunsworn-108902950.wang HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:24:22 +0200] "GET /a28d13udexterous/109361993/baguio.art HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:24:16 +0200] "GET /dupabilitydupability/underagef7c707h-108817600.software HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.172 - - [23/Dec/2019:08:24:10 +0200] "GET /emissary_111090867/a7b33dlperpetrationemissary.wang HTTP/1.0" 404 3752 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Of course there are no such pages. There are no links on this site. In the sitemap too.
I checked the IP, it's Google.
Calls from different IPs. This behavior of the bot seriously loads sites. As a matter of fact DDOS turns out.
I do not want to ban, because it is fraught with consequences and falling out of the search.
How to solve the problem?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
I
Ivan Shumov, 2019-12-25
@inoise

Put a CDN in front of the site, for example, cloudflare and set up caching on it (at least for 404)

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question