D
D
Drammm2018-09-01 08:53:08
bash
Drammm, 2018-09-01 08:53:08

Reverse DNS query automation?

I have a heavily stuffed Nginx config because I really don't like bad bots.
one of the conditions in the config I collect in a separate log requests from bots that mow down the useragen for search bots, but they are not.
But since the subnets of search engines can change, I would like to set up an automatic check, for example, the top 10 IP from this log.
Please help me write a bash script to automate the check.
For example, such a team

cat /var/log/nginx/access.zakos_pod_good_bot.log | awk '{if ($9=="503") {print $1}}' | sort | uniq -c | sort-nr | head

I get TOP 10 IP , then I write them to a file

#!/bin/bash
cat /var/log/nginx/access.zakos_pod_good_bot.log | awk '{if ($9=="503") {print $1}}' | sort | uniq -c | sort-nr | head > dns.txt

Further, I see the logic of work as follows - we check each of these IPs with the Host command, for example
host 95.213.246.202
Host 202.246.213.95.in-addr.arpa. not found: 3(NXDOMAIN)

etc. Somewhere we write this data, in some kind of log. Next, we scan this log for the presence of

yandex.ru
google.com

Since the IP of Yandex and Google bots returns

host 37.9.113.197
197.113.9.37.in-addr.arpa domain name pointer 37-9-113-197.spider.yandex.com.
host 66.249.76.57
57.76.249.66.in-addr.arpa domain name pointer crawl-66-249-76-57.googlebot.com.

If Yandex or Google is found while scanning the log, we send a notification to the admin.
Help with bash script please

Answer the question

In order to leave comments, you need to log in

1 answer(s)
I
ISE73, 2018-09-03
@ISE73

Call host with xargs -n 1 and write the responses to a file.
Then grep and notify the admin by the return code.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question