W
W
weranda2018-04-09 15:51:20
Browsers
weranda, 2018-04-09 15:51:20

How to identify a fake user-agent and should they be blocked?

Hello
Who is visiting the site? — Humans and bots. People use browsers, bots - XZ than, probably, everything that is possible. I would like to get the experience and recommendations of knowledgeable people on this issue.
By and large, the true users of the site are people and search engines, everything else is superfluous, well, maybe a few more useful services, correct me if I'm wrong.
Over time, I think more and more about blocking unwanted bots, sending them a little bit from different sides to hell. Now I'm thinking about blocking software with fictitious user-agents, at least some part of this action will be cut off. And with that, I have some questions for you:

  1. Is there a script on the network for determining all active user-agents by browsers, their versions and operating systems, or "all by myself, all by myself, by hand"?
  2. if there is such a script, then its behavior / algorithm of work is interesting when new versions of browsers and OS appear - what will happen?
  3. a consequence from the previous question: is it really necessary to somehow keep track of all new versions of browsers, OS and PS, or are there any options?
  4. is it worth it?

PS
In general, the task is this - did not introduce himself or introduced himself, but with a fake browser or one of the search engines, then "welcome to the pasture of bots."
PPS
Yes, I understand that the bot can also substitute the correct user-agent, but this does not change the essence of the issue.

Answer the question

In order to leave comments, you need to log in

4 answer(s)
M
Maxim Timofeev, 2018-04-09
@weranda

You are following in the footsteps of Don Quixote. This is a war on windmills, which will strain only a part of the innocent users. The bots will quietly laugh watching this.
And what does the script have to do with it, just a list of them is needed. The script - there search simple. But what is the probability that everything is taken into account in the list and it is fresh?
Takes a list, iterates, compares with your string. Read about loops and conditional statements.
Yes, no, of course not. Browser developers, before rolling out their code, will definitely call you in advance. And Google developers sit and think:
- Let's not change the name of the search bot, suddenly weranda will block it.
- Exactly, you need to call to warn
- And in general, find all similar lists on the Internet and add more
- Great idea, we just have nothing to do
no. I do not see a single plus and a bunch of minuses.

S
Stanislav B, 2018-04-09
@S_Borchev

is it worth it?

no.
change the user agent for the bot - 1 second.

C
CityCat4, 2018-04-10
@CityCat4

blocking software with fictitious user-agents

You first give the definition of what a fictitious User-agent is :) I give you so many user agents in a minute - never fictitious - that you will be surprised :) Do you seriously think that the world revolves around two or three distributions?
Mozilla/5.0 (compatible; Konqueror/14.1; Linux) KHTML/TDEHTML/14.1.0 (like Gecko) is the user agent. The real
Mozilla/4.0 (compatible; MSIE 4.01; Windows NT 5.0) is also a user agent. And no less real :)
Lynx/2.8.3dev.6 libwww-FM/2.14 - and this is the user agent!
Mozilla/5.0 (X11; U; Linux x86_64; en-CA; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) - and even that is it :D

X
xmoonlight, 2018-04-09
@xmoonlight

Here

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question