How to detect a sharp change in the number of requests to the web server?

Question

Monitoring

Hello.

The point is the following. Clients make requests to the WEB server. It is necessary to detect a sharp change in the number of requests and send a message.
An example is the following statistics on requests:



As you can see, from 07:00 to 07:05 there was a significant change in the number of requests. This is easy to see with the eyes, but how can this mechanism be implemented in a script? In my opinion, this has some similarities with stock quotes, I heard that trading terminals can detect such changes and send an alert, please tell me how such changes are detected?

PS
The column " dynamics in % " shows the change in the number of requests compared to the previous check and is calculated using the formula($total_last_req/$total_prev_req)*100-100 if the result is negative, then the number of requests has fallen by the specified percentage, if positive, then it has increased accordingly.
But this is not enough, since the instantaneous change, due to the small number of requests, can fluctuate significantly and give false-positive results, it is necessary to do the calculation based on the last five measurements.
In general, tell me please, what method can be used to detect such a change?

Answer 1

In general, trading terminals do this using the Kalman filter .
But as I said above, I was sure that such deep analytics was not needed here and found a simpler solution:

Averaging allows you to save the WARNING state over several checks, which, in general, was required:

For the sake of interest, I tried another 50 percentile, but too quickly "fading" the WARNING state, simple averaging worked best. Thus, monitoring has time to notice that something anomalous has happened (an alert arrives only after four unsuccessful checks) and fals positiv is no longer there.

Answer 2

The task of searching for outliers and changing the model of the behavior of the time series. Deeply studied in theory and has long been used in intrusion detection systems and other cybersecurity systems.
It can be set in various settings - from simply fixing the fact of an increase in the request flow to the task of early warning about a possible anomalous change in traffic.
The problem does not really have a single solution. In addition, the use of Separate methods often results in a relatively high percentage of Type I and Type II errors, and for intrusion detection purposes,
this is very annoying. Therefore, today they are trying to apply not a specific method, but an ensemble of methods. As separate criteria, everything that is known to science is used - from the elementary Student's t-test to LSTM networks.
Unfortunately, there is little literature in Russian, and the quality of the available literature leaves much to be desired. In English, only books over the past 9 years, I know about two and a half dozen titles.
Well, just the latest that came across - 2020 edition.
Data Science in Cybersecurity and Cyberthreat Inte...
And articles on private topics - including the one we are talking about - only I have accumulated as many Gigabytes, and in Internet bookmarks - a couple of hundred links.
Why is the information poorly distributed and it seems that there is virgin land here - I wrote a little already here:
What existing IDS (intrusion detection systems) use neural networks?

Answer 3

This is called monitoring and alerting systems. Datadog, Prometheus, Splunk. You can see others