Search for "studs"

Question

Python

Colleagues, a fairly simple task has appeared, in which, however, I am not entirely sure of the correctness of my judgments.

And so, suppose the input to the program is an endless stream of floating point numbers, each of which differs from the previous one within certain limits.
The scope is not strictly defined and will be different in each case. Somewhere the numbers will differ by one or two, somewhere by 0.0001, somewhere by hundreds.
The goal of the program is to find some normal deviation for these numbers and a deviation that is greater than normal. In case of excess, notify about it.

I am sure that I simply do not have enough knowledge of mathematical statistics to apply some simple formula to find this very normal deviation for each stream.

Thank you for your attention to the question

PS If somewhere I used the term incorrectly, I'm sorry, I really forgot everything that was in the course of mat statistics)))

Answer 1

In my opinion, it would be nice for you to consider the standard deviation.

Answer 2

There was a similar task with the search for the average price in the market, with the rejection of what you call them "hairpins".
I followed a simple path: I looked for a deviation in percentage from the average value,
and I got the average value as follows: I sorted prices in ascending order, discarded 10% of the cheapest and 10% of the most expensive ... and already among the remaining 80% I got the usual arithmetic mean.

Answer 3

And within the limits of the “endless” stream of numbers, the characteristics of the data (the magnitude of the deviation) can change? Or is data from only one case fed to one "run"?

Answer 4

If the input stream is infinite, then you need to calculate the standard deviation (sigma) over the window (over the last N values). Next, we discard those values ​​that deviate from the window average by more than three sigma, for example.