Filter out noise in data?

Question

Python

There is a task to filter data from GPS sensors taken from several vehicles for the n-th period of time. GPS gives a pretty decent error. If you build a graph, the outlines of the roads are clearly visible, but there is a lot of noise, the points can "jump" from the road up to 200 meters. What algorithms (probably machine learning) can filter this data?
There was an idea to use RANSAC, but the route is not straightforward.
There is also an idea to divide the road into straight-line pieces, but, alas, I don’t know the algorithm. Any advice on where to dig?

Answer 1

From the coordinates and time you can get the speed and direction of the vehicle
.

Answer 2

What algorithms (probably machine learning) can filter this data?

Machine learning methods are trendy among IT people, but... often meaningless and useless in digital signal processing (DSP). There are so many noise filtering techniques and methods for this business, ranging from the classics of spectral transformations and digital filtering, to newfangled things like wavelet transforms and autoregressive filters. ... so much that I don’t want to specify anything specifically without seeing the signal in detail. And the results are simply stunning.
Take books on digital signal processing, and even better - find advisers from among those involved in digital signal processing (often in applied science circles at universities) in your close circle.

Answer 3

There is a Kalman filter for this .
Last quote from here . And this article finally removes all questions.

Answer 4

If you want a bike, you can do this.
1) enter the noise model. For example, "with a probability of 90%, the error is Gaussian distributed with a sigma of 15 m, and with a probability of 10%, the point can be anywhere."
2) for a set of 3, 4 or 5 consecutive points (x, y, t), enter a criterion - how likely is it that each of these points hit 90% (taking into account the technical capabilities of the car and the possible curvature of the road). Determine the penalty, which is the greater, the lower the probability. You also define a penalty for the point hitting 10%.
3) look through the points and sort through the options, which points are in 90%, which are not (dynamic programming by the mask of the last points that got there). The goal is to find the option with the least penalty.
4) by this moment the points are divided into good (which are described by Gaussian noise) and bad. Take good points and go through them with some kind of smoothing filter. True, it will have to be selected so that it takes into account the possibility of turning at a right angle.
This is suitable for points taken from one machine. If there are many points, they form a wide band, and the information where they came from is lost, then some other methods are needed.
And if you don’t want to reinvent the wheel, study what the Kalman filter is. Maybe you'll get lucky. Personally, I prefer a bicycle (I think this or a similar model is called the Witterbi method - but maybe not).

Answer 5

Algorithm for averaging statistical values: habrahabr.ru/post/134375