How can the trajectories of points move to a single coordinate system?

Question

Algorithms

Coordinates (X, Y) of a set of points come to the program.

Over time, the coordinates of all points can both shift relative to the center and rotate around the central point.

Point coordinates may slightly change relative to each other, points may appear and disappear.
It is necessary to bring all points to a single coordinate system.
What is the best way to implement the algorithm for solving this problem in C++?
Is it worth trying to use a neural network or is ordinary mathematics enough?

Answer 1

Matlab has Multi-object tracking , but due to sanctions, now this software is most likely unavailable. If only to read the documentation.

Answer 2

In 3D graphics, a special matrix is ​​used to shift, rotate, and scale coordinates.
https://en.wikipedia.org/wiki/Transformation_matrix

Answer 3

Enough of the least squares method.
I understand that you need to track the overall shift and rotation so that individual changes in points are minimal.
First, for each point in the previous image, find the position in the next one. You can take the nearest point if there are small changes from frame to frame. Otherwise, you will have to fence some kind of gradient descent. If the change exceeds a certain threshold value, consider that the point has disappeared and do not include this pair in consideration.
When for each point you know where its past and next position is, your task is to find the minimum of the function
sum_i (xp_i-xp'_i)^2+(yp_i-yp'_i)^2, where
xp' = xn*cosa - yn* sina + dx
yp' = xn*sina + yn*cosa + dy.
Here (xp, yp) are the coordinates of the i-th point in the previous frame, (xn, yn) are the coordinates in the next frame, cosa, sina, dx, dy are unknown. Take derivatives with respect to dx, dy and equate a to 0. You get 3 equations with 3 unknowns. Of the first two, it is elementary to express dx, dy through cosa/sina. Substituting in the latter, you get an equation like cosa ^ 2 A + sina cosa B + sina ^ 2C + D = 0. You can multiply D by cosa ^ 2 + sina ^ 2, then divide everything by sin ^ 2 and solve the quadratic equation with respect to the tangent. Then find the solution through the arc tangent.
If the changes from frame to frame can be large, then you need to minimize a slightly different function:
sum_i min_j ((xp_j-xp'_i)^2+(yp_j-yp'_i)^2) - for each point we take the minimum over all possible preimages and sum it up. Since it is no longer differentiable, we will have to use some more tricky optimization method for dx, dy, a.