How to track the movement of the camera by changing the depth map?

Question

3D

I want to briefly talk about the problem. I have a kinect camera and firstly I save the desired starting position of the environment, that is, point cloud of start position. That is, the first frame is a 3d Point cloud. Then I move, turn, or all together the camera and I need to determine where to move and turn it again to return to the start. Now I compare each frame c with the start using ICP, but it will no longer work if the rotation is already 180 degrees is allowed.
How can you do it? What can be counted?
And what tags can be used :)

Answer 1

What you want is called "SLAM"
Simultaneous Localization and Mapping
There is a good series of lectures here