How to compare 3d models of objects with the image received from the camera?

Question

Python

I recently wanted to make a program - a Rubik's cube assistant. To do this, I used openCV, I was able to relatively fully recognize and select 9 colored squares (one side) with contours and select the entire side with a contour. The next step was to extract the color of the square and write it to a 3x3x3 array. And now the question arose, how to understand the position of the cube? How to understand which of the 9 selected contours is which square, where to write it? Considering that the whole cube can stand at an arbitrary angle. So I thought, is there any way to compare the 3D model of the cube with the selected contours?

Answer 1

take a lot of pictures from different angles , and then compare

Answer 2

Assume that the middle faces are fixed.