Question about deriving gluLookAt function in OpenGL?

Question

3D

Good evening everyone,
I'm sorry if I'm asking a question that has been repeatedly discussed, and if there is a ready answer, then please poke my nose into it. A little background: I'm trying to understand the basics of 3D graphics in order to eventually write a small example where there is a free movement of the "camera" in space with the rotation of the world and so on. I agree, there are examples, but they are built on other people's decisions, which, at times, are incomprehensible to me. And I have been struggling with some questions for a long time, that I have already used up a lot of sheets :(
While studying gluLookAt, I noticed that the up vector usually points in one axis (y or -z), regardless of where the “camera” is looking (yes, I know that you need to move the world in order to emulate the concept of movement in OpenGL) . Theoretically, the up vector should be orthogonal to the direction of the camera (or the difference between the points - center and eye, that is, again, a vector with the end of "where to look" and the beginning of "where"). The Internet writes that it doesn't matter and often the up vector just looks "up", no matter where the camera is pointing.
Can you explain to me why that is? Because when I tried to emulate camera tilt, I had the task of rotating the up vector relative to the camera direction vector, and for me it was difficult.
And the second question, it's related to the title, please help me with an explanation of how the gluLookAt function itself is output. On the Internet, I found a small site (sorry, in German) that describes how to implement this function, and I did it (with operations on vectors and matrices, I know a little).
If possible, could you explain to me why the transformation matrix that we build at the end is exactly what we need (it is clear that it gives us the correct result).
Look here:
When we get a normalized camera direction vector and a normalized up vector, we vector-multiply them with each other:
s' = fx UP'
The geometric meaning of this product is a new vector that is perpendicular to the plane passing through the two vectors and directed “to the right” (it seems that this is called that three vectors form the right coordinate axes, or the right-hand rule applies to them).
Okay, let's say we normalize our vector and multiply it by the camera's direction vector again, and then normalize:
u = sxf
Great, a new vector perpendicular to both vectors is obtained. But I can’t understand (sorry for stupidity) - why, and how do we move on to such a matrix:

|  s[0]  s[1]  s[2]  0 |
M = |  u[0]  u[1]  u[2]  0 |
    | -f[0] -f[1] -f[2]  0 |
    |   0     0     0    1 |

It's possible, a rotation matrix, but here I don't see sequential multiplication of rotation matrices about individual x, y, z axes (such as on wikipedia ). Moreover, we do not have any cosines and angles that we could calculate from the available two vectors.
Thank you more in advance.

Answer 1

As you wrote, the transformation matrix is ​​the transition matrix from one basis to another. It can contain both rotation and scaling and other linear distortions.
One of the properties of the transformation matrix is ​​that its rows are the basis vectors of the new coordinate system. That is, when you multiply a matrix by a vector, you are successively scalar multiplying the original vector and rows (the basis vectors), getting the original vector in the new coordinates. Since each scalar product will show how much of one or another basis vector is in the original vector. For the transformation matrix to be orthonormal (contain only rotations), it must consist of mutually perpendicular and normalized vectors.
So, the rows of the matrix we have are the vectors of the new basis in the coordinates of the original basis. We must find them.
We know one of the vectors - this is the direction of view, we normalize it and write it immediately to the matrix (for OpenGL, the view axis is the Z axis, therefore we write our vector to the third row of the matrix). The second vector is orthogonal to the first and to some exemplary upward line. we obtain it by the cross product of the look vector and this auxiliary vector. We get a vector directed to the right (or to the left, depending on the order of multiplication). After normalization, we write it in the first row of the matrix (in OpenGL, the X axis coincides with the X axis of the screen and is directed to the right). The remaining vector is obtained by the cross product of the vectors already obtained and written to the second row of the matrix (in OpenGL, the Y axis is directed upwards). It should be already normalized.
But there is a nuance. If the look vector coincides with the auxiliary upward vector, then their vector product will be zero and the matrix cannot be calculated.

Answer 2

Since no one has suggested anything specific...
Here in this book (easy to search and download) from page 554 (but you can start reading earlier) describes how to work with a UVN camera. Based on your M matrix example, such a camera could be called a transposed "SU-F".