How to get rid of glare on a photo of matte paper using anchor points?

Question

Algorithms

For example, there is a poster with an image hanging on the wall. If you take a picture of it, the colors in the photo will be different due to the lighting. Black (according to the meaning on the poster) will not be completely black, that is, not #000. Red - not quite red, etc.
Moreover, if the poster is outside, then the light will be from the sky, that is, bluish (cloudy day). If indoors, the light will depend on the lamps used.
The maximum that I can do is to dot the photo, where what color should be.
For example, #564e57 (highlighted) should become #4c1a11 (real),
#d8e2ea should become #fde8d5.
Etc. I can point out about 10 key points.
Is it possible to somehow derive the lighting formula based on this information and make inverse transformations to real colors for all other pixels in the image?

Answer 1

In photo processing programs there is a “white balance” correction function - this is for correcting for the color of lighting (cloudy day / incandescent lamps). You poke an eyedropper on an area of ​​the image that is obviously neutral in color, for example, a special gray cardboard, and the hue correction for the entire image is calculated, the colors become more believable.
Contrast is adjusted in much the same way. With an eyedropper for a "black" poke on an area that is obviously black - and the levels are adjusted, making this color true black. This affects the entire image and darker areas will also turn flat black, losing detail.
The same with the lightest area. Also your pipette.
So with three samples - a non-colored area, the darkest and the lightest - you can quickly adjust the image.
I won’t tell you what mathematics is behind this, you need to study the issue.
Here on SO discussed.

Answer 2

Key points should at least have application limits, and given the specifics of this photo + noise, it will be extremely problematic to do this with the help of software, I have never seen anything like this.
If you set the condition that all photos will be with the same location of this poster in the frame, then you can set a stencil, borders, and already fill them with a new color. But how it is feasible, think for yourself.