How to automatically select an object in an image?

Question

Video processing

There is a video converted to a sequence of frames. On some frames there is only a background picture, on others with objects. Both are selected by the user. What algorithms exist for high-quality automatic selection of objects, taking into account the fact that shooting takes place not in a photo studio, but in real urban conditions? I mean, the wiggle of the foliage and the slight change in light due to partly cloudy conditions are natural attributes of such a shoot. That is, it would be desirable that selection of objects was carried out programmatically.
Here's what I got using the difference between frames:

But I'm not satisfied with the result due to numerous artifacts.

Answer 1

The simplest thing in this case is to use the same difference, but not for the image as a whole, but for individual channels. Including saturation and hue.
Having found a mask for each pair of channels, they can be combined and a more accurate mask can be obtained.
And in the case of the car, in the center of the car you have a big black spot, because the difference in the green channel is offset by the inverse difference in red and blue. In total, something turned out to be about 0.
If we sum up the difference modules for each channel, this problem would not exist.
If you want to go further - you need to split the images into levels of detail (see wavelet photo processing). Finding the masks for each level separately and combining them will probably significantly improve the result.

Answer 2

In the general case, I can advise this method: opencv-python-tutroals.readthedocs.org/en/latest/p...
Although, in fact, a lot of interesting things can be done on the difference. You can lower the threshold, and remove all the small connectivity components. This can be done through morphological transformations or just run BFS in each component.

Answer 3

Try Matlab (Image processing toolbox).

Answer 4

If you remove small spots in your mask, and fill the spots inside the car object with the color of the mask to get a uniform color, you should get something like this: