How to find and replace the floor in the picture?

Question

.NET

Good afternoon.

I'll try to describe our task: we have a picture with a room that has some kind of floor.
We also have another picture with an example of some kind of flooring (for example, laminate, tile, vinyl, rugs, etc.).

You need to create a new picture where you simply replace the floor from the original picture with the floor from the second picture.
Roughly speaking, the user has a photo of his room. And he will be able to see how different floor coverings will look with his photo of the room.

I would like to understand in which direction to look. Do I need to use Machine Learning to identify and replace flooring? Or is it necessary to use Computer Vision (like OpenCV)?

We use the following stack: .NET, NodeJS.

I will be glad to any advice and recommendations.

Thanks in advance.

Answer 1

I'll add to the previous answers.
Do not forget that you need to take into account the spatial perspective, and it depends on the angle of each particular photo. You also need to somehow preserve the distribution of light and shadows. I agree that

using photoshop is pretty tedious to do

. It is possible to automate, but it is a job for the development team. If you can automate, you will get rich selling this software.

Answer 2

There are two tasks for the computer. vision:

1. find the floor in the image and make a mask for it
   
2. determine the perspective grid corresponding to the floor
   

About the 1st part, google "image segmentation". For example, here they are trying to separate the floor in the photo.
The 2nd part is also solved algorithmically, for example, when automatically merging panoramas and leveling the horizon of images. They find straight lines in the picture and, based on them, guess the rest.
For a little more believable illumination of the new floor, you can blur the original area to the max and use it as a layer of brightness with a small% influence.

Answer 3

Photoshop.
Of course, everything you described is probably feasible, but very dreary, since everything will depend on many factors (photo quality, light in the room, colors in the photo, etc.)
To begin with (to test), I would advise you to choose a good photo of the room on the photo stock to cut it out the floor in Photoshop and just substitute your floor coverings as the background of the floor.
I would try to consider this option.

Answer 4

just replace the floor from the original picture with the floor from the second picture.

It's rather tedious to do this through Photoshop, and you want to do it through cv, but as you said it depends a lot on what you want, if the floor contrasts sharply enough from other objects, then through opencv you can cut out the objects and overlay them on a new background, and if you want to do for any picture, then xs that will help you. Yes, and at least throw 1-2 pictures of what is and what you need to get, otherwise it’s so difficult to advise something specific.