How to increase the accuracy of the system for searching for goods in the photo?

Question

Neural networks

I am working on a search system for goods located in the photo (mostly clothes). The logic is approximately similar to the logic of the google lens. The algorithm works well for photos in which a person or a single product is well positioned, but if a person is standing in an unnatural position or the background is very bad, the algorithm does not work well.
I use Mask RCNN for searching, as well as ResNet for searching for similar products by features.
Maybe someone has suggestions about improving the accuracy of the algorithm?

Answer 1

To begin with, decompose errors (for example, where the detector is wrong, and where is the actual search for features), and then decide (throw data, complicate / retrain the model, etc.)