B
B
becks2013-03-15 10:48:36
OOP
becks, 2013-03-15 10:48:36

How to find out whether or not image #1 includes an object from image #2 using OpenCV?

There was an urgent task to determine whether an object from an image is present in another image. If, in other words, there is a picture with an object, there is nothing in this picture except the object. In another picture, this object (maybe a slightly modified rotation/shrink/color change) may or may not be present. And you need to answer whether this object is in the picture-scene.
I heard a lot about OpenCV and so I decided to use it. Found an example of SURF Homography. What I got:
1) If the desired object without distortion is on the stage, then the number of “good_matches” is small (5 ± 2);
max_dist = 0.51 min_dist = 0.03
2) If the desired object is rotated 90 degrees on the stage, then the number of "good_matches" is also small (5 ± 2);
max_dist = 0.55 min_dist = 0.04
3) If there is an object conditionally similar to the desired one on the scene, then the number of “good_matches” is already greater (15 ± 5);
max_dist = 0.59 min_dist = 0.11
4) No object or similar object in the scene, number of "good_matches" > 30.
max_dist = 0.79 min_dist = 0.4
FLANN was used to find max_dist and min_dist.
Is it possible to draw conclusions on the basis of only max_dist and min_dist, which border values ​​should be considered?
Is there another way to determine the presence of an object in an image?
Please do not judge strictly, the second day for OpenCV.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
Z
Zenker, 2013-03-16
@Zenker

Do I understand correctly that you calculate the distances between the descriptors and look for the most similar points on the sample and the current image? If so, then this approach does not take into account the relative position of the points, so it is hardly possible to judge the presence of an object in the image. I myself am not very familiar with OpenCV, so I will try to describe in general terms a possible solution:
1) Using local feature detectors ( SIFT / SURF / ORB , etc.), find feature points on the sample and image
2) For the found points, calculate the values ​​of the descriptors in their vicinity. A descriptor is a multidimensional vector (essentially a set of numbers) that characterizes a point. The more similar two descriptors are (the smaller the distance between them), the more likely it is that we are talking about the same point on the sample and the current image.
3) To each special point on one image, match a point from another by the minimum distance between descriptors (the number of points found on the images will most likely be different, but the same sample point may be the best candidate for several points from the image). Such a comparison will most likely be wrong for the most part, so you need to filter out false matches.
4) The most important stage. False matches are usually well filtered by the RANSAC algorithm. The bottom line here is this: the algorithm tries to find a transformation from the points found above that would make it possible to best match the two images. If such a transformation is found and a sufficiently large number of points confirms this model (such points are called inlines), then most likely the object is present in the image.
Specifically, OpenCV has a FindHomography functionjust need to deal with it. If I understand correctly, srcPoints and dstPoints are arrays of singular points on the sample and image, where elements with the same indexes are pairs of matched points. Returns the transformation matrix H, but how the success of its search is signaled - you need to figure it out. Perhaps google some other implementation. I would try to evaluate the result by the proportion of inlines in the total number of points.
You can read more about RANSAC and a similar task in general here: engineering.purdue.edu/kak/courses-i-teach/ECE661.08/solution/hw4_s1.pdf

B
becks, 2013-03-18
@becks

Many thanks for the comprehensive answer. Itself on days off still esteemed on a subject, now I understand already more. I will sign what and how I do, already taking into account your answer.
1) With the help of SURF ( class SurfFeatureDetector , method detect ) I determine the key points on the sample and image.
2) For found points, I calculate descriptor values ​​( class SurfDescriptorExtractor , compute method ).
3) I match points on the target object and in the scene, I get the so-called matches ( FLANN , FlannBasedMatcher class, match method ).
Here I just wanted to measure the distance between the "matches", but not by min, max as described in the post, but by the average. And, if this value is less than some boundary value, then this indicates that picture1 is in picture2. But since you suggest that the number of false positives is large, I refuse this option for now.
4) In the example that I just found (reworking), just findHomography is used . docs.opencv.org/doc/tutorials/features2d/feature_homography/feature_homography.html
Mat H = findHomography( obj, scene, CV_RANSAC );
CV_RANSAC - I just use RANSAC.
But I don’t understand its purpose at all and how to get exact matches from the transformation matrix H or something else that can be used.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question