How to classify images by color?

Question

Algorithms

Now I will write about real household items.
I have a black table.
A photo of this table can be taken separately in order to understand what colors are there.
There are 4 photos:
A green apple on a black table.
Green apple on a black table but close-up
Banana on a table
Banana on a table but close-up
Need to upload these photos to the server should group the photos by color.
those. the algorithm must somehow distinguish apples from bananas.
It is clear that it is difficult to distinguish apples from cucumbers, but we will attribute this to an error, as a rule, my things have different colors, the maximum that can be found is a dark cucumber and a light green apple.
But of course it would be ideal to distinguish them.
Can anyone give me an idea in which direction to go?
The first thing that comes to mind is to take a black table and break it into an array of colors + add some magic to understand its ranges of shades. get an array = [black, gray, light gray, bright black]
Then we take a photo of a cucumber and split it into the same array, from the array we remove all the shades that we have in the first array. got shades of apple, for example = [green, dark green, white(highlights), etc.]
and so on.
As a result, we will be able to understand the approximate belonging of objects to their category.
But as for me, everything in the algorithm is very bad here ((
A close-up apple and an apple on the table can give different colors.
I will also be happy with libraries for working with colors, all kinds of filters for images, etc.
so far only found this gka.github.io/chroma.j

Answer 1

You can take an array of colors and scan a photo. To begin with, reveal the minimum color of the cucumber, the maximum color of the cucumber, then the same with the banana, how to do this is another question, let's say in some flash, take the minimum green color on the picture and the maximum green and fill in all the intermediate colors, for example, according to the principle of the color wheel same Color Wheel. Then skip this photo with already known ranges, was there a color in the picture from the array of cucumber color ranges? Enter this color into the cucumber array. Next, we look at how many numbers from the total number of values ​​​​of the color range are in the cucumber array, determine for ourselves the threshold (or by some formulas), how many matches are needed to unambiguously say that this is a cucumber. We do the same with a banana, then compare again, a banana with a cucumber, how many banana colors matched the range of cucumber colors. And we get something like: Is a banana a cucumber? No, it means it's a banana. Is a cucumber a banana? No, it's a cucumber.
Naturally, it will be difficult to unequivocally say whether this is a cucumber or a green apple, but we can distinguish very different vegetables and fruits. Even if the dark areas on the cucumber, such as shadows or noise, will also apply to the array of table colors, they will not be enough to say that this is a table, the main thing is to determine the threshold of the total mass, how many matches are needed to say that this is a cucumber. Let's say if it's less than 80%, then it's not a cucumber. Natural for red, yellow, blue, etc. items need to be configured again. The solution is oak and most likely idiotic, but it can give you an idea. Good luck!

Answer 2

To add to DollyPapper's answer , you can only use colors that are not shadows or highlights. Those. filter out too dark and too light shades, taking the average in lightness. Then we should ideally get the local color of the item. And in this case, cucumber, apple and banana will be different. Since the cucumber has a cold and darkish color, the apple is actually light, closer to light green, but the banana is yellow. Although, of course, any method will be ineffective if the photo is in the wrong lighting ... Lighting will generally be the main factor ...
But it is also possible to predict the lighting. Then you need to first check for the overall shade of the photo. For example, the light of an ordinary lamp - yellows, daylight - cools, etc. Something like this...)