How to cut background rectangles using OpenCV and Python?

Question

Python

Hello!
Something is not going well for me with opencv in python. probably because he programmed for a very long time in a completely different direction.
My problem is this and I will be grateful for examples, tips, etc.:
There is a certain picture (usually a scanned document) in which there are gray rectangles. my task is to cut out these rectangles (later to recognize, but there are no problems with this). There can be any number of them in the picture and in any place.
I came up with this code:

#!/usr/bin/env python
import cv2
import os

img_path = os.path.join('img', '1.jpg')
image = cv2.imread(img_path)

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gray = cv2.bilateralFilter(gray, 11, 17, 17)
edged = cv2.Canny(gray, 30, 200)

cv2.imshow('gray', gray)
cv2.waitKey(0)

cv2.imshow('edged', edged)
cv2.waitKey(0)


(_, cnts, _) = cv2.findContours(edged.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

for c in cnts:
    peri = cv2.arcLength(c, True)
    approx = cv2.approxPolyDP(c, 0.02 * peri, True)

    if len(approx) == 4:
        cv2.drawContours(image, [approx], -1, (0, 255, 0), 3)


cv2.imshow('result', image)
cv2.waitKey(0)

This code finds a lot of garbage - this is the first. Secondly, it finds all rectangles in general, and not just those with a background. Thirdly, I feel in my gut that some code is not correct.
Tell me, please, what to correct-add-to study in order to solve my problem?
Original picture:

Answer 1

I would probably divide the task:
1) Find all the gray rectangles - already there.
2) Exclude from the found what is not necessary - according to the histogram of colors, or by applying filters there - this is already a question not about python, but RO.
I think I don't quite understand the question