I
I
Igor2019-11-26 04:19:06
MySQL
Igor, 2019-11-26 04:19:06

How long can a request be?

Hello colleagues!
The customer requires the implementation of a method that would perform the cloning of some records.
For example:
On the example of gas stations
There is a network of gas stations, their number reaches 61,000 thousand.
Each gas station has a number of services provided.
It is clear that this is the sale of different types of fuel.
Each gas station has its own list of types, it looks like this:
Gas station A - 17
Gas station B - 18

INSERT INTO `features` (`id`, `name`, `type`, `value`, `organization_id`) VALUES
  (36, 'Заправка', 'enum', NULL, 17),
  (71, 'Заправка', 'enum', NULL, 18);

INSERT INTO `features_values` (`id`, `feature_id`, `name`) VALUES
  (111, 71, 'Аи-92'),
  (115, 36, 'Аи-95'),
  (113, 36, 'Аи-98'),
  (112, 71, 'Дт');

The customer selects any company and makes adjustments to the list of the current company.
And attention! What would be the option to apply the adjustment to all companies
For example, conditionally add "Propane" to all 61,000
To do this, I wrote a function, below is its implementation.
$features_repository->cloning($feature->getName(), $item);
Here is an example of a function.
The function works correctly, except that some part of it is quite expensive.
/**
     * @Rest\Route(
     *     path="/features.edit",
     *     methods={"POST"}
     * )
     * @param ObjectManager $manager
     * @param Request $request
     * @param RestMessage $restMessage
     * @return JsonResponse
     * @throws RestException
     * @throws DBALException
     */
    public function edit(ObjectManager $manager, Request $request, RestMessage $restMessage)
    {
        /** @var FeatureRepository $features_repository */
        $features_repository = $manager->getRepository("App:Feature");

        $feature = $features_repository->findOneBy([
                "id" => $request->get("feature_id", 0)]
        );

        if (!is_object($feature)) {
            RestError::exception(RestCode::RECORD_NOT_FOUND);
        }
        
        switch ($request->get("type", false)) {
            case "bool":
            {
                $feature->setType("bool");
                $feature->setName($request->get("name"));
                $feature->setValue($request->get("value"));
                foreach ($feature->getValues() as $value) {
                    $manager->remove($value);
                    $manager->flush();
                }
                $manager->persist($feature);
                break;
            }
            case "text":
            {
                $feature->setType("text");
                $feature->setName($request->get("name"));
                $feature->setValue($request->get("value"));
                foreach ($feature->getValues() as $value) {
                    $manager->remove($value);
                    $manager->flush();
                }
                $manager->persist($feature);
                break;
            }
            case "enum":
            {
                $feature->setType("enum");
                $feature->setName($request->get("name"));
                $feature->setValue($request->get(null));

                /** @var FeatureValue $old_fv */
                $not_persist = [];
                foreach ($feature->getValues() as $old_fv) {
                    $remove_old_fv = true;
                    foreach ($request->get("value", []) as $new_value) {
                        if ($new_value == $old_fv->getName()) {
                            $remove_old_fv = false;
                            $not_persist[] = $new_value;
                            break;
                        }
                    }

                    if ($remove_old_fv) {
                        $manager->remove($old_fv);
                        $manager->flush();
                    }
                }

                foreach ($request->get("value", []) as $item) {

                    foreach ($not_persist as $np) {
                        if ($item == $np) continue 2;
                    }

                    $feature_value = new FeatureValue();
                    $feature_value->setName($item);
                    $feature_value->setFeature($feature);

                    $feature->getValues()->add($feature_value);
                    $manager->persist($feature_value);

                    // Дорогой mysql запрос
                    if ($request->get("cloning", false)) {
                        $features_repository->cloning($feature->getName(), $item);
                    }
                }

                break;
            }

            default:
            {
                RestError::exception(RestCode::REQUEST_WITH_INVALID_PARAMETERS);
            }
        }

        $manager->flush();
        return $restMessage->success("Ok");
    }

Pay attention to the comment "Dear mysql query"
Clone function
/**
     * @param string $feature_name
     * @param string $feature_value_name
     * @throws DBALException
     */
    public function cloning(string $feature_name, string $feature_value_name)
    {
        $conn = $this->getEntityManager()->getConnection();
        $sql = "
            INSERT IGNORE INTO    features_values (`feature_id`, `name`)  (
                SELECT DISTINCT   f.id, :feature_value_name
                FROM              features f 
                LEFT JOIN         features_values fv 
                ON                f.id = fv.feature_id  
                WHERE             f.name = :feature_name AND f.`type` = :feature_type 
            );
        ";

        $stmt = $conn->prepare($sql);
        $stmt->execute([
            "feature_type" => EnumFeatureType::ENUM_TYPE,
            "feature_name" => $feature_name,
            "feature_value_name" => $feature_value_name
        ]);
    }

Scheme
5ddcfec8834ad006782884.png
Most importantly, the features_values ​​table will contain hundreds of millions of records ((
What to do????
Fork the process, let it live for a day?
Resources:
5ddc7f9e2f7f6796561751.png

Answer the question

In order to leave comments, you need to log in

3 answer(s)
A
Andrey, 2019-11-26
@VladimirAndreev

Why do you need feature_values ​​in an expensive sql query?

R
Rsa97, 2019-11-26
@Rsa97

Why do you need a LEFT JOIN in your query, the result of which is not used anywhere?

I
index0h, 2019-11-26
@index0h

Maybe it's better to introduce the concept of common feature_values? And for a specific gas station, if necessary, add an entry with custom data, or a deletion flag.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question