How to perform a bulk operation with a database?

Question

1C-Bitrix

You need to write properties for all products based on existing properties.

The problem is that the operation turned out to be voluminous and the server, after 10-15 minutes, returns a 504 error.

There are 20k goods in total, but the situation is worsened by the fact that the properties are multiple, and GetList duplicates the record, which causes the number of iterations to increase tenfold.

How can the problem be solved?

P.S. request code

$vaz2107 = array(
    39819, 39820, 39821, 39822, 39823, 39824, 39825,
    'ID_GROUP' => 44219
);

$res = CIBlockElement::GetList(
    array(),
    array(
        'IBLOCK_ID' => 42,
        'PROPERTY_APPLICABILITY_SECT_VALUE' => $vaz2107,
        'ACTIVE' => 'Y',
    ),
    false,
    false,
    array('ID', 'IBLOCK_ID', 'PROPERTY_APPLICABILITY_SECT', 'PROPERTY_APPLICABILITY_GROUP')
);

while ($data = $res->fetch()) {
    $arPropertyGroup[] = $data['PROPERTY_APPLICABILITY_GROUP_VALUE'];

    // ID Группы, куда добавляем
    array_push($arPropertyGroup, $vaz2107['ID_GROUP']);

    $do = CIBlockElement::SetPropertyValuesEx($data['ID'], 42, array(
        'APPLICABILITY_GROUP' => $arPropertyGroup
    ));
}

if ($do == null) {
    echo 'ok';
} else {
    echo 'no';
}

Answer 1

To prevent GetList from duplicating records, you need to change the storage type of infoblock properties to storage in a separate table:

And you also have some strange situation happening here:

$arPropertyGroup[] = $data['PROPERTY_APPLICABILITY_GROUP_VALUE'];

    // ID Группы, куда добавляем
    array_push($arPropertyGroup, $vaz2107['ID_GROUP']);

    $do = CIBlockElement::SetPropertyValuesEx($data['ID'], 42, array(
        'APPLICABILITY_GROUP' => $arPropertyGroup
    ));

You accumulate data in the $arPropertyGroup array at each iteration . And at each iteration, you assign all the accumulated previous values ​​to each subsequent element in the 'APPLICABILITY_GROUP' => $arPropertyGroup loop . This is probably not what you expect in the end) Well, the plus is the increased load on the database.

Answer 2

Create a console script similar to "crown agents" and run it through the console.
Regarding agents: https://dev.1c-bitrix.ru/learning/course/index.php...
Make an analogue of the file /bitrix/php_interface/cron_events.php, remove everything related to agents and write your code