How to execute a long script in PHP?

Question

linux

Good afternoon!
The task is to execute a long script in php, which is executed through the console (cron).
When trying to execute a script php -f test/update.php
Sample script:

// допустим $items имеет милион записей
ob_start();
$number = 0;
foreach($items as $key=>$item)
{
     // скрипт обновления данных 

     if($number>=1000){
          $number = 0;
          ob_flush();
    flush();
     }

     $number ++;
} 
ob_end_clean();

Mistake:

[[email protected] www]# php -f test/update.php
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 1198085 bytes) in /home/bitrix/ext_www/website/www/test/update.php on line 168

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 1198085 bytes) in /home/bitrix/ext_www/website/www/test/update.php on line 168

[[email protected] www]# php -f test/update.php
PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 82 bytes) in /home/bitrix/ext_www/website/www/bitrix/modules/main/classes/general/usertype.php on line 2797

Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 82 bytes) in /home/bitrix/ext_www/website/www/bitrix/modules/main/classes/general/usertype.php on line 2797

Answer 1

your problem is not with time, but that the memory is running out, or raise the memory limit or optimize the algorithm.
very suspicious code, if this is a console, then why are there ob_* functions and flush ?
data does not need to be accumulated in memory save the processed data directly to the database or files

Answer 2

Maybe you should try generators, if the php version allows, then the memory issue will disappear

Answer 3

Through ob_start(); you accumulate the result in a buffer. Obviously, you just need to use some kind of external storage - a database or a trivial file.