How to organize mathematical operations with a large amount of data?

Question

MySQL

Good afternoon, I need theoretical advice on working with a large amount of data.
The table stores approximately 10 to 100 thousand rows of data (from 12 to 25 columns)
. arithmetic) values ​​for some columns.
Now, when receiving more than 10k lines at once, I get an overflow of RAM.
Is it possible to somehow do data calculation operations in batches, taking into account the calculation of average data at the same time?
For a better understanding:
arithmetic averages are considered for one column, excluding zero values
the second column counts the sum of all records multiplied by 100 (to convert to a percentage) the
third column counts as the sum of the first / the sum of the second
How could this process be optimized?
Thanks for any help
PS
Stack: PHP7.1, Slim framework, Mysql 5.7, Eloquent ORM database driver

Answer 1

You can make a view with calculations, you will pull the results like a regular table.
You can put all the results in the cache and reset it when a new entry appears. Regarding batch processing in elepahant:

Chunking Results
If you need to process a lot (thousands) of Eloquent records, using the chunk command will allow you to do without eating all of your RAM:

User::chunk(200, function($users)
{
    foreach ($users as $user)
    {
        //
    }
});
The first argument passed to the method is the number of records you wish to receive per "chunk". The Closure passed as the second argument will be called for each chunk that is pulled from the database.