How to reindex part of a PHP array?

Question

PHP

Guys, there is a task to optimize the work of php with big data. I think the discussion will be helpful.
The essence of the problem: there is 24GB data, a large number of operations are performed on the data, the data is cut into 8 arrays (for parallel processing) and is contained in the server's RAM. In total, we have one array weighing about 3GB and the number of elements about 100000 , due to processing, some elements of the array are deleted (90500, 90501, "delele_key", 90503 ). But the order of the array keys and the vertical order of the values ​​are important. That is, when deleting a value, the array must be reindexed as with the array_values() function. But this function does not have any adjustable parameters and recalculates (processes) the entire array, even if 99996 value has been removed from 100000. The server physically freezes. What can you think of to re-index only a part of the array elements located to the right on the number line. In my example, only from 99996 to 100000, i.e. affect only 4 elements?

Answer 1

For a long time already everything is thought up - a DBMS. And do not shove 3gb into the RAM.

Answer 2

Of course, there are databases - one of the databases is "hot" on ssd, the data in the php array is very dynamically used by php, constant editing and use is made. The database contains already processed and structured data. Working processes work directly with the database. I will describe the structure and nature of the data using a simple example: imagine a situation when they brought us 50 kg of mixed cereals (rice, buckwheat, pearl barley, pits, millet and gravel (removable garbage)). We have a locker with containers for each type of cereal - this is! a database!, from this locker, at the moment, the remains of cereals are being sorted out for cooking. It is very convenient and fast to take cereals from the locker, since everything is in its place and signed. Would it be logical to add 50 kg of mixture to the locker and slow down the work processes? No. We need to pour a bag of mixture on the floor and apply sorting tools to it, and sorted out into containers for further use. But, there are cases when the cereals are poorly mixed or there are only two types of them - then it is not logical to sift out all 50 kg, but you can separate a fragment with unnecessary garbage, sift it out. When we sift, we get 2 heaps of the desired product - a large and a small one. Now we are trying to weigh both piles before sending them to the locker, but only a small one is needed. It would be desirable to calculate the large weight as = total weight (50kg) - weight of a small pile - weight of garbage. But array_values() weighs everything... Sort of... there are times when cereals are poorly mixed or there are only two types of them - then it is not logical to sift out all 50 kg, but you can separate a fragment with unnecessary garbage, sift it out. When we sift, we get 2 heaps of the desired product - a large and a small one. Now we are trying to weigh both piles before sending them to the locker, but only a small one is needed. It would be desirable to calculate the large weight as = total weight (50kg) - weight of a small pile - weight of garbage. But array_values() weighs everything... Sort of... there are times when cereals are poorly mixed or there are only two types of them - then it is not logical to sift out all 50 kg, but you can separate a fragment with unnecessary garbage, sift it out. When we sift, we get 2 heaps of the desired product - a large and a small one. Now we are trying to weigh both piles before sending them to the locker, but only a small one is needed. It would be desirable to calculate the large weight as = total weight (50kg) - weight of a small pile - weight of garbage. But array_values() weighs everything... Sort of...