What is the optimal way to bypass several files while changing them at the same time?

Question

Algorithms

There are three different files (F1, F2, F3) with more than 100 fields each, but with the presence of three required fields dep_no user_id & code (dep_no and user_id are included in the composite key, you can position both by [user_id] and by [dep_no : user_id] You
need to sequentially bypass these files (first F1, then F2 and F3 ... although the order of the value does not matter), and if code in one of them is !=0, then transfer this value to other files with the same user_id with minimal access to Valid
operations: Locate (key) - positioning by key without reading the record (allows you to find out if there is such a record),
First () - select the first record and read
Next ( ) the same but the next
ifEOF () check for the end of the file
getRecNo() - getting physical number
setRecNo () positioning by physical number.
Basically, you can use STL, but there are a lot of records in the file (several million), in some files user_id is not a unique key.
The simplest option suggests itself: first, we sequentially bypass the first file F1, if we find code> 0, then by user_id we are positioned in F2, F3 and do Update. But when we are already bypassing F2, then having reached user_id with code> 0, this entry could already be abated in F1. Very stringent performance requirements. Here you can use std::set, add custom user_id to set and check for their presence when traversing F2 & F3 files
, maybe there is something more optimal?

Answer 1

Divide, conquer...

1. Read all files into large array
   
2. (I would also write md5)
   
3. Make the required changes to the resulting data structure
   
4. Write files (if md5 was written - you can check the need for writing by detecting the fact of changes)