How to properly lock a file?

Question

linux

There are two (or more) writers who write to a file
. To do this, they lock the file (flock), but the problem is that they do not just add information to it. To maintain data integrity, they create a copy of it and add the necessary information to the copy, after which they rename the new file to the old one and remove the lock.
The problem is that when the second process finally waits for the file to be unlocked, it will write to the old file, and data loss results.
See solutions in the form of creating a "file.lock" file for each file, are there any more beautiful solutions. for example with temporary files?

Answer 1

It's just that your file division scheme itself is wrong, especially with renaming. Since it turns out that this is not one file, but a bunch of files with sometimes the same name.
Actually, swapping or writing multiple processes to one file is never good!
It is better to take an example from syslog or a similar service, where ONLY ONE process (service) writes to a file, renames it and hosts it as it wants, and all the rest work with this process through sockets.
To facilitate the task, you can use queues like zeromq or rabbitmq, or some similar framework.
If, for some reason, sockets cannot be used, then you will have to do it this way:
1) Write to a temporary file.
2) We capture the lock on the target file and overwrite its contents
3) remove the lock, delete the temporary file.
But, damn it, how will all this slow down and fail at large volumes !!!
PS. In fact, locks were invented in order to ensure consistency when ONE writes and OTHER reads. Or in order for a bunch of processes to append to the end of the file. Captured-recorded-released. For everything else, the OS base has IPC, pipes and sockets (including UNIX file ones).
Oh, I found the classics in Russian - citforum.ru/operating_systems/linux_pg/lpg_02.shtml