How to update files and data from different sources?

Question

PHP

Hello!
I am writing a kind of file storage manager that combines several sources (both local and all kinds of clouds and FTP servers). There was a question of updating the data:
- checking files for physical existence
- checking for a match when uploading / transferring a file from their "storage" to the "storage"
- checking for the presence of a file (its url, name and other parameters) in the manager's database
, etc. .
Question: how to algorithmize checks? In the case of a local folder with 10 files on the server, this is more likely not a question: we scan the folder and parse the changes, but if we imagine that there can be a dozen connected servers (conditionally), and each of them has several hundred, or even thousands files that can be downloaded not only through the interface and access to which can be interrupted (ftp is not available, or someone simply deleted the file ..)
How to "check the clock" correctly?

Answer 1

https://ru.wikipedia.org/wiki/rsync
man rsync
man find
man md5sum