How to index all files in AWS S3 compatible storage?

Question

API

Hello!
We use HotBox from Mail.ru Cloud Solutions. This is an AWS S3 compatible data store.
There is a need to cache all files on the HotBox and save them to the database. (paths to files, their names and modification date). The structure of files and folders is arbitrary. There may be a file at the root and a large nesting of folders with a file at the end. The cache needs to be kept up to date regularly. When uploading a new file, the server already knew about the new file for a couple of minutes. While there can be a very large number of files. (10,000+ files added daily)
How can this be done? As far as I can see, there is no way in the API to sort all keys by modified date. Only get the keys in alphabetical order.
Suggest possible solutions please. It is possible without specific examples, only the logic of work, because. I can’t imagine how this can be implemented without creating delays in updating with a large number of files.
API documentation in Russian: https://mcs.mail.ru/help/storage-api

Answer 1

Why do you need an S3 layer when you can directly use AWS services yourself? And everything would be simple there - put a lambda function on the appearance of a new file and it will write this information to DynamoDB, for example