How to clean up stale data in a high load collection?

Question

NoSQL

Hello. There are two shards, a collection with a load of about 600 add and update requests per second. There was a problem with cleaning up obsolete records. The first solution was to write a utility that would run during the minimum load and remove obsolete entries. But after launching the utility, the removal blocked the work (as expected) and at the same time worked for 12 hours, which is unacceptable. Then we introduced a TTL index, but this also led to a large number of queues and an increased load on the CPU. Advise on the best approach to deleting obsolete data on a heavily loaded system. Thanks in advance.

Answer 1

Have you tried capped collection?

Answer 2

Add 2 more shards, and periodically switch from one to another. While the second one is working, clear the first one.

Answer 3

Yes, and how to synchronize data between two clusters?