Answer the question
In order to leave comments, you need to log in
What to tweak in the settings of MongoDB or the OS itself to be ready for a surge of inserts?
We have MongoDB 2.4.9 (I don’t risk switching to 2.6 in production yet).
About once an hour there is a task to quickly update about 20 million records. This is done by completely removing the old collection (drop) and actively inserting new 20 million records. This takes approximately 10 minutes. The total volume of melons is about 1 gigabyte.
There is another collection on the same MongoDB server. In which there is a record of about 2-3 documents per second. And reading about once per second. If there is no active write to the first collection now, then this reading takes no more than 0.01 seconds. And when the recording is in progress (within 10 minutes), the reading speed starts to jump VERY strongly in the range from 0.1 to 1 second.
I read that MongoDB used to lock the entire database when writing. But in the latest versions, this problem got rid of. Transferring the collection, which is being actively written to, to another database did not lead to an improvement in the picture.
Actually a question - whether it is possible to achieve that reading from other collections did not suffer during the active record in this ill-fated collection?
MongoDB is running in a virtual machine. Increasing the number of allocated cores did not improve the situation. RAM allocated 8 gigs.
In which direction would you advise to look?
Answer the question
In order to leave comments, you need to log in
Increasing the number of allocated cores did not improve the situation.Because the process rests on io and locks in MongoDB when writing have nothing to do with it.
The total volume of melons is about 1 gigabyte.Push the entire collection into RAM.
RAM allocated 8 gigs.
About once an hour there is a task to quickly update about 20 million records.But I'd rather review the algorithm first, why drop/insert and not update?
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question