S
S
Stanislav2019-08-02 16:33:09
MongoDB
Stanislav, 2019-08-02 16:33:09

What is the correct way to update a large number of mongodb documents?

Maybe someone can help with the problem.
It is necessary to bypass about 100k+ documents every day, collect information and update them, this is done in order to obtain a certain rating of documents.
In terms of execution time, it is not so critical, but I would like to somehow sort out the records without heavy loads on the server.
For example, there is a prepared data array, say with 100k elements

[
 _id: ObjectId(....) // ID документа которой необходимо обновить
 popular: {
  day:  1230 // Поле которое необходимо создать
 }
]

I use async/await for all of this with something like this for updating documents
function isPopularToDay(request) {
    return Promise.all(request.map(async e => {
        return Collections.updateOne({ _id: e._id }, { $set: { "popular.day": e.count } }).then()
    }))
}

While there were few documents (about 10k), everything was normal, but now there are a lot of documents requiring changes, and I would like to update the data more than once a day, but every 2-3 hours, but the loads do not allow.
Now I'm thinking in the direction of .reduce() to stretch the entire execution for 10-20 minutes.
Maybe there are other options that are more correct and less expensive?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
R
RidgeA, 2019-08-02
@ms-dred

I think the most correct option would be to use bulk operations https://docs.mongodb.com/manual/reference/method/B...

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question