How to get a large amount of data in batches in mongoose + nodejs?

Question

MongoDB

There is a method in the controller that requests a large amount of data from the database through such a chain Model.find(obj).select(obj).populate(str).exec(fn)
. There is a lot of data and mongo falls on a timeout. I decided to look for how to receive data in batches. Stumbled upon a solution with paging using .skip()and .limit(). But I did not understand how to get all the data in this way. Send recursive requests with a change in the transmitted values ​​in .skip()and .limit()? Maybe there is a normal solution, how to get a data footcloth without overloading the server
Please help me, I'm new to mongo and node
Thanks in advance!

Answer 1

Everything turned out to be easy
Made with the helpcursor()

var bigData = [];
var cursor = Model.find(obj).select(obj).populate(str).cursor({batchSize:100});
cursor.
  on('data', doc => {
     if(doc) {
       bigData.push(doc);
     }
  })
  .on('end', () => {
     // производим дальнейшие действия с данными
  })

Answer 2

I think this is what you need link