F
F
fAra0N252014-11-03 11:19:49
MongoDB
fAra0N25, 2014-11-03 11:19:49

How to correctly insert a large number of documents in mongodb?

Through mongoose, I insert batches of documents into one collection, thus (I omitted everything unnecessary):

docs1 = [1,2...n] // length = 50
Collection.create(docs1, function () {
   // Что то делаю
    docs2 = [1,2...n] // length = 10 тыс
    Collection.create(docs2)
})

And in the end it turns out that the documents from the doc2 array appear at the beginning of the collection, although I expect them to be inserted after the doc1 array, the collection is empty at the beginning of the operation. Tell me what am I doing wrong? I can't figure out why this is the result.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
B
Boniface, 2014-11-03
@Boniface

Hello! It's up to you, of course, but I strongly recommend not to use any ORMs. The usual js driver is very good and convenient.
On business. See if mongoose has the ability to do Bulk inserts. Here's what it looks like in the standard version (without orm)

var bulk = db.items.initializeUnorderedBulkOp();
bulk.insert( { item: "abc123", defaultQty: 100, status: "A", points: 100 } );
bulk.insert( { item: "ijk123", defaultQty: 200, status: "A", points: 200 } );
bulk.insert( { item: "mop123", defaultQty: 0, status: "P", points: 0 } );
bulk.execute();

Read more here

F
fAra0N25, 2014-11-04
@fAra0N25

In fact, everything turned out to be easier, apparently a small amount of experience affects.
When I tested the code on small inserts, there were few documents in the collection 50 or less, then when requested through the Mongo console, through RobotoMongo, and through moongose, the documents are returned in the order in which they were inserted, and when there are more documents in the collection (there were 15k), then the selection occurs from the end of the collection (findOne in mongoose, find in Monogo console, and through RobotoMongo).
I decided that I added the created field and added sort: {created: 1} to the request, i.e. ascending. Temporarily solved the problem.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question