B
B
bozuriciyu2019-10-09 21:59:41
MongoDB
bozuriciyu, 2019-10-09 21:59:41

Monga saves 600k records for almost a day?

There is a large array with data (sparged CSV) - approximately 600 records.
I save them like this

let count = 1;
    for (const videoArray of parsed) {
      console.log(count);
      const videoData = {};
      videoData.title = videoArray[1];
      videoData.description = videoArray[2];
      videoData.screenshot = videoArray[3];
      videoData.tags = videoArray[4].split(',');
      await videos.insertOne(videoData);
      count++;
    }

8 hours have passed, and only 230k records have been saved (according to the console). Is this normal, or did I do something wrong? for...ofit was not necessary?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alex Wells, 2019-10-09
@bozuriciyu

You save each record separately, and you also wait until each one is saved... Optimize: shove your parsed in chunks of 1000-5000 records at a time (depending on the size of the data).

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question