Answer the question
In order to leave comments, you need to log in
How to write more than 4 million documents to mongodb collection?
Situation
There is a need to write more than 4 million documents to the mongodb collection. The size of the document is not large, 20 fields, the longest field is no more than 255 characters.
At some point, when writing another document, PHP gave an error " Uncaught exception 'MongoCursorException' with message 'localhost:27017: Remote server has closed the connection' ".
The following appeared in the mongodb log " MapViewOfFileEx for ~/MongoDB-2.4.6/test.7 failed with error errno:8 ***. (file size is 536608768) in MemoryMappedFile::createPrivateMap Fatal Assertion 16167 ", from which I concluded that the database has exceeded its maximum allowable size (which was indicated by the calculation of the total size of files storing this database).
Question
Poke in the right direction. Maybe scaling, maybe some tools to bypass restrictions (like GridFS), maybe configuration settings for mongo itself.
Answer the question
In order to leave comments, you need to log in
If you only have a 32-bit system, you can try sharding. But I still recommend switching to 64-bit.
Monga uses mmap and resp. does not allow working with databases >2Gb on 32-bit systems .
For normal operation, a 64-bit platform, OS and monga itself are required.
If you only have x32 and a non-tiny base, then monga just isn't for you.
is there an alternative to mondo? the same nimble, but without such a limitation?
and what is the limit on the maximum size of the database for the x64 version?
try adding to config /etc/mongodb.conf
smallfiles = true
noprealloc = true
this will minimize files to 512MB on disk
smallfiles => Use a smaller initial file size (16MB) and maximum size (512MB) then
do
sudo -u mongodb mongod - -repair --dbpath /var/lib/mongodb/
if mongo won't start, then
rm /var/lib/mongodb/mongod.lock
sudo service mongodb restart
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question