Answer the question
In order to leave comments, you need to log in
How (with what) to parse a huge JSON file?
Friends asked a problem: there is a huge json file with texts, numbers and boolean values.
What is the most convenient way to check it and extract data from it with the prospect of saving it in other formats such as csv or txt?
Huge - it's one and a half to two gigs.
Preferred language is php at LAMP/WAMP. If there is something very good that does not require deep learning of C # / C ++, it will also do. Third-party software will also do (I googled it, I didn’t find anything interesting).
PS I don't know where my friends got such a file from and how they made it. This question does not concern me.
Answer the question
In order to leave comments, you need to log in
Luckily, there is a great library for parsing large JSON files: https://github.com/kuma-giyomu/JSONParser
Here's another one:
https://github.com/salsify/jsonstreamingparser
if there is knowledge of java, even superficial, then Jackson . I don’t know any other event parsers, but only such parsers can read huge files. The typical "counted and processed" approach will not work here.
However, if you put more restrictions on php on your own machine, then file_get_contents + json_decode will take something around 4 GB of RAM, so if there is more of it, then it’s quite possible to do everything in a simple way.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question