How to work with large JSON file?

Question

PHP

There is a JSON file about 600 MB. The site is on Wordpress and it has a search type form that returns the searched data from JSON.

Here is the question, how to work with such JSON? Or can it be converted to something else?

Before that I worked with json max. 20 Mb and resolved on JS, there were no problems. And then 600 MB and you just can’t throw out all 600 MB to the user ...

Answer 1

Parse JSON into a database. With the right indexes, searching will be very fast.

Answer 2

I had to work with JSON files of several gigs in size (6 - 7, if memory serves). Such a large file is loaded for a long time and monolithically, it does not fit well into memory in the form of an object model, in general, the idea of ​​\u200b\u200bcreating such files is not good, but the data provider provides them in this form and it is necessary to work.
Therefore, I built a simple utility , which is a kind of SAX parser, but for JSON.
The utility takes a huge JSON as a text data stream, responds to opening and closing tags, and outputs pieces element by element as a sequence of individual JSON lines separated by paragraphs (in accordance with RFC7464 ).
It is much easier to deal with such a stream of small pieces of the same type.
Then, by the way, I was told that the well-known and wonderful JQ utility can also work in this mode (SAX). But by that time I had already solved my task and JQ did not go deep into the intricacies of the command line parameters for this mode.