H
H
Hanry6542020-03-12 21:19:08
PHP
Hanry654, 2020-03-12 21:19:08

How to sort json like on JSON placeholder?

There is a json file with 70k lines, is it possible to do something like on https://jsonplaceholder.typicode.com/todos?_page=2

Tobish page=1 - the first ten is displayed
page=2 - the second ten is displayed, etc.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
Sergey Pankov, 2020-03-12
@Hanry654

You need to first convert your file (in which, apparently, there is a list at the top level) into a list of jsons nested in this list:
jq -c ".[]"
Then split into groups of N pieces:
split -l 10 -d - my_chunk_prefix_
We get my_chunk_prefix_* files, but each of them does not contain valid json, but json -strings. They need to be converted to regular josns.

for f in my_chunk_prefix_*; do cat $f | sed '1s/^/[/; $!s/$/,/; $s/$/]/' | jq "" > $f.json ; rm $f ; done

Everything. On this you have a paginated json file as a static file.
So again:
jq -c ".[]" my_big.json | split -l 10 -d - my_chunk_prefix_

for f in my_chunk_prefix_*; do cat $f | sed '1s/^/[/; $!s/$/,/; $s/$/]/' | jq "" > $f.json ; rm $f ; done

So-so solution for readability, I'm sure that json assembly from strings could be done without sed on pure jq, but there is no time to read the manual.
If the splitting needs to be done dynamically, then I would store the large json as one large jsons (json lines), then I would make a shell script with pagination parameters and select the necessary N lines in it modulo the number of elements per page and convert it to json on the fly.
In general , read the jq manual , it is possible that there is a ready-made way to do pagination in one simple step instead of all these bash dances.
You can visually practice with jq magic here: https://jqplay.org/.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question