Answer the question
In order to leave comments, you need to log in
What program can quickly convert json to csv?
Please tell me which program can quickly convert complex and large (3 GB) JSON to CSV.
I tried to do it through Excel, it does well, but it takes a long time. Maybe there are other programs.
PS There are over 500 columns in the file.
Answer the question
In order to leave comments, you need to log in
In the first one, you need to manually specify the columns, the number of columns is large.
The second one fails to install. Some mistakes.
import csv
import sys
import typing
import json
def deep_walk(j, path=()):
if isinstance(j, dict):
for k, v in j.items():
yield from deep_walk(v, path + (f'.{k}',))
elif isinstance(j, list):
for i, v in enumerate(j):
yield from deep_walk(v, path + (f'[{i}]',))
else:
yield path, j
def json2csv(data: typing.Union[dict, list], dest: typing.TextIO = None):
field_set = set()
records = []
if isinstance(data, dict):
data = [data]
for item in data:
record = {''.join(path).lstrip('.'): value for path, value in deep_walk(item)}
records.append(record)
field_set.update(record.keys())
w = csv.DictWriter(dest or sys.stdout, fieldnames=list(sorted(field_set)))
w.writeheader()
w.writerows(records)
if __name__ == '__main__':
if sys.argv[1:]:
with open(sys.argv[1]) as f:
json2csv(json.load(f))
else:
json2csv(json.load(sys.stdin))
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question