How to parse a large JSON file > 500mb?

Question

JSON

There is a huge json file with entries like this:

[
{"id":95689,
"name":"Имя",
"type":"message",
"description":"Описание",
"count":138},
{"id":959,
"name":"Имя",
"type":"message",
"description":"Описание",
"count":38}
]

How to effectively parse such a file? What library? Please provide examples.
Everything that I googled either does not work efficiently, or I just did not figure out how to use it correctly ...

Answer 1

What's the problem? not satisfied with the parsing speed, what have you tried?

Answer 2

You need to use a streaming JSON parser. Google it.

Answer 3

JSON.net has JsonTextReader.
It's most likely streaming.

string json = @"{
    'CPU': 'Intel',
    'PSU': '500W',
    'Drives': [
      'DVD read/writer'
      /*(broken)*/,
      '500 gigabyte hard drive',
      '200 gigabype hard drive'
    ]
}";

JsonTextReader reader = new JsonTextReader(new StreamReader(json));
while (reader.Read())
{
    if (reader.Value != null)
        Console.WriteLine("Token: {0}, Value: {1}", reader.TokenType, reader.Value);
    else
        Console.WriteLine("Token: {0}", reader.TokenType);
}

// Token: StartObject
// Token: PropertyName, Value: CPU
// Token: String, Value: Intel
// Token: PropertyName, Value: PSU
// Token: String, Value: 500W
// Token: PropertyName, Value: Drives
// Token: StartArray
// Token: String, Value: DVD read/writer
// Token: Comment, Value: (broken)
// Token: String, Value: 500 gigabyte hard drive
// Token: String, Value: 200 gigabype hard drive
// Token: EndArray
// Token: EndObject

Answer 4

I tried ServiceStack.Text
Parsing faster than Json.Net, but I could not figure out how to add something like filters there, so as not to parse the entire collection.
I'll try again using JsonTextReader, I'll write back a little later.

Answer 5

/// <summary>
    /// Deserialize json string in file to specified object
    /// </summary>
    /// <typeparam name="T">Specified object type</typeparam>
        /// <param name="filepath">Path to text file contained json string</param>
    /// <returns></returns>
    public static T DeserializeJsonFromFile<T>(string filepath)
    {
      T obj;

            using (var reader = new StreamReader(filepath))
      {
        using(var jsonReader = new JsonTextReader(reader))
        {
          var serializer = new Newtonsoft.Json.JsonSerializer();
          obj = serializer.Deserialize<T>(jsonReader);
        }
      }

      return obj;
    }