Saving the tree structure to a file, best way?

Question

Java

Good day everyone, I ran into the problem of choosing a data structure and a file for storing the N-tree.
The object, in fact, is the following:

class Node{
   int data;
   Node[] childArray;
}

So I ran into problems:
a) Such a structure as an object in memory, with a number of nodes from ~ 10 ^ 6, consumes a lot of RAM memory. When converting an object to primitives and working with an array of data with offsets, the problem is solved and the memory is not eaten so much (about 3-5 times). t/e/ something like this:

int node = 0b1111111111_00000000_11111111;
// 1ые 8 бит отвечают за данные
// 2ые 8 бит child_size
// 3ие 8 бит информируют о необходимом смещении в массиве, чтобы дойти к следующему элементу на текущем уровне

b) Writing a tree to a file. When saving an object in XML / JSON, we get a very significant file from 50mb at N=~10^6. When creating your own bike and rewriting the tree for a linear record in a file, we get a very satisfactory version with 6-8 MB output, but there is a problem with scalability. T / e / to add an arbitrary node to the tree, you should go through all the nodes to correct the offsets of the nodes in the file and overwrite. Are there non-gluttonous data formats for storing an N-nar tree with scalability?

Answer 1

to add an arbitrary node to the tree, you should go through all the nodes to correct the node offsets in the file and overwrite.

It's inevitable anyway. The only thing that can be done to reduce disk I / O is, instead of writing a lot of pieces (via seek->write->seek->write), use the memory mapping mechanism built into the OS (in Java it is implemented via MappedByteBuffer ), then the optimization problem caching and flushing buffers to disk falls entirely on the OS.