K
K
kodwi2014-07-20 13:58:10
C++ / C#
kodwi, 2014-07-20 13:58:10

Why does a C# application eat up 5 times the expected amount of memory?

I am writing an application to work with a file of a certain structure, the file size is ~ 40 mb. The configuration for this file is an XML file ~ 650 kb. When opening a file, I first read the configuration in XmlDocument, then I read the bytes of each element from the file into a List<>, linking them to the corresponding element from the configuration (i.e., I read the bytes of the element, created an object of the self-written Element class for it, where I wrote these bytes and reference to the node corresponding to the XmlNode element from the configuration, then added this object to the List<>). In total, approximately 4 - 4.2 kk elements (40 mb) are read. It seems to me that the application should eat up a maximum of 70-80 MB of RAM (file + links to configuration nodes), but in reality it comes out 300+ MB (in release). The code seems to be not complicated, there is nowhere to optimize. What could be the problem?

Answer the question

In order to leave comments, you need to log in

7 answer(s)
A
Alexander Vishnyakov, 2014-07-24
@kodwi

1. List<> allocates memory in chunks "in reserve". This is called capacity and the property of the same name is responsible for it . If you know that your list will contain, on average , no more than N elements, then you need to pass these N elements to the appropriate List<> constructor.
2. Bytes can be counted and alignment can be used . The process is time-consuming, but if there is a lot of “empty space” now, then in the end the desired ~ 80MB may well come out.
3. Well, the mentioned memory leaks. GC.Collect and memory profiling will help you.
Well, for people whining about the slowness of “managed” languages, large DLLs and 100 MB of memory for .NET by default and offering to switch to other languages, I suggest going to learn from English-speaking colleagues who don’t whine about every case, but try to really help figure it out in a problem, in this case — to increase productivity. Sorry, boiled up.

S
Sergey, 2014-07-20
Protko @Fesor

What's the problem? If you want to know exactly how much memory your program will use - write in C.
Are you sure that after parsing the document, no objects remain hanging? Maybe the GC has not worked yet, maybe the overhead for marshaling and so on ... In general, as for me, not such a big overhead came out.

B
barkalov, 2014-07-20
@barkalov

In the "adult" studio there is a memory analysis - ANALIZE - Launch Performance Wizzard.
Possibly leaking through circuits. Or the collector is not keeping up, try GC.Collect().

A
Alexander Taratin, 2014-07-20
@Taraflex

There is another question, how exactly do you determine how much the program eats?
If the code is not complicated, then I would rewrite the program in D - syntactically almost the same C #, but allows you to work at a lower level, maybe this could help you.

V
Vitaly, 2014-07-21
@vipuhoff

To check, insert GC.Collect() in each place that is executed more than 100 times and see what happens, if less memory is consumed, it means that somewhere garbage does not have time to be collected. Then you can remove one by one what you installed and see after which the memory began to grow again, in that place and figure out what is wrong in it.

D
Deerenaros, 2014-07-22
@Deerenaros

Oh, oh, it's scary.
Well, think for yourself, what is C#? This is CIL . And runtime CIL'a oh how the memory devours.
Actually, that's all for a recent question here about what a good programmer should know. And he must know what he is working with. If you knew that C# is a "virtual" language, that is, there is no real machine that would execute its code. It is compiled into the same CIL that will be executed by the virtual machine. She immediately reserves some memory, one hundred to one hundred and fifty megabytes, in order to quickly place newly created objects in it. In addition, several very large DLLs are loaded on which the .NET application "standardly depends".
However, it is not so easy to control memory in C either. At least there will be half a meter of overhead, if not cut with a knife..NET can also be cut with a knife , but it’s clear that it won’t work as efficiently as C.

A
aush, 2014-07-21
@aush

Take WinDbg, connect sos/psscor, take a dump and see what kind of objects you have in memory. You don’t need much experience here, there is a lot of information on analyzing .NET memory in this way on the Internet.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question