T
T
Timofey2016-02-23 20:37:47
C++ / C#
Timofey, 2016-02-23 20:37:47

How to read a large file of about 2 million lines in C#?

Colleagues, there is a task to process a file of 2,000,000 lines. Using the StreamReader class only makes it possible to process about 100,000 rows, and then the error "Additional information: CLR could not transition from COM context 0x103bec8 to COM context 0x103bf80 in 60 seconds. It is most likely that the thread that owns the destination context / apartment, is idle or performing a very long operation without pumping Windows messages."
Code example:
StreamReader sr = new StreamReader(fileName, Encoding.GetEncoding(866));
stringline = String.Empty;
while ((line = sr.ReadLine()) != null)
{
richTextBox4.AppendText(sr.ReadLine() + "\n");
}

Answer the question

In order to leave comments, you need to log in

3 answer(s)
M
Melz, 2016-02-23
@timzab

For visual tracking of recoding, 20 lines are enough, or how many lines fit into your window.
If you need more - reload the scrolling with the mouse and the scrollbar and read new lines.
You can write something like this:

protected void MyFile(string FilePath, string NewFilePath)
{
    using (StreamReader vReader = new StreamReader(filePath, Encoding.GetEncoding(866)))
    using (StreamWriter vWriter = new StreamWriter(newFilePath, true , Encoding.GetEncoding(866)))
    {
        while (!vReader.EndOfStream)
        {
            string vLine = vReader.ReadLine();
            vWriter.WriteLine(vLine);
        }
    }
}

If you still want in memory, then take MemoryMappedFile

M
Michael, 2016-02-23
@Sing303

And that doesn't work?
richTextBox4.LoadFile(fileName, RichTextBoxStreamType.PlainText);

T
Timofey, 2016-02-23
@timzab

The point is not to look at richTextBox4, but to track whether the text has been re-encoded, and after the end of the process, write the file with the required encoding.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question