How to convert large text file to utf 8?

Question

linux

There are 40 gigs of a text file (was obtained by parsing sites). Naturally, in the encoding file there is a wagon and a bogie. Can you please tell me how to convert all this to utf 8?
PS There are a lot of different encodings in the file.

Answer 1

The correct answer is that it was necessary to convert when parsing.

Answer 2

I doubt that there is a way to convert a file to one encoding when different parts of it have different encodings.
And in general, why did you need to put everything in one file?
Wouldn't it be easier to write the received data to the database, a 40GB file...

Answer 3

the enconv command had to be used when parsing individual files.

Answer 4

It is necessary to split the file back, determine the encoding in each piece, re-encode it into a common encoding, and then connect the re-encoded back.