R
R
rust212015-04-04 23:46:12
Unicode
rust21, 2015-04-04 23:46:12

How to read C++ Unicode file with Russian text?

Hello everyone) There was a task to read Unicode files with content in Russian. To solve the problem, I created a test file in Notepad, filled it with Russian text and saved it as Unicode. After I tried to read it in the following ways:
1)

std::wstring readUtf16(const std::string &filename) {

    std::ifstream file(filename.c_str());
    std::stringstream ss;
    ss << file.rdbuf() << '\0';
    return std::wstring((wchar_t*)ss.str().c_str());
}

Somewhere in the program:
std::wstring ws = readUtf16("test.txt");

    std::string s(ws.begin(),ws.end());
    std::cout << s.size() << " "<< s;

This method displays the correct length of the string s, but displays krakozyabry in the console (I tried to insert setlocale at the beginning)
2) I tried to use the approach using wstring and wchar_t using wifstream. Zero effect at all (nothing is output to the console).
3) Tried to use encoders from codecvt like this:
const std::locale utf16_locale_in = std::locale(std::locale::empty(),
                                                new std::codecvt_utf8_utf16<wchar_t>());
    {
        std::wifstream input("test.txt");
        std::wstring ws;
        input.imbue(utf16_locale_in);
        std::getline(input,ws);
        std::wstring_convert<std::codecvt_utf8_utf16<wchar_t>,wchar_t> converter;
        std::string s = converter.to_bytes(ws);
        std::cout << s << " " << s.size();
    }

Again, no effect. There were no problems with reading the English text. Ask for help from knowledgeable people.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
X
xibir, 2015-04-05
@xibir

it works like this for me:
std::ifstream f("/temp/u.txt", std::ifstream::binary);
f.seek(0, f.end);
int size = f.tellg();
f.seek(0, f.beg);
std::u16string us1;
us1.resize(size / 2);
f.read((char*)&us1[0], 2 * us1.length());
// us1 now contains a utf16 string

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question