V
V
vipermagi2016-10-26 00:40:54
C++ / C#
vipermagi, 2016-10-26 00:40:54

What problems arise when working with Unicode, and what solutions do they have in C++?

I'm trying to comprehensively review for myself the scope of working with text in general and adjusted for Unicode and C ++.
Cross-platform also want to take into account.
Question 1 : what are the known problems (if possible a complete list :) ) when working with text in general, for which a special solution is needed in Unicode?
Question 2 : what are the default tools for working with all this in C ++ and how far do they fall short / fall short of the ideal?
Question 3 : how much additional libraries might be needed?
At the moment, I have set myself the following tasks:

  • Iterator for moving through visual familiarity or codepoints
  • Determining the size (bytes) of a substring and cutting, pasting, etc.
  • Search and regex
  • Convert between UTF-* and other encodings
  • Read/Write both file contents and file system paths

Maybe there are some other things that need to be approached in Unicode?
The first thought is to take locale.h, regex.h and std::string and that's enough. But I think this idea is far from reality, I just don’t know how far.
The second thought is not to take a steam bath and immediately take ICU into any project. But this is fat and somehow the toad presses.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
sitev_ru, 2016-10-26
@sitev_ru

Look here https://github.com/sitev/cjCore/blob/master/src/ob... my String class. This is a UTF-32 string. There are pros and cons. We can join forces, write our own, more correct String!

A
Adamos, 2016-10-26
@Adamos

UTF8-CPP - used when I had to write without using cross-platform frameworks. Solve the main problems.
The main thing is to convert all incoming strings to utf-8 and don't forget to convert them to the output encoding, if necessary. And all processing is only in utf-8, ready-made solutions for this are more than enough.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question