A
A
alexh78542019-07-11 11:53:34
PHP
alexh7854, 2019-07-11 11:53:34

How to add to the database and process texts of different encodings in php?

Hello.
There is such a task - you can upload text files to the site. Files are divided into pages of 2000 characters and then the text can be viewed page by page. Also, the text is processed by regular expressions.
The site is multilingual, you can download texts in any language.
I do not quite understand what to do with encodings? By default, unicode / utf-8 seems to be used everywhere.
But what if the file encoding is not utf-8?
What encoding to choose for the database? How to avoid problems with regular expressions, they do not seem to work if the encoding is not unicode.
Tried to load text in another encoding, an error occurs due to encoding Incorrect string value: '\xF0\x90\x8D\x83\xF0\ .
I changed the base encoding to utf8mb4, then to utf-32, but the error still remained.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
P
Pavel, 2019-07-11
@PavelFokeev

But what if the file encoding is not utf-8?

Convert to UTF-8 and move on.

E
Evgeny Matveev, 2019-09-16
@ematveev

iconv()

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question