V
V
Valeriy Maksimov2018-12-04 10:28:11
MySQL
Valeriy Maksimov, 2018-12-04 10:28:11

How to solve encoding error (Python and MySQLdb)?

For help! I started working with Python very recently. Wrote a scraper that pulls metadata from Google Play. The data is taken and displayed normally (i.e. Cyrillic does not fray), but when I try to write this data to the database (I use the MySQLdb module), as soon as an entry with Cyrillic is found in the field, it throws the following:

_mysql_exceptions.OperationalError: (1366, “Incorrect string value: '\\xF0\\x9F\\x8D\\x80' for column 'title' at row 1”)

Base, tables and fields encoding utf8_general_ci.
I establish a connection with the database as follows:
conn = MySQLdb.connect('localhost', 'VProgramMist', '1998Vm0000', 'mobasta_history', charset='utf8', init_command='SET NAMES UTF8')

I write the data to the database as follows:
sql_str = """INSERT
             INTO history (app_id, title, score, price, free, iap,
             iap_range, size, installs, content_rating,
             date_point, category, collection)
             VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)"""

        cursor.execute(sql_str, (
            row['app_id'], row['title'], row['score'],
            row['price'], row['free'], row['iap'],
            row['iap_range'], row["size"], row['installs'],
            row['content_rating'][0], datetime.date.today(),
            row['category'][0], row['collection']))

Python version 3.7
MySQL version - 5.7
PS Decided to record every read record. I saw that Cyrillic lines are written to the database. Those. he writes something, but on something (for example, when he comes to the Yula application), everything goes to hell ...

Answer the question

In order to leave comments, you need to log in

1 answer(s)
V
Vadim Shatalov, 2018-12-04
@netpastor

https://mysqlclient.readthedocs.io/user_guide.html...
use_unicode
If True, CHAR and VARCHAR and TEXT columns are returned as Unicode strings, using the configured character set. It is best to set the default encoding in the server configuration, or client configuration (read with read_default_file). If you change the character set after connecting (MySQL-4.1 and later), you'll need to put the correct character set name in connection.charset.
If False, text-like columns are returned as normal strings, but you can always write Unicode strings.
This must be a keyword parameter.
charset
If present, the connection character set will be changed to this character set, if they are not equal. Support for changing the character set requires MySQL-4.1 and later server; if the server is too old, UnsupportedError will be raised. This option implies use_unicode=True, but you can override this with use_unicode=False, though you probably shouldn't.
If not present, the default character set is used.
This must be a keyword parameter.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question