O
O
Oleg Kulakov2022-02-10 18:55:30
Python
Oleg Kulakov, 2022-02-10 18:55:30

How can pandas data encoding problems be fixed when writing to Postgresql?

I download OKATO data from this resource https://rosstat.gov.ru/opendata/7708234640-okato .
File encoding - windows-1251
Next, I create a Dataframe and write to the PostgreSQL database:

import pandas as pd
df = pd.read_csv(target_link, encoding="windows-1251", delimiter=";")
df..to_sql(...)

When executing this code in Jupyter Lab on my laptop, I get entries in the database with the correct Latin alphabet.
However, when I run the same code on the server in Apache Airflow, I get the following picture in the database:
620534f40aa8e407864871.png
I can't figure out what the problem is?

My laptop
OS is macOS Monterey 12.1
Python version 3.9.7
Pandas version 1.3.4

Server:
Python version 3.7.10
Pandas version 1.1.4
Airflow version 1.10.15 I

use the same database in both cases. Encoding in the database - UTF-8

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question