How to merge csv files?

Question

Python

I found this code (below) on the Internet, rewrote it and changed it to fit my files. No matter how hard I tried (I have little experience with python) I couldn't fix it.

import os
import glob
import pandas as pd
os.chdir("C:\\Users\\Владелец\\PycharmProjects\\pythonProject\\eova")

extension = 'csv'
all_filenames = [i for i in glob.glob('*.{}'.format(extension))]

#combine all files in the list
combined_csv = pd.concat([pd.read_csv(f) for f in all_filenames ])
#export to csv
combined_csv.to_csv( "combined_csv.csv", index=False, encoding='utf-8-sig')

Lots of errors occur: "Traceback (most recent call last):
File "C:\Users\Owner\PycharmProjects\pythonProject\eova\wawa.py", line 10, in
combined_csv = pd.concat([pd.read_csv(f) for f in all_filenames ])
File "C:\Users\Owner\PycharmProjects\pythonProject\eova\wawa.py", line 10, in
combined_csv = pd.concat([pd.read_csv(f) for f in all_filenames ])
File "C:\Users\Owner\PycharmProjects\pythonProject\venv\lib\site-packages\pandas\io\parsers.py", line 610, in read_csv
return _read(filepath_or_buffer, kwds)
File "C:\Users\Owner\ PycharmProjects\pythonProject\venv\lib\site-packages\pandas\io\parsers.py", line 468, in _read
return parser.read(nrows)
File "C:\Users\Owner\PycharmProjects\pythonProject\venv\lib\site-packages\pandas\io\parsers.py", line 1057, in read
index, columns, col_dict = self._engine.read(nrows)
File "C:\Users\Owner\PycharmProjects\pythonProject\venv\lib\site-packages\pandas\io\parsers.py", line 2061, in read
data = self._reader.read(nrows)
File "pandas\_libs\ parsers.pyx", line 756, in pandas._libs.parsers.TextReader.read
File "pandas\_libs\parsers.pyx", line 771, in pandas._libs.parsers.TextReader._read_low_memory
File "pandas\_libs\parsers. pyx", line 827, in pandas._libs.parsers.TextReader._read_rows
File "pandas\_libs\parsers.pyx", line 814, in pandas._libs.parsers.TextReader._tokenize_rows
File "pandas\_libs\parsers.pyx", line 1951, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: Expected 2 fields in line 32, saw 3
"
How can they be fixed without hurting the essence of the code?

Answer 1

wowilon , for this code to work, the files must be of the same type, with the same number of columns. In the error code, it just indicates that in some files there are 2 columns, in others 3.
If you are sure that the files have the same structure, you can check that the column separator matches. In libraries for working with csv and pandas, including by default, a comma is used, while in the Russian locale, a semicolon is used by default. If your csv contains a semicolon, then you must explicitly specify in the parameter:

combined_csv = pd.concat([pd.read_csv(f, sep=";") for f in all_filenames ])