Answer the question
In order to leave comments, you need to log in
How to write the result of each process into a separate corresponding file (Multiprocessing, Python)?
Briefly:
There is a script that splits a large array of data into groups. Each such group of data runs in a separate process. It is necessary to make sure that each such group of data is written to the target file for it.
I need to make the code find the file where the data needs to be written
Simplified version of my code (for clarity)
# Разбили массив данных на три группы. Каждому процессу - одну группу из списка
data_group =
# Функция записи данных в конкретный файл
def write_data(data, file_name):
with open(file_name, mode='w', encoding="utf-8") as file:
names = ["Колонка1", "Колонка2"]
file = csv.DictWriter(file, delimiter = ",", lineterminator = "\r",
fieldnames = names)
# Запись данных в колонку
for item in data:
file.writerow(item)
# Функция анализа данных
def func(data_group):
data = []
# Какой-то анализ группы данных с последующей записью в файл
for item in data_group:
first = item[0]
second = item[1]
third = item[2]
data.append(first, second, third)
write_data(data, file_name)
# Запуск в 3 процессах
with Pool(3) as pool:
pool.starmap(func, product(data_group, file_name))
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question