C
C
centrin02016-03-11 11:06:35
Python
centrin0, 2016-03-11 11:06:35

How to execute a task in parallel and get the result?

There is a project in python3.
There is a function that performs long calculations. At the end it returns a number.
These numbers, after several passes of calculations, need to be added.
How to start the execution of a function in several threads, wait for them to complete and get the values ​​from the thread?
Which library to use, thread, multiprocessing?

Answer the question

In order to leave comments, you need to log in

3 answer(s)
S
Sergey Gornostaev, 2016-03-11
@centrin0

Processor bottleneck? If yes, then multiprocessing. If IO, then thread is also possible, but gevent is better.
Something like

import multiprocessing as mp

def slow_calc(data):
    ...
    return result


pool = mp.Pool(processes=mp.cpu_count())
for res in pool.map(slow_calc, data_chunks):
    ...

A
Alexander, 2016-03-11
@Avernial

First example with Pool: python docs

V
Vov Vov, 2016-03-11
@balamut108

Now the most progressive library in this direction is asyncio, so dig better in that direction.
Here is the link: https://docs.python.org/3/library/asyncio.html

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question