Answer the question
In order to leave comments, you need to log in
How to organize multithreading to run subprocess?
There is a need to run subprocess in threads with a controlled number of threads, with the ability to add tasks on the fly.
While it is implemented like this, I start the number of processes from the multiprocessing module equal to the number of processes, I throw the tasks that need to be performed into the queue, as the tasks run out, the threads died, I start new tasks, I look at the previous threads is_alive(), if they are dead, I kill terminate() and start the threads again, if new tasks have been added, the previous ones have not yet completed, they are simply queued and executed by old threads, since they have not yet died.
map is not satisfied with the fact that there is no way to add new tasks, only by creating a new pool.
Is there a more rational solution? To run several subprocesses with a controlled number of threads and the ability to replenish the tasks that need to be run. Data from subprocess is not required, just run in async
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question