How to start executing multiple Futures at the same time?

Question

Python

I am writing a backend on tornado, there is a blocking function

class Recognition:
  _thread_pool = ThreadPoolExecutor(max_workers=cpu_count())
  ...
  @run_on_executor(executor="_thread_pool")
    def blocking_func(self,  n):
        # code
        return result

This blocking function must be called for all elements of the array, I do this:

@gen.coroutine
def get_res(self):
  a = [1, 2, 4, 7, 9...] # 13 элементов
  result = yield map(self.blocking_func, a)
  raise gen.Return(activations)

Well, accordingly I call:
r = yield self.get_res()
Execution of a blocking function for one element takes 0.35 seconds. But the get_res execution takes 4-5 seconds. Tried gen.multi() , doesn't help.
Perhaps some kind of asynchronous map is needed?

Answer 1

In general, it's not about Future, but about GIL. The blocking function was CPU dependent, so the ThreadPool didn't help. I solved the problem through multiprocessing.Pool, apply_async and gen.Task.