C
C
cascado2016-06-20 18:47:25
Python
cascado, 2016-06-20 18:47:25

Why do coroutines, when run with asyncio, even if they exit gracefully, remain in memory?

Good afternoon. Essence of the question:
a limited pool of coroutines is used, when filling it is expected using the following construction:
done, pending = await asyncio.wait(pool, return_when=asyncio.FIRST_COMPLETED)
Moreover, if you constantly display the number of all tasks in memory using asyncio.Task. all_tasks(loop), then it is constantly growing, i.e. I delete even completed tasks from the pool, but they hang in memory. When there are more than 940, I start getting "MemoryError. Fatal read error on socket transport..." exceptions. I suppose the problem is aiohttp.get mishandling, but specifically.. What is wrong with them or me? I am attaching the code. https://gist.github.com/anonymous/3b4b4e2357c75de0...

Answer the question

In order to leave comments, you need to log in

1 answer(s)
C
cascado, 2016-06-24
@cascado

In general, the operation scheme of the event machine is such that completed tasks will be thrown out into the cold at the next opportunity, and will not necessarily be deleted immediately. A couple of scenarios were tested, when a large number of tasks spawn, the reactor starts to discard the excess more quickly. The memory error was obtained as a result of an oversight of a small but unpleasant bug.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question