L
L
Lorem Ipsum2014-12-06 14:23:24
Python
Lorem Ipsum, 2014-12-06 14:23:24

What Python libraries can you recommend for multithreading?

I want to start right away with the goal - I need to quickly parse some data from web pages, usually these are HEAD requests, GET requests with an incomplete response (for example, the first 128 bytes of the response), as well as banal things.
Previously, I used the Threading and Queue Python modules directly for this, and they completely suited me, since I needed more or less stable work on 10-20 threads. Now I need to increase this number by 20-30 times.
What libraries can you advise or tell me in which direction it is worth digging.
Thank you.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
T
thenno, 2014-12-06
@thenno

Python 3 has out of the box asyncore and asyncio for all kinds of async. You can dig in this direction.

L
lPolar, 2014-12-07
@lPolar

Grab and its Spyder are parsing-oriented, multi-threading is well implemented there, plus there are docks in Russian.
Post on Habré from the author of the module - habrahabr.ru/post/134918

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question