S
S
semolex2015-02-19 15:54:45
Python
semolex, 2015-02-19 15:54:45

How to make a multi-threaded URL parser in Python?

Hello!
I would like to ask you how to make a multi-threaded url resource parser using Python?
There is this code:

import urllib2
import threading

class JsonRequester(object):
    resource = None
    def __init__(self, *args):
        self.resource = args
    def get_json_req(self):
        for url in self.resource:
            req = urllib2.urlopen(url)

How to make a separate thread for each of the arguments that are passed in the initializer?
Thank you!

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
alex stephen, 2015-02-19
@semolex

habrahabr.ru/post/149420 here in this article the threads are described in some detail

A
Alexey Cheremisin, 2015-02-19
@leahch

But on my favorite twisted - technicae.cogitat.io/2008/06/async-batching-with-t...
And yes, not a single thread was affected, everything is in one thread!

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question