Answer the question
In order to leave comments, you need to log in
How to speed up proxy checker in python?
I check the proxy in python, if the proxy is dead, this check is very slow, is there any way to speed up this check?
import urllib2
import socket
def is_bad_proxy(pip):
try:
proxy_handler = urllib2.ProxyHandler({'http': pip})
opener = urllib2.build_opener(proxy_handler)
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib2.install_opener(opener)
req=urllib2.Request('http://www.yandex.ru/')
sock=urllib2.urlopen(req)
except urllib2.HTTPError, e:
print 'Error code: ', e.code
return e.code
except Exception, detail:
print "ERROR:", detail
return True
return False
def main(proxyList):
socket.setdefaulttimeout(100)
for currentProxy in proxyList:
if is_bad_proxy(currentProxy):
print "Bad Proxy %s" % (currentProxy)
else:
print "%s is working" % (currentProxy)
Answer the question
In order to leave comments, you need to log in
stackoverflow.com/questions/16646322/setting-the-t...
PS requests are easier to use.
You can use gevent, an example on your topic is here .
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question