E
E
Elios2015-06-24 23:08:22
Python
Elios, 2015-06-24 23:08:22

How to speed up proxy checker in python?

I check the proxy in python, if the proxy is dead, this check is very slow, is there any way to speed up this check?

import urllib2
import socket

def is_bad_proxy(pip):    
  try:
    proxy_handler = urllib2.ProxyHandler({'http': pip})
    opener = urllib2.build_opener(proxy_handler)
    opener.addheaders = [('User-agent', 'Mozilla/5.0')]
    urllib2.install_opener(opener)
    req=urllib2.Request('http://www.yandex.ru/')
    sock=urllib2.urlopen(req)
  except urllib2.HTTPError, e:
    print 'Error code: ', e.code
    return e.code
  except Exception, detail:
    print "ERROR:", detail
    return True
  return False

def main(proxyList):
  socket.setdefaulttimeout(100)
  for currentProxy in proxyList:
    if is_bad_proxy(currentProxy):
      print "Bad Proxy %s" % (currentProxy)
    else:
      print "%s is working" % (currentProxy)

Answer the question

In order to leave comments, you need to log in

2 answer(s)
R
Roman Kitaev, 2015-06-24
@strelov1

stackoverflow.com/questions/16646322/setting-the-t...
PS requests are easier to use.

T
throughtheether, 2015-06-25
@throughtheether

You can use gevent, an example on your topic is here .

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question