Answer the question
In order to leave comments, you need to log in
Why doesn't grequests with proxy work on hosting?
Good afternoon, I'm writing a parser for one site, split it into streams, it works according to the following principle - an array of proxies and an array of links commensurate with it are passed, everything is parsed using grequests. The question is that on my machine (windows 10) everything is parsed normally, here is the code:
proxies = []
authes = []
for proxy in raw_proxies:
host = proxy.split('@')[0]
auth_arr = proxy.split('@')[1]
auth = HTTPProxyAuth(auth_arr.split(':')[0], auth_arr.split(':')[1])
authes.append(auth)
proxies.append(dict(http='socks5://'+auth_arr+"@"+host, https='socks5://'+auth_arr+"@"+host))
for link, name, proxy, auth in zip(urls, names, proxies, authes):
rs.append(grequests.get(link, proxies=proxy, auth=auth))
results = grequests.map(rs)
def with_proxy():
proxy = '****:*****@******:*****'
host = proxy.split('@')[0]
auth_arr = proxy.split('@')[1]
auth = HTTPProxyAuth(auth_arr.split(':')[0], auth_arr.split(':')[1])
proxies=dict(http='socks5://'+auth_arr+"@"+host, https='socks5://'+auth_arr+"@"+host)
rs = [grequests.get('https://google.com', proxies=proxies, auth=auth)]
result = grequests.map(rs)
print('Запрос с прокси->', result)
def without_proxy():
rs = [grequests.get('https://google.com')]
result = grequests.map(rs)
print('Запрос без прокси->', result)
with_proxy()
without_proxy()
Запрос с прокси-> [<Response [200]>]
Запрос без прокси-> [<Response [200]>]
Запрос с прокси-> [None]
Запрос без прокси-> [<Response [200]>]
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question