Answer the question
In order to leave comments, you need to log in
Python parser does not work with proxy, what should I do?
I made a python parser, it works without a proxy, but when I add a proxy, such an exception is displayed
Traceback (most recent call last):
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 667, in urlopen
self._prepare_proxy(conn)
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 932, in _prepare_proxy
conn.connect()
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connection.py", line 317, in connect
self._tunnel()
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\http\client.py", line 900, in _tunnel
(version, code, message) = response._read_status()
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\http\client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\socket.py", line 669, in readinto
return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\adapters.py", line 439, in send
resp = conn.urlopen(
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
retries = retries.increment(
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\util\retry.py", line 446, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='2ip.ru', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Rasul\Desktop\Multi parser\Multi_parser.py", line 24, in <module>
parse()
File "C:\Users\Rasul\Desktop\Multi parser\Multi_parser.py", line 13, in parse
response = requests.get(URL, proxies=proxies)
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\adapters.py", line 510, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='2ip.ru', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)))
>>>
import requests
from bs4 import BeautifulSoup
from random import choice
def parse():
URL = 'https://2ip.ru/'
proxies = {
"http": 'http://24.181.205.134:54321',
"https": 'https://24.181.205.134:54321'
}
response = requests.get(URL, proxies=proxies)
soup = BeautifulSoup(response.content, 'html.parser')
items = soup.findAll('div', class_ = 'ip-info_left')
comps = []
for item in items:
comps.append({
'title': item.find('div', class_='ip').find('span').get_text(strip=True)
})
for comp in comps:
print(comp['title'])
parse()
input("a")
Answer the question
In order to leave comments, you need to log in
Use a different proxy
Your example with a different, working proxy worked well for me.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question