R
R
ReDeNDeR2020-12-27 17:41:38
Python
ReDeNDeR, 2020-12-27 17:41:38

Python parser does not work with proxy, what should I do?

I made a python parser, it works without a proxy, but when I add a proxy, such an exception is displayed

Traceback (most recent call last):
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 667, in urlopen
    self._prepare_proxy(conn)
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 932, in _prepare_proxy
    conn.connect()
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connection.py", line 317, in connect
    self._tunnel()
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\http\client.py", line 900, in _tunnel
    (version, code, message) = response._read_status()
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\http\client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\socket.py", line 669, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\urllib3\util\retry.py", line 446, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='2ip.ru', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Rasul\Desktop\Multi parser\Multi_parser.py", line 24, in <module>
    parse()
  File "C:\Users\Rasul\Desktop\Multi parser\Multi_parser.py", line 13, in parse
    response =  requests.get(URL, proxies=proxies)
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\Rasul\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\adapters.py", line 510, in send
    raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPSConnectionPool(host='2ip.ru', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)))
>>>


here is my code
import requests
from bs4 import BeautifulSoup
from random import choice

def parse():
        URL = 'https://2ip.ru/'
        proxies = {
            "http": 'http://24.181.205.134:54321',
            "https": 'https://24.181.205.134:54321'
            }
        
        
        response =  requests.get(URL, proxies=proxies)
        soup = BeautifulSoup(response.content, 'html.parser')
        items = soup.findAll('div', class_ = 'ip-info_left')
        comps = []
 
        for item in items:
            comps.append({
                'title': item.find('div', class_='ip').find('span').get_text(strip=True)
                })
            for comp in comps:
                print(comp['title'])
parse()       
input("a")

Answer the question

In order to leave comments, you need to log in

1 answer(s)
V
Vladimir Kuts, 2020-12-27
@fox_12

Use a different proxy
Your example with a different, working proxy worked well for me.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question