A
A
Andrey Ivanov2022-04-08 22:31:23
proxy
Andrey Ivanov, 2022-04-08 22:31:23

Why doesn't proxy work when using aiohttp library?

I am studying the use of proxies in the requests and aiohttp libraries.
I wrote a little test code, but I can't figure out why using proxy in aiohttp library gives an error.
I bought a proxy for the test, when buying I provided information about which ports to use:
http/https 45785
socks5 45786

Here is the code itself:

import requests
import asyncio
import aiohttp
from bs4 import BeautifulSoup
from aiohttp_proxy import ProxyConnector


headers = {
    'User-Agent' : 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.119 YaBrowser/22.3.0.2430 Yowser/2.5 Safari/537.36'
}

proxies = {
    'https' : 'http://user:[email protected]:45785',
}

async def get_location(url):

    response = requests.get(url=url, headers=headers, proxies=proxies)
    soup = BeautifulSoup(response.text, 'lxml')
    ip = soup.find('div', class_='ip').text.strip()
    location = soup.find('div', class_='value value-country').text.strip()
    print(f'ПРОВЕРКА ПРОКСИ В REQUESTS:\nIP: {ip}\nLocation: {location}')

    print()

    connector = ProxyConnector.from_url('http://user:[email protected]:45785')
    async with aiohttp.ClientSession(connector=connector) as session:
        async with session.get(url, headers=headers) as response:
            text = await response.text()
            soup = BeautifulSoup(text, 'lxml')
            ip = soup.find('div', class_='ip').text.strip()
            location = soup.find('div', class_='value value-country').text.strip()
            print(f'ПРОВЕРКА ПРОКСИ В AIOHTTP:\nIP: {ip}\nLocation: {location}')


async def main():
    await get_location('https://2ip.ru')
    
if __name__ == '__main__':
    asyncio.run(main())


Part of the code using requests works fine, but when using aiohttp - an error is thrown that the parameter is set incorrectly:
ПРОВЕРКА ПРОКСИ В REQUESTS:
IP: 5.188.181.182
Location: Испания, Мадрид

Traceback (most recent call last):
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 969, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore  # noqa
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
    await waiter
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\asyncio\proactor_events.py", line 395, in _loop_writing
    self._write_fut = self._loop._proactor.send(self._sock, data)
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\asyncio\windows_events.py", line 529, in send
    self._register_with_iocp(conn)
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\asyncio\windows_events.py", line 718, in _register_with_iocp
    _overlapped.CreateIoCompletionPort(obj.fileno(), self._iocp, 0, 0)
OSError: [WinError 87] Параметр задан неверно

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\MyProjects\tst.py", line 38, in <module>
    asyncio.run(main())
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "D:\MyProjects\tst.py", line 35, in main
    await get_location('https://2ip.ru')
  File "D:\MyProjects\tst.py", line 26, in get_location
    async with session.get(url, headers=headers) as response:
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1117, in __aenter__
    self._resp = await self._coro
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 520, in _request
    conn = await self._connector.connect(
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp_proxy\connector.py", line 100, in connect
    return await super(ProxyConnector, self).connect(
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 535, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 890, in _create_connection
    _, proto = await self._create_proxy_connection(req, traces, timeout)
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1139, in _create_proxy_connection
    transport, proto = await self._wrap_create_connection(
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp_proxy\connector.py", line 91, in _wrap_create_connection
    return await super(ProxyConnector, self)._wrap_create_connection(
  File "C:\Users\79204\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 975, in _wrap_create_connection
    raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 2ip.ru:443 ssl:default [Параметр задан неверно]


What is the error and how can it be corrected?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Dr. Bacon, 2022-04-08
@TomRiddle

1. how to use a proxy in aiohttp - https://docs.aiohttp.org/en/stable/client_advanced... sorry didn't notice that socks5 is there, nevertheless you need to read the docs ;) either connector = ProxyConnector.from_url(' socks5 :// user:[email protected]:45785 '), or explicitly set proxy_type=ProxyType.SOCKS5
2. requests inside async, blocks the entire event loop

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question