I
I
Issue2020-12-04 02:55:50
linux
Issue, 2020-12-04 02:55:50

Why doesn't the requests library work when checking code status in Python?

I wrote a script that checks the status of the server in this case, everything worked:

#!/usr/bin/env python3
import requests
import sys

domain = str('http://' + sys.argv[1])
position = 0
lines = 1
while position != lines:
  response = requests.get(domain)
  if response.status_code == 200:
    print('[200] Все хорошо: ' + str(domain))
  elif response.status_code == 404:
    print('[404] Страница отсутствует ' + str(domain))
  position += 1

Terminal output:

> python3 requester.py ya.ru
[200] All is well: https://ya.ru
> python3 requester.py ya.ru/bgdtrubi
[404] No page: https://ya.ru/bgdtrubi


But the response code does not always match the given ones, so I edited it to understand what code the script returns:
#!/usr/bin/env python3
import requests
import sys

domain = str('http://' + sys.argv[1])
position = 0
lines = 1
while position != lines:
  response = requests.get(domain)
  if response.status_code == 200:
    print('[200] Все хорошо: ' + str(domain))
  else:
    print('[' + str(response.status_code) + '] Не все хорошо: ' + str(domain))
  position += 1

When entering an existing domain, the terminal output is:

> python3 requester.py ya.ru
[200] All is well: ya.ru
> python3 requester.py ya.ru/bgdtrubi
[404] Not all is well: ya.ru/bgdtrubi

And then the problems started when I try to check the free domain - it doesn't even return the code. Terminal output:

Traceback (most recent call last):
File "/home/usr/.local/lib/python3.6/site-packages/urllib3/connection.py", line 157, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/home/usr/.local/lib/python3.6/site-packages/urllib3/util/connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/usr/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/home/usr/.local/lib/python3. 6/site-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.6/http/client.py", line 1264 , in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1310, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1259, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1038, in _send_output
self.send(msg)
File "/usr/lib/python3.6/http/client.py", line 976, in send
self.connect()
File "/home/usr/.local/lib/python3.6/site-packages/urllib3/connection.py", line 184, in connect
conn = self._new_conn()
File "/home /usr/.local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/usr/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/home/usr/.local/lib/python3.6 /site-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/home/usr/.local/lib /python3.6/site-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='yqqqqa. ru', port=80): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',))

During handling of the above exception,another exception occurred:

Traceback (most recent call last):
File "requester.py", line 9, in
response = requests.get(domain)
File "/home/usr/.local/lib/python3.6/site-packages/requests/api .py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/home/usr/.local/lib/python3.6/site-packages/requests/api. py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/home/usr/.local/lib/python3.6/site-packages/requests/sessions.py ", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/home/usr/.local/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/home/usr/.local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host ='yqqqqa.ru', port=80): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known',))


Googling gave me almost nothing, so many lines, it's not clear what exactly to google. I tried to disable certificate verification from one article: It didn't work... Please tell me what to do about it. How can I still get an answer (there is a domain and with what code, or does not exist)?
response = requests.get(domain, verify=False)

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
Sergey Karbivnichy, 2020-12-04
@paulenot

1) Please note that http and https are different things.
2) In this case (as well as in many others) it is necessary to handle exceptions.
The documentation for the requests library has a list of exceptions and their description: - Requests exceptions
Here you can handle the exception when we enter the address of a non-existent site (or not valid):

import requests
import sys

domain = 'http://' + sys.argv[1]

try:
  response = requests.get(domain)
  if response.status_code == 200:
    print('[200] Все хорошо: ' + domain)
  else:
    print('[' + str(response.status_code) + '] Не все хорошо: ' + domain)
except requests.ConnectionError:
  print(f'Сайта {domain} не существует')

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question