Answer the question
In order to leave comments, you need to log in
Why doesn't http get request work for a single site?
Request the following function:
def simple_get(url):
"""
Attempts to get the content at `url` by making an HTTP GET request.
If the content-type of response is some kind of HTML/XML, return the
text content, otherwise return None.
"""
try:
with closing(get(url, stream=True)) as resp:
if is_good_response(resp):
return resp.content
else:
return None
from mathematicians import simple_get
import os
from bs4 import BeautifulSoup
raw_html = simple_get('https://dictionary.cambridge.org')
html = BeautifulSoup(raw_html, 'html.parser')
print(html)
Answer the question
In order to leave comments, you need to log in
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question