T
T
TchernyavskyDaniil2018-04-29 13:08:30
Python
TchernyavskyDaniil, 2018-04-29 13:08:30

Web scraping Python, 503 error, how to bypass?

Good afternoon. I'm trying to get to the site https://bhf.io/ using BS4 + Requests,
but I get a 503 error on a get request:
requests.get('URL', headers=header)

header = {
    'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
    'accept-encoding': 'gzip, deflate, br',
    'accept-language': 'en-US,en;q=0.8',
    'upgrade-insecure-requests': '1',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36'
}

blacklist_bhf = 'https://bhf.io/forums/137/'

def beautifulsoup(url_content):
    return BeautifulSoup(url_content, 'lxml')

# Error 503 here
page_bhf = requests.get(blacklist_bhf, headers=header)
print(page_bhf)

soup = beautifulsoup(page_bhf.content)

Can someone please suggest how to get around this? Preferably with an example. Thanks :)

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question