Answer the question
In order to leave comments, you need to log in
I am learning to parse the site, it does not display anything, what should I do?
import requests
from bs4 import BeautifulSoup as bs
headers = {'accept': '*/*',
'user-agent': 'Mozilla/5.0(X11;Linux x86_64...)Geco/20100101 Firefox/60.0'}
base_url = 'https://www.scoreboard.com/ru/football/#/'
out = 'output.txt'
def sb_parse(base_url, headers):
session = requests.session()
request = session.get(base_url, headers=headers)
if request.status_code == 200:
soup = bs(request.content, 'html.parser')
divs = soup.find_all('table', attrs={'class': 'soccer'})
for div in divs:
href = div.find('span', attrs={'padr'}).text
print(href)
else:
print('ERROR')
sb_parse(base_url, headers)
Answer the question
In order to leave comments, you need to log in
Chrome has a useful DevTools thing, it has a Network tab, if you go to it, you will see that a bunch of requests are leaving and in one of them you can see the data for the table:
Request URL: https://d.scoreboard.com/ru/ x/feed/f_1_0_5_ru_1
but this does not mean that you will also just receive an answer, at a cursory glance it looks like you first need to make a session.
and there is also a web socket) and data is running there. you can connect to it
And note that there is Accept-Encoding: gzip, deflate, br
there the table is loaded by ajax, of course, from BS there is no sense here. See where it comes from and in what form and take it to another address.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question