U
U
Underground27132020-07-19 09:15:28
Python
Underground2713, 2020-07-19 09:15:28

How to scrape data from a website?

Data must be obtained from here smarthand.pro/ps/#Time2goo

import requests
from bs4 import BeautifulSoup
URL = 'https://smarthand.pro/ps/#Time2goo'
r = requests.get(URL)
soup =BeautifulSoup(r.text, 'html.parser')
items = soup.find('div', class_='buttons-stat-left-side')


I set a request to the site, I get an html page, but there are no necessary data in it, that is, elements with the required data.
This is what needs to be parsed. 5f13e3273516e441322331.jpeg
But I find the necessary data in the development panel, in the Elements tab in
5f13e3deca693058557661.jpeg
How can I get them from there, I don’t understand. Searching for a div by a given class does nothing.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
N
nokimaro, 2020-07-19
@Underground2713

In the dev tools, the network tab and the xhr section
Refresh the page, see the necessary requests
Repeat them using requests.

S
soremix, 2020-07-19
@SoreMix

I don't know how to get them from there. Searching for a div by a given class does nothing.

They are not on the page, you are looking at the rendered code when python sees the source. Press Ctrl+U and search there. But as mentioned above, in this case, you need to look at what requests this data comes from.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question