M
M
MinTnt2021-01-04 00:17:38
Python
MinTnt, 2021-01-04 00:17:38

How to delay page loading while parsing?

For some of my purposes, I wanted to parse the logger I created on iplogger.ru
But the problem is that when loading a site with a list of clicks on the link, there is a "Loading list" and you need to wait a while until it passes. So with requests.get , it will only get the site data before it loaded the list.

That's interesting. Is it possible to somehow make a delay so that, for example, it connects to the site, and only after 10 seconds it counts it?
Or is it possible to somehow update the site data variable (also after 10 seconds)?

Ps since it's already night, it's difficult to express thoughts normally, therefore, if you need clarification on the issue, then in the comments.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
soremix, 2021-01-04
@MinTnt

No, requests work differently.
GET (like others, however) always returns the source code of the page. You can view this with CTRL+U. All additional downloads and other naovroty - either background requests, or the work of JS scripts.
Open F12 - Network. Reload the page and see what requests are sent. Most likely the necessary information will be in the XHR tab. Actually, find the right request and repeat it. If there is no data in any request (which is unlikely, I think), then they were loaded by some kind of script and you need to parse this through selenium

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question