How to scrape a site that loads as needed?

Question

Python

Goodnight!
I want to parse news from one site. The problem is that the entire tape is not loaded there at once. You need to wind it up to the end and only then it will load. How can this problem be solved?
For example:
News 1
News 2
News 3
As soon as I looked at them, News 4 and News 5 are loaded, etc.
The link to the site remains unchanged

Answer 1

You need to open dev tools and see what ajax request is sent to get the news. This is the request that needs to be sent.
and don't rush to use heavyweights like selenium ! :)

Answer 2

Find the last element (node) in the list: elementand scroll to it or a little lower (adding an additional 'top'constant to it).

function scrollIt(element) {
  window.scrollTo({
    'behavior': 'smooth',
    'left': 0,
    'top': element.offsetTop
  });
}

(in any case - processing on JS)