How to implement such a parser ("eternal parser")?

Question

Python

Hello everyone, tell me what I need to learn in order to parse data that immediately appears on the site, for example, after 2 minutes, the info that I need and I want to get it will appear not by running my code at a certain time or every 2 minutes, but just so that he somehow immediately received this information and parsed for me, like an online parser. I heard something about websockets, maybe they will help, figs

Answer 1

Either you agree with the site owner to send the data as soon as they are received, for example, send an email, or pull the URL of your site.
Otherwise, it is NOT possible, except to simply periodically re-request data on the site on a schedule.

Answer 2

You need to watch exactly how the content appears on the site.
1. Either the site constantly sends requests to a special endpoint to receive updates, takes them from there and adds them to the site
2. The site receives updates via web socket
3. Statistics are not updated live, but appear after the page is refreshed.
Each situation will have a different approach.

Answer 3

Make your f5 parser a koon. I think a period of 1 second will be enough