V
V
Vyacheslav Correct2017-10-07 15:05:30
Parsing
Vyacheslav Correct, 2017-10-07 15:05:30

What tools to parse a constantly changing AJAX page?

Good day) I asked myself this question, first with a commercial one, then with the idea of ​​​​curiosity. I have long since moved away from trying to program anything more than "a script on my knee to send mails from the database via cron".
In general, there is a task. The site itself is Marathon Bookmaker. There are live events on the main page. The coefficients are constantly updated in them.
In the developer tools, you can see that ajax requests are constantly sent at some interval (perhaps I am mistaken in determining the nature of the requests) to update these numbers in the windows in real time.
In connection with this, the question arose, in which direction to dig and what tools to use to get the entire page every 1-2 minutes, but also so that everything changes in real time.
My thoughts are that using PHANTOMJS, parse the entire page and display it on localhost, and update the page content using the same ajax requests that go on the site itself.
If these are the right thoughts, tell me if only one phantom and php is enough for me to implement this task?
Can the phantom see these requests and forward them to me in real time?
Wouldn't it be too noticeable to protect against parsing on the site from which I get everything?

Answer the question

In order to leave comments, you need to log in

3 answer(s)
D
Dmitry, 2017-10-07
@Sad_Bro

Ajax requests are sent at some interval..

maybe try to send requests similar to yourself and parse the response, and if json is returned there, then it's even easier.

U
Uno, 2017-10-07
@Noizefan

Post the full response of the ajax request.

M
MonsterAndrew, 2017-10-07
@MonsterAndrew

Why parse the entire page when you can catch an ajax request and send the same one. And if there is json, then almost nothing needs to be done.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question