Answer the question
In order to leave comments, you need to log in
What is the best way to get data from the site?
Greetings.
There is a site, the data on it changes quite often, maybe once every 1 second, the websocket works (which appears after authorization), the content of the site itself is loaded / generated by JavaScript.
I stumbled upon phantomjs, read it, it seems to cope with this task ...
But still, advise what is the best and fastest way to get data from such a site?
upd
I decided to use phantomjs, it remains only to deal with it better....
Answer the question
In order to leave comments, you need to log in
Yes, you need a Phantom.
Just specify a timeout delay in the loop, the time the site loads and parse.
Here is an example code for you:
var webPage = require('webpage');
var page = webPage.create();
page.open('https://example.ru/', function (status) {
window.setTimeout( function() {
var content = page.content ;
/* Парсите полученный html код динамического сайта */
fs.write("./info.log",'Время: ' + new Date().toLocaleString()+' | Value:'+ content +'\n' , 'a');
phantom.exit();
}, 10000);
});
For interactive sites with dynamic content delivery, node.js is fine - but there's a lot to learn.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question