O
O
oo22maxi2015-09-05 04:41:05
PHP
oo22maxi, 2015-09-05 04:41:05

What is the best way to get data from the site?

Greetings.
There is a site, the data on it changes quite often, maybe once every 1 second, the websocket works (which appears after authorization), the content of the site itself is loaded / generated by JavaScript.
I stumbled upon phantomjs, read it, it seems to cope with this task ...
But still, advise what is the best and fastest way to get data from such a site?
upd
I decided to use phantomjs, it remains only to deal with it better....

Answer the question

In order to leave comments, you need to log in

2 answer(s)
Y
yesworld, 2016-03-16
@oo22maxi

Yes, you need a Phantom.
Just specify a timeout delay in the loop, the time the site loads and parse.
Here is an example code for you:

var webPage = require('webpage');
var page = webPage.create();

page.open('https://example.ru/', function (status) {
    window.setTimeout( function() {
        var content =  page.content ;
        /* Парсите полученный html код динамического сайта  */
        fs.write("./info.log",'Время: ' + new Date().toLocaleString()+' | Value:'+ content +'\n' , 'a');
     
        phantom.exit();
    }, 10000);
});

D
Dmitry Novikov, 2015-10-01
@dmitriy_novikov

For interactive sites with dynamic content delivery, node.js is fine - but there's a lot to learn.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question