T
T
teertsneerg2019-08-05 02:14:35
JavaScript
teertsneerg, 2019-08-05 02:14:35

Why can't I get nodejs site code?

I'm trying to parse the source code of the site with this script:

var request = require('request');

request({
    method: 'get',
    url: 'https://www.sitename',
    gzip: true,
        followRedirect: true,
    headers: {
      'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36'
    }
  }, (err, resp, body) => {
        console.log(err); 
        console.log(body);     
});

With most sites, everything is fine, the code is displayed, but there are problems with this. Everything is displayed through the browser, if I make a request through the script, I get the following error:
{ Error: read ECONNRESET
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:111:27) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
Alexander, 2019-08-05
@teertsneerg

Perhaps this site uses something more complex than just a request / response get request, for example, using a web socket, here you need to delve into the operation of the site.
If you do not want to spend time on this, you can look at puppeteer - a library from the developers chrome, which runs a headless version of chrome that can parse the page just as if it were manually opened in the browser

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question