N
N
Nikita2019-03-20 16:55:01
Node.js
Nikita, 2019-03-20 16:55:01

How to load a page completely with Puppeter?

When I load the page in the browser, it opens completely (with all the data), and when I load it through Puppeter with all the same arguments, it looks like a template is loaded, and not filled with data. Maybe the site is protected from programmatic login? Is there any way to bypass this protection?
I load the page like this:

const puppeteer = require('puppeteer');

        (async () => {
            const browser = await puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']});
            const page = await browser.newPage();
            await page.goto('url, {waitUntil: 'load', timeout: 0});
            await page.type('#login', login);
            await page.type('#password', password);
            await page.click('#sub-btn');
            await page.waitForNavigation().catch(() => console.log("catched"));
            await page.goto('url2',  { waitUntil: 'networkidle2' });
            await page.addScriptTag({url: 'https://code.jquery.com/jquery-3.2.1.min.js'});

const mainData = await page.evaluate(() => {
            try{
                return document.documentElement.outerHTML;
            } catch (e) {
                return e.toString();
            }
        });

        res.send(mainData);

        await browser.close();
}

Answer the question

In order to leave comments, you need to log in

2 answer(s)
N
Nikita, 2019-03-23
@snitron

Turned out it was the wrong URL.

H
hzzzzl, 2019-03-20
@hzzzzl

if on a quick hand, then try await page.waitFor(5000); so that exactly all javascripts are loaded and bring data from the server

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question