E
E
Eugene2017-02-15 16:44:23
Node.js
Eugene, 2017-02-15 16:44:23

What is the best way to organize parsing with authorization?

Task: You need to log in to the site (post request), and then parse the page with data (with a paginator).
I think to use request + cheerio for this. The
question is how best to proceed so as not to run into a callback hell, which is already planned.
Tell me your recipes for solving this problem.

request.get('auth get url', options, function (error, response, body) {
    if (!error) {
        var $ = cheerio.load(body);

        options.form = {
            admin_user: {
                email: '***',
                password: '***',
                otp_attempt: '***',
                remember_me: 1
            },
            authenticity_token: $('input[name=authenticity_token]').val(),
            utf8: $('input[name=utf8]').val()
        };

        request.post('auth post url', options, function (error, response, body) {
            request.get('parse data url', options, function (error, response, body) {
            
            });
        });

    } else {
        console.log("error: " + error);
    }
});

Answer the question

In order to leave comments, you need to log in

1 answer(s)
Z
Zakharov Alexander, 2017-02-15
@AlexZaharow

Slightly different from the technology you are using, but I would advise you to do it almost directly in the browser itself via CefSharp . But that's only if you know C#. In fact, you will in real life observe the loading of pages.
This is how, for example, how a page drain from a guest site might look like:
6ea17a7e113c4c0db58f568720aa8294.png

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question