Answer the question
In order to leave comments, you need to log in
What is the best way to organize parsing with authorization?
Task: You need to log in to the site (post request), and then parse the page with data (with a paginator).
I think to use request + cheerio for this. The
question is how best to proceed so as not to run into a callback hell, which is already planned.
Tell me your recipes for solving this problem.
request.get('auth get url', options, function (error, response, body) {
if (!error) {
var $ = cheerio.load(body);
options.form = {
admin_user: {
email: '***',
password: '***',
otp_attempt: '***',
remember_me: 1
},
authenticity_token: $('input[name=authenticity_token]').val(),
utf8: $('input[name=utf8]').val()
};
request.post('auth post url', options, function (error, response, body) {
request.get('parse data url', options, function (error, response, body) {
});
});
} else {
console.log("error: " + error);
}
});
Answer the question
In order to leave comments, you need to log in
Slightly different from the technology you are using, but I would advise you to do it almost directly in the browser itself via CefSharp . But that's only if you know C#. In fact, you will in real life observe the loading of pages.
This is how, for example, how a page drain from a guest site might look like:
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question