Nigtmare.js explain the behavior of the evaluate method?

V

Vitaly2016-06-24 14:38:56

JavaScript

Vitaly, 2016-06-24 14:38:56

Good day everyone. I am looking for support exclusively from those who have worked with nightmare and understand it in action.
The essence of the question is as follows, based on an example from the documentation

var Nightmare = require('nightmare');
var nightmare = Nightmare({ show: true })
nightmare
  .goto('http://yahoo.com')
  .type('form[action*="/search"] [name=p]', 'github nightmare')
  .click('form[action*="/search"] [type=submit]')
  .wait('#main')
  .evaluate(function () {
    return document.querySelector('#main .searchCenterMiddle li a').href
  })
  .end()
  .then(function (result) {
    console.log(result)
  })
  .catch(function (error) {
    console.error('Search failed:', error);
  });

I am interested in the method of collecting information from a multi-page table, since evaluate passes the collected data through return, it is not entirely clear how you can use evaluate several times in one call, and even more so in a loop, and whether it is possible to act like this at all.
Conventionally, I'm interested in the following
1. went to the page
2. received data from the page via evaluate (returned somewhere)
3. went to the next tab through the click event and repeated step 2
in the simple case, the number of tabs is known in advance and cycles are not used.
3.a launched a click and evaluate in a loop and then returned the aggregated data to the user through then
Separately, I don’t quite understand before calling nightmare, you can declare any variable, for example var arr = [] and then try to push into it through evaluateb, however, abuse begins that this variable is not declared (I assume that asynchrony is to blame for this).
In general, I am ready to supplement the question with comments, detailing as necessary.
Thanks in advance.

Reply

Answer the question

In order to leave comments, you need to log in

2 answer(s)

L

likemusic, 2018-01-18
@vshvydky

Working code example:

const Nightmare = require('nightmare');
const nightmare = Nightmare({ show: true });
let results = [];
let res = nightmare
    .goto('http://yahoo.com')
    .type('form[action*="/search"] [name=p]', 'github nightmare')
    .click('form[action*="/search"] [type=submit]')
    .wait('#main')
    .evaluate(function () {
        return document.querySelector('#main .searchCenterMiddle li a').href;
    })
    .then(function(yahooResult) {
        console.log('Yahoo result:', yahooResult);
        results.push(yahooResult);
    })
    .then(function () {
        return nightmare.goto('http://google.com')
            .type('form[action*="/search"] [name=q]', 'github nightmare')
            .click('form[action*="/search"] [type=submit]')
            .wait('#main')
            .evaluate(function () {
                return document.querySelector('#search .g h3 a').href;
            })
        ;
    })
    .then(function (googleResult) {
        console.log('Google result:', googleResult);
        results.push(googleResult);
    })
    .then(function () {
        return nightmare.end();
    })
    .then(function() {
        console.log('Full results:', results);
    })
    .catch(function (error) {
        console.error('Search failed:', error);
    })
;

Nearly all nightmare methods return a pointer to themselves so that method calls can be chained together (like in jQuery).
The `evaluate()` method also returns this (not a promise as many people think)!
The exception methods that do not return `this` are the `.then()` and `.catch()` methods - they both return a Promise.
Promises themselves also have `.then()` and `catch()` methods, so when writing a script, you must clearly understand and follow the following methods on the Promise or NightmareJS object.
Also, nightmarejs, when calling chain methods, in fact simply adds the necessary actions to the command buffer. Commands from the buffer are only started when the `.then()` and `.catch()` methods are called.
So code like the one below won't even launch a browser or do anything at all, because it does not call methods that return promises.

Code that won't even launch the browser

nightmare
    .goto(url1).click(selector1).evaluate(func1)
    .goto(url2).click(selector1).evaluate(func1)
    .end()

But the following code will be executed by the browser:

Code to be executed in the browser

nightmare
    .goto(url1).click(selector1).evaluate(func1)
    .goto(url2).click(selector1).evaluate(func1)
    .end()
    .catch(errorHandler)

Also keep in mind that if the function called in the `.then()` method returns a promise, then that promise will also be resolved/fulfilled and the result of its work will be the return value passed to the second top-level `.then()`.

Code example with nested promises (https://jsfiddle.net/5kp62v7w/)

promise1 = Promise.resolve('Promise 1')
  .then(function(){ var str = 'Promise 1-1'; console.log(str); return str;})
  .then(function(){ var str = 'Promise 1-2'; console.log(str); return str;})
  .then(function(){ var str = 'Promise 1-3'; console.log(str); return str;})
  .then(function(){ 
    return Promise.resolve('Promise 2')
      .then(function(){ var str = 'Promise 2-1'; console.log(str); return str;})
      .then(function(){ var str = 'Promise 2-2'; console.log(str); return str;})
      .then(function(){ var str = 'Promise 2-3'; console.log(str); return str;})
      .then(function(){
        return Promise.resolve('Promise 3')
          .then(function(){ var str = 'Promise 3-1'; console.log(str); return str;})
          .then(function(){ var str = 'Promise 3-2'; console.log(str); return str;})
          .then(function(){ var str = 'Promise 3-3'; console.log(str); return str;})
        ;
      })
    ;
  })
;

promise1
  .then(function(res) {console.log('Result value:', res );})
;

Result of code execution

Promise 1-1
Promise 1-2
Promise 1-3
Promise 2-1
Promise 2-2
Promise 2-3
Promise 3-1
Promise 3-2
Promise 3-3
Result value: Promise 3-3
Therefore, when receiving data sequentially, the code should be of the following structure:
Example code when calling the `.evaluate()` method multiple times
nightmare.goto(url1).click(selector1).evaluate(func1).then(saveHandler1)
  .then(
     return nightmare.goto(url2).click(selector2).evaluate(func2).then(saveHandler2)
  )
  .then(
     return nightmare.goto(url3).click(selector3).evaluate(func3).then(saveHandler3)
  )
  .then(
     return nightmare.end()
  )
  .catch(errorHandler)
;
Loops can be implemented via the reduce() method, but at the moment I find it easiest to write using await/async functions.
An example of using `.evaluate()` in a loop with async/await functions
var urls = ['http://example.com', 'http://example2.com', 'http://example3.com'];

var results = []
async function collectTitles(urls){
  for ( url of urls ) {
    results.push(await nightmare.goto(url).wait('body').title())
  }
  
  await nightmare.end()
  console.dir(results)
}

collectTitles(urls)
You can't do that. The code passed to the `.evaluate()` method works in the browser environment, not the node server, so the only way to pass data from the browser code to the nodejs code is to return the value from the browser function using the return statement and use then this value inside `.than()` function as function argument:
An example of passing a value from a browser to an executable script
var nodeArray = [];

nightmare
   .goto(url)
   .evaluate(function(){
})
   .then(function(){ var ret = []; /* do some work with ret */ return ret;  }) //т.к. этот код выполняется в окружении браузера, то у него нет доступа к переменным объявkенным в окружении nodejs, в том числе и к nodeArray. Значение `ret` будет передано первым аргументом в последющей then()-ф-ции
   .then(function(browserRet){ nodeArray = nodeArray.concat(browserRet);  }) // browserRet равен значению ret вычисленному в браузере, но переменная ret для окружения сервера не определена.
In order to understand what is happening in the script, I recommend that you enable the output of debug messages at startup as described in the documentation .
I also recommend that you read the code examples and comments to them in the rosshinkley/nightmare-examples repository
Report

_
_ _, 2016-06-24
@AMar4enko
var Nightmare = require('nightmare');
var nightmare = Nightmare({ show: true })ж
var results = [];
nightmare
  .goto('http://yahoo.com')
  .type('form[action*="/search"] [name=p]', 'github nightmare')
  .click('form[action*="/search"] [type=submit]')
  .wait('#main')
  .evaluate(function () {
    return document.querySelector('#main .searchCenterMiddle li a').href;
  })
  .then(results.push.bind(results))
  .goto('http://google.com')
  .type('form[action*="/search"] [name=p]', 'github nightmare')
  .click('form[action*="/search"] [type=submit]')
  .wait('#main')
  .evaluate(function () {
    return document.querySelector('#main .searchCenterMiddle li a').href;
  })
  .then(results.push.bind(results))
  .end()
  .then(function() {
    console.log(results);
  })
  .catch(function (error) {
    console.error('Search failed:', error);
  });