D
D
Daniel Newman2015-03-07 21:45:28
API
Daniel Newman, 2015-03-07 21:45:28

How to make an EventEmitter for multiple simultaneous requests in NodeJS?

I'm requesting two APIs that return source and data store data. I fill the database of my application with them, comparing the source and stored data, for subsequent updating:

var dataStoredLoad = function( dataListOffset ) {
    eventEmitter.emit('dataStoredFetched');
}
var dataSourceLoad = function( dataListOffset ) {
    eventEmitter.emit('dataSourceFetched');
}

var middlewareDataToUpdate = function ( dataStoredLoad , dataSourceLoad ) {
    dataStoredLoad.forEach( function (dataStored, index) {
        dataSourceLoad.forEach( function (dataStored, index) {
            if ( dataStored.id == dataSource ) {
                 // key.value comparsion procedure and .pop() / .push()
            }
        }
    }
}

Both APIs work async, and the comparison only needs to be run when both data arrays are loaded into the database.
It is not known which API will return the values ​​first.
What can be done here and how beautifully others write it?
UPD:
Inside functions, API requests are made in a non-blocking async thread, and data fetching is done in a series of API calls, i.e. the solution with async.parallel suggested also below always falls into "callback(?)" function(err, results) where results is equal to: { one: ' ', two: ' ' }
Apparently I somehow have to question reformulate, but I do not understand how.
UPD2:
var answerAPI = crunchbase.request(arg1) ;
not null, from the moment of launch, but immediately some kind of '', waiting for the response of the API server. which throws us further along this waterfall of asynchronous functions to the result function, the function is processed with empty values, and then (in my case) the results of API requests fall into the console.
async.series({
  one: function(callback) {
      var answerAPIone = crunchbase.request(arg1);
      callback(null, answerAPIone );
  },
  two: function(callback) {
      var answerAPItwo = airtable.request(arg1);
      callback(null, answerAPItwo);
  }
},
function(err, results) {
  console.log( 'ИТОГО: <', results.one, '><', results.two, '>' );
  // ИТОГО: <><>
});

Everything rested on the correct writing of a function with a callback and the use of async for its intended purpose. The remaining options have remarkably expanded horizons. Thanks to all.

Answer the question

In order to leave comments, you need to log in

5 answer(s)
T
Timur Shemsedinov, 2015-03-07
@danielnewman

https://github.com/caolan/async

async.series({
  one: function(callback) {
    setTimeout(function() {
      callback(null, 1);
    }, 200);
  },
  two: function(callback) {
    setTimeout(function() {
      callback(null, 2);
    }, 100);
  }
},
function(err, results) {
  // results is now equal to: { one: 1, two: 2 }
});

Y
Yuri Puzynya, 2015-03-08
@3y3

I advise you to look towards promises.
Start with the canonical promise
library. Once you get the hang of it, you may need a more complex combine, look towards Q.

// npm install promise

var fs = require('fs');
var Promise = require('promise');

// В качестве источника данных в данном случае файловая система,
// в вашем случае это видимо будет net.Connection
function dataStoredLoad( dataName ) {
    return new Promise(function(resolve, reject) {
        fs.readFile(dataName, 'utf8', function(err, data) {
            if (err) return reject(err);

            resolve(data);
        });
    })
}

// Это другой способ записать то, что написано в dataStoredLoad
function dataSourceLoad( dataName ) {
    return Promise.denodeify(fs.readFile)(dataName, 'utf8');
}

Promise.all(dataStoredLoad, dataSourceLoad).then(function(res) {
    var stored = res[0],
          sourse = res[1];

    stored.forEach( function (dataStored, index) {
        sourse.forEach( function (dataStored, index) {
            if ( dataStored.id == dataSource ) {
                 // key.value comparsion procedure and .pop() / .push()
            }
        }
    }
});

D
dtestyk, 2015-03-08
@dtestyk

Once, for a similar problem, a colleague suggested counting events.
also, you can use Promise.all(db_promise, src_promise), with some effort and Promise.race will even be able to handle the timeout.

A
Alexey Komarov, 2015-03-12
@Alex7Kom

First, in the case of async, parallel is more suitable for you than series. async.parallel does not wait for the callback from the first function to be called and immediately starts the second one after the first one returns.
Secondly, the APIs you request must be (and most certainly are) asynchronous, and therefore immediately return nothing to you. You must pass a callback to this API, inside which to call the asinka callback.
Something like this:

async.parallel({
  one: function(callback) {
      crunchbase.request(arg1, function(error, answerAPIone){
        callback(null, answerAPIone);
      });
  },
  two: function(callback) {
      airtable.request(arg1, function(error, answerAPItwo){
        callback(null, answerAPItwo);
      });
  }
},
function(err, results) {
  console.log( 'ИТОГО: <', results.one, '><', results.two, '>' );
});

A
Alex, 2015-03-07
@isqua

https://github.com/dfilatov/vow

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question