G
G
Grigory Vasilkov2017-06-09 00:37:20
Node.js
Grigory Vasilkov, 2017-06-09 00:37:20

NodeJS - How to make a cyclic then on Promise () - make requests in batches of 5 pieces?

Can you please explain how to write request queues without recursion if you use promises? I'm completely confused. I post my code, and I don’t understand where to insert the time delay correctly.

do {
      current_partnums = partnums.splice(0, limit);
      Promise.all(ps).then((function (current_partnums) {
        ps = current_partnums.map(function (partnum) {
          // get google results for each part
          return getGooglePartResults(partnum)
            // after search parse pages
            .then(function (part) {
              return parseGooglePartResults(part);
            });
        });
      }).bind(null, current_partnums));
    } while (partnums.length);

Of course, Google gives out what the captcha is and what the bot is when a pack of 1000 search queries comes in. I would like to set a delay of 5 seconds between each request.
I added a request with a timeout to ps.push(), but it simultaneously creates 1000 timeouts and they all end after 3 seconds, and not after each request 3 seconds. Help

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Alexander Aksentiev, 2017-06-09
@Sanasol

https://github.com/sindresorhus/p-queue

C
Coder321, 2017-06-09
@Coder321

If I understood the question correctly, then such a banal solution should help

return new Promise((resolve) => {
    setTimeout(() => {
        return resolve(parseGooglePartResults(part));
    }, 5000)
})

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question