How to wait for for loop with async requests to finish in node.js?

Question

I want to make multiple requests in node.js to get a couple of external API responses, to combine them into one array. I'm using a for loop to achieve this. Here's my code:

res.setHeader('Content-Type', 'application/json');
const sub = req.query.days_subtract;
const enddate = req.query.end_date; 

var array = [];

for (var i = 0; i < sub; i++) {
  request("https://api.nasa.gov/planetary/apod?date=" + subtractDate(enddate, i) + "&api_key=DEMO_KEY", function(error, response, body) {
    array.push(body); 
    // console.log(body);
  });
}
res.send(array);

but this piece of code returns [] all the time. I know it's because the for loop only starts these async requests, but it doesn't wait for them to finish. I tried to use async/await but that didn't work either. So how to wait for this loop to finish getting requests and to finish pushing them to the array so it can be shown to users?

attach code with async await, it should work or use sync-request library — bxN5
– bxN5, Commented Oct 9, 2018 at 15:35

Chris Diggs · Accepted Answer · 2020-04-28 17:22:05Z

28

For your use case, using await with Promise.all is probably the most performant way to go about it. Your code should look something like this:

res.setHeader('Content-Type', 'application/json');
const sub = req.query.days_subtract;
const enddate = req.query.end_date;

var promiseArray = [];

for (var i = 0; i < sub; i++) {
    promiseArray.push(new Promise((resolve, reject) => {
        request("https://api.nasa.gov/planetary/apod?date=" + subtractDate(enddate, i) + "&api_key=DEMO_KEY", function(error, response, body) {
        if (error) reject(error);
        else resolve(body)
    })
  }))
}
res.send(await Promise.all(promiseArray));

edited Apr 28, 2020 at 17:22

Chris Diggs

3232 silver badges9 bronze badges

answered Oct 9, 2018 at 15:44

Ayush Gupta

9,36511 gold badges63 silver badges98 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Kon · Accepted Answer · 2019-10-21 13:49:16Z

9

Use something like request-promise. If you await each request in the for loop, then all requests are done in series. If each request is independent and doesn't need to be done one after the other, this is inefficient. The best way to handle something like this is to do them all in parallel and then wait for them all to complete. This is done by creating an array of request promises and then using await Promise.all(promiseArray) to wait for all promises to resolve.

var promises = [];

for (var i = 0; i < sub; i++) {
  const promise = request("https://api.nasa.gov/planetary/apod?date=" + subtractDate(enddate, i) + "&api_key=DEMO_KEY");
  promises.push(promise);
}

const array = await Promise.all(promises);

edited Oct 21, 2019 at 13:49

answered Oct 9, 2018 at 15:45

Kon

4,0994 gold badges26 silver badges38 bronze badges

5 Comments

Ayush Gupta Over a year ago

request does not return a promise by default

Kon Over a year ago

That's why I suggested using request-promise.

Ayush Gupta Over a year ago

ah nvm, focused on the code not the text, apologies

Ramesh_D Over a year ago

great bro.. its very simple

jet_choong Over a year ago

Using request-promise indeed make the code cleaner and more readable

Evert · Accepted Answer · 2018-10-09 15:35:15Z

2

To use async/await, you need to use a HTTP library that returns promises instead of using callbacks.

It's well worth using a library that can do this. A good option for a different library is node-fetch, which effectively implements the "Fetch API" as implemented by browsers.

You can then simply await fetch(url).

async/await works so well and is becoming so common-place, I can highly recommend switching to frameworks and libraries that treat it as a first-class citizen instead of an afterthought. This includes Express, which is also a callback-based library, instead of a Promise-based one. But that's slightly off-topic.

answered Oct 9, 2018 at 15:35

Evert

101k18 gold badges136 silver badges221 bronze badges

Comments

Aymeric Bouzy aybbyk · Accepted Answer · 2018-10-09 16:03:07Z

As others have said, you should probably use Promise for your asynchronous code, and the async / await syntax is very legible.

However, you seem to prefer using callbacks. In that case, you could write your code this way :

res.setHeader('Content-Type', 'application/json');
const sub = req.query.days_subtract;
const enddate = req.query.end_date; 

var array = [];
var resultsCount = 0;

for (var i = 0; i < sub; i++) {
  request("https://api.nasa.gov/planetary/apod?date=" + subtractDate(enddate, i) + "&api_key=DEMO_KEY", function(error, response, body) {
    if (error) { res.status(500) }
    array[i] = body
    // console.log(body);
    resultCount++;
    if (resultCount === sub) {
      res.send(array);
    }
  });
}

Basically, the idea is to call the res.send method only once all requests have returned their result.

Collectives™ on Stack Overflow

How to wait for for loop with async requests to finish in node.js?

4 Answers 4

Comments

5 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

5 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related