NodeJs : Multiple loop of HTTP requests execute simultaneously

Question

As per the above question,it was answered how to perform a loop of http requests with an array of urls which works fine.But what I am trying to achieve is to perform another loop of http requests which should be done only after the completion of the first loop (i.e) it should wait for the first loop of http requests to complete.

// Import http
var http = require('http');

// First URLs array
var urls_first = ["http://www.google.com", "http://www.example.com"];

// Second URLs array
var urls_second = ["http://www.yahoo.com", "http://www.fb.com"];

var responses = [];
var completed_requests = 0;

function performHTTP(array) {
for (i in urls_first) {
        http.get(urls[i], function(res) {
            responses.push(res);
            completed_requests++;
            if (completed_requests == urls.length) {
                // All download done, process responses array
                console.log(responses);
            }
        });
    }
 }

In the above snippet ,i have added another array of urls.I wrapped the for inside a function to change the array each time called.Since i have to wait for the first loop to complete, i tried async/await like below.

async function callMethod() { 
    await new Promise (resolve =>performHTTP(urls_first)))   // Executing function with first array
    await new Promise (resolve =>performHTTP(urls_second)))  // Executing function with second array
}

But in this case both the function calls get executed simultaneously(i.e)it does not wait for the first array execution to complete .Both execution happens simultaneously,which i need to happen only after the completion of one.

You can use promise, callback, async await or eachSeries, async library. I prefer : caolan.github.io/async/v3/docs.html#eachSeries — Yogesh Patel
– Yogesh Patel, Commented Sep 18, 2019 at 6:28
what i believe is await will wait for any previous await to complete.Since i want several loops ,to callback would be complex.Instead i even tried using .then which didnt work either like ` var promise1 = await new Promise (resolve =>performHTTP(urls_first)) promise1.then(await new Promise (resolve =>performHTTP(urls_second)))` — Sathya Narayanan GVK
– Sathya Narayanan GVK, Commented Sep 18, 2019 at 6:34

qphi · Accepted Answer · 2019-09-18 06:41:02Z

2

You need to make your request inside a Promise :

function request(url) {
    return new Promise((resolve, reject) => {
       http.get(url, function(res) {
        // ... your code here ... //
        // when your work is done, call resolve to make your promise done
         resolve()
       });
    }
}

And then resolve all your requests

// Batch all your firts request in parallel and wainting for all
await Promise.all(urls_first.map(url => request(url)));
// Do the same this second url
await Promise.all(urls_second.map(url => request(url)));

Note, this code is not tested and may contains some mistake, but the main principle is here.

More information about Promise : https://developer.mozilla.org/fr/docs/Web/JavaScript/Reference/Objets_globaux/Promise

answered Sep 18, 2019 at 6:41

qphi

1214 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sathya Narayanan GVK Over a year ago

tried this ,only the first await works ,it does not goto second await

Lvcas · Accepted Answer · 2019-09-18 06:22:29Z

0

Check out how to use .then() to call second performHttp right after the first one is completed.

answered Sep 18, 2019 at 6:22

Lvcas

416 bronze badges

1 Comment

Sathya Narayanan GVK Over a year ago

calling the function inside .then does not work for me.

Yogesh Patel · Accepted Answer · 2019-09-18 06:50:07Z

0

You can call services using eachSeries.

https://www.npmjs.com/package/async-series

series([
  function(done) {
    console.log('First API Call here')
    done() // if you will pass err in callback it will be caught in error section.
  },
  function(done) {
    console.log('second API call here')
    done()
  },
  function(done) {
    // handle success here
  }
], function(err) {
  console.log(err.message) // "another thing"
})

answered Sep 18, 2019 at 6:50

Yogesh Patel

3142 silver badges11 bronze badges

Collectives™ on Stack Overflow

NodeJs : Multiple loop of HTTP requests execute simultaneously

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related