Populating async array with a function called right before.

Question

var request = require('request'),
requests = [],
values = [],

request("url1", function());

function() {
.....
  for (x in list){
   requests.push(requestFunction(x));
  }
}

requestFunction(x){
  request("url2", function (e,r,b) {
    ....
    return function(callback) {
    values[i] = b
    }
  });
}

async.parallel(requests, function (allResults) {
    // values array is ready at this point
    // the data should also be available in the allResults array
    console.log(values);
});

I new to node. Issue is that the request needs to be called to populate the requests callback array. But the issue is the async.parallel will run before the requests array is full and need run all the callbacks. Where do I move this async so it runs after the requests array is full?

royhowie · Accepted Answer · 2014-07-05 04:50:28Z

2

Asynchronous programming is all about chaining blocks. This allows node to efficiently run its event queue, while ensuring that your steps are done in order. For example, here's a query from a web app I wrote:

app.get("/admin", isLoggedIn, isVerified, isAdmin, function (req, res) {
    User.count({}, function (err, users) {
        if (err) throw err;
        User.count({"verified.isVerified" : true}, function (err2, verifiedUsers) {
            if (err2) throw err2;
            Course.count({}, function (err3, courses) {
                // and this continues on and on — the admin page
                // has a lot of information on all the documents in the database
            })
        })
    })  
})

Notice how I chained function calls inside of one another. Course.count({}, ...) could only be called once User.count({"verified.isVerified" : true}, ...) was called. This means the i/o is never blocked and the /admin page is never rendered without the required information.

You didn't really give enough information regarding your problem (so there might be a better way to fix it), but I think you could, for now, do this:

var request = require('request'),
    requests = [],
    values = [],
    length;     // a counter to store the number of times to run the loop

request("url1", function() {

    length = Object.keys(list).length;
    // referring to the list below;
    // make sure list is available at this level of scope

    for (var x in list){
        requests.push(requestFunction(x));
        length--;
        if (length == 0) {
            async.parallel(requests, function (allResults) {
                console.log(values);    // prints the values array
            });
        }
    }
}

function requestFunction(x) {
    request("url2", function (e,r,b) {
        values[i] = b;
        return b;
    }
}

I am assuming that requestFunction() takes a while to load, which is why async.parallel is running before the for (var x in list) loop finishes. To force async.parallel to run after the loop finishes, you'll need a counter.

var length = Object.keys(list).length;

This returns the number of keys in the list associative array (aka object). Now, every time you run through the for loop, you decrement length. When length == 0, you then run your async.parallel process.

edit: You could also write the requests.push() part as:

requests.push(
    (function () {
        request("url2", function (e,r,b) {
            values[i] = b;
            return b;
        }
    })()
);

I think it's redundant to store b in both values and requests, but I have kept it as you had it.

edited Jul 5, 2014 at 4:50

answered Jul 5, 2014 at 4:06

royhowie

11.2k14 gold badges54 silver badges67 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

CWon Over a year ago

Thats very helpful. I'm having an issue now were the requests array is undefined. Am I returning the function incorrectly in requestFunction?

royhowie Over a year ago

@CWon Is this for a specific library? Regardless, requestFunction doesn't really make sense. You return a function: return function(callback) { … }, but that lambda doesn't return anything.

royhowie Over a year ago

@CWon I changed the requestFunction in my original answer and have added another method for pushing to the requests array.

Collectives™ on Stack Overflow

Populating async array with a function called right before.

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related