Node.js async module waterfall - dynamically load and execute functions

Question

I am attempting to dynamically load a series of anonymous functions and execute them, passing the results of the previous to the next.

Here is an example function:

module.exports = function (data) {
    // do something with data
    return (data);
}

When the functions are loaded (they all sit in separate files) they are returned as an object:

{ bar: [Function], foo: [Function] }

I would like to execute these functions using async.waterfall. This takes an array of functions, not an object of functions, so I convert as follows:

var arr =[];
        for( var i in self.plugins ) {
            if (self.plugins.hasOwnProperty(i)){
                arr.push(self.plugins[i]);
            }
        }

This gives:

[ [Function], [Function] ]

How can I now execute each function using each async.waterfall passing the result of the previous function to the next?

SOLUTION

Thanks to the comments from @piergiaj I am now using next() in the functions. The final step was to make sure that a predefined function was put first in the array that could pass the incoming data:

var arr =[];

    arr.push(function (next) {
        next(null, incomingData);
    });

    for( var i in self.plugins ) {
        if (self.plugins.hasOwnProperty(i)){
            arr.push(self.plugins[i]);
        }
    }

    async.waterfall(arr,done);

Are all your functions synchronous? If they are, you don't need async.waterfall. — Mike S
– Mike S, Commented Aug 2, 2014 at 0:30

piergiaj · Accepted Answer · 2014-08-02 00:49:52Z

1

If you want them to pass data on to the next one using async.waterfall, instead of returning at the end of each function, you need to call the next() method. Additionally, you need to have next be the last parameter to each function. Example:

module.exports function(data, next){
    next(null, data);
}

The first parameter to next must be null because async.waterfall treats it as an error (if you do encounter an error in one of the methods, pass it there and async.waterfall will stop execute and finish passing the error to the final method).

You can then convert it like you already to (object to array), then call it like so:

async.waterfall(arrayOfFunctions, function (err, result) {
     // err is the error pass from the methods (null if no error)
     // result is the final value passed from the last method run    
  });

answered Aug 2, 2014 at 0:49

piergiaj

6293 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ben Over a year ago

Many thanks for the comment, thats makes good sense. Im getting an error thrown on the exported function on the next(null, data); line - TypeError: undefined is not a function. Im guessing I need to pass in the inbound data to the first function. Any ideas how I should do that?

Kelz Over a year ago

No need to add module.exports to the function unless you're making it public to other files. Just declare the function normally. If you're doing module.exports you'll need to add an = between that and the function as well.

Collectives™ on Stack Overflow

Node.js async module waterfall - dynamically load and execute functions

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related