Asynchronous for loop: how do I make the for loop wait for function to finish?

Question

I'm racking my brain trying to figure out how to sequence this / place callbacks to get what I need.

I have a loop that checks for the existence of files. I'd like it to callback when it's done, but the for loop and the callback finish before the "fs.open" finishes... typical asynchronous problem.

I am using node v0.10.29, express v3.14.0, and am looking at using the "async" library, but again, just can't figure out the logic that I need...

Here is what I have:

Input

function checkForAllFiles(callback)
{
    var requiredFiles = [];
    requiredFiles[requiredFiles.length] = "../Client/database/one.js";
    requiredFiles[requiredFiles.length] = "../Client/database/two.dat";
    requiredFiles[requiredFiles.length] = "../Client/database/three.xml";
    requiredFiles[requiredFiles.length] = "../Client/database/four.dat";
    requiredFiles[requiredFiles.length] = "../Client/database/five.dat";
    requiredFiles[requiredFiles.length] = "../Client/database/six.xml";

    var missingFiles = [];
    for(var r=0; r<requiredFiles.length; r++)
    {
        fs.open(requiredFiles[r], 'r', function(err, fd){
            if(err)
            {
                missingFiles[missingFiles.length] = err.path;
                console.log("found missing file = ", err.path);
            }
        });
        console.log("r = ", r);
    }
    console.log("sending callback: ", missingFiles);
    callback(missingFiles);
}

Output

0
1
2
3
4
5
sending callback: []
found missing file: ../Client/database/three.xml

Desired Output

0
1
found missing file: ../Client/database/three.xml
2
3
4
5
sending callback: ["../Client/database/three.xml"]

Just for fun, you can use the Array.push() method for adding items to to your requriedFiles array: requiredFiles.push("../Client/database/one.js"); for example — SamT
– SamT, Commented Aug 1, 2014 at 17:18

Mike S · Accepted Answer · 2014-08-01 17:35:37Z

3

I would use the reject method in the async module (which I see you've already found). What it will do is return an array in its callback that contains any elements that don't match a specified check function. For the check function, I'd recommend just using fs.exists instead of watching for an error on fs.open.

Using those functions you can actually reduce the whole check to one line. Something like this:

function checkForAllFiles(callback)
{
    var requiredFiles = [];
    requiredFiles[requiredFiles.length] = "../Client/database/one.js";
    requiredFiles[requiredFiles.length] = "../Client/database/two.dat";
    requiredFiles[requiredFiles.length] = "../Client/database/three.xml";
    requiredFiles[requiredFiles.length] = "../Client/database/four.dat";
    requiredFiles[requiredFiles.length] = "../Client/database/five.dat";
    requiredFiles[requiredFiles.length] = "../Client/database/six.xml";

    async.reject(requiredFiles, fs.exists, callback);
}

callback will get called with an array that contains just the files that don't exist.

answered Aug 1, 2014 at 17:35

Mike S

42.3k11 gold badges91 silver badges85 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Chris Tavares Over a year ago

You don't need all those array set lines, you can just initialize it directly: var requiredFiles = [ '../Client/databse/one.js', '../Client/databse/two.dat', ...etc ]

Mike S Over a year ago

Very true. I was just copying the method in the question to try to make it a little clearer where things should go.

piergiaj · Accepted Answer · 2014-08-01 17:21:50Z

1

Use the async library and the eachSeries method. Example:

async.eachSeries(array,
     function(element, next) {
          // do something with element
          next();
     }
);

It will sequentially go through the array and process each element. Calling next goes to the next element. Series makes sure it does it in the order of the array, otherwise the order of going through the array is not guaranteed. If you have other async functions called within it, just pass the next function around and call it when done with all the needed functions and the next array element will be processed.

Maybe something like this:

var missingFiles = []
async.eachSeries(requiredFiles, function(file, nextFile){
 fs.open(file, 'r', function(err, fd){
        if(err)
        {
            missingFiles[missingFiles.length] = err.path;
            console.log("found missing file = ", err.path);
        }
       nextFile();
    });
    console.log("r = ", file);
});
console.log("sending callback: ", missingFiles);
callback(missingFiles);

edited Aug 1, 2014 at 17:21

answered Aug 1, 2014 at 17:15

piergiaj

6293 silver badges9 bronze badges

2 Comments

go-oleg Over a year ago

If you don't need the checks to happen in order, you can use async.each which should run faster. Also, you can use fs.exists instead of fs.open and if all your required files are in the same directory, you can use one fs.readdir call to get the list of files in that directory and do an intersection with your requiredFiles array.

Katie Over a year ago

This was definitely worth a shot but for some reason, the callback still kept getting executed after the fs :( even when I used the eachSeries(arr, iterator, callback) syntax :(

Collectives™ on Stack Overflow

Asynchronous for loop: how do I make the for loop wait for function to finish?

Input

Output

Desired Output

2 Answers 2

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Input

Output

Desired Output

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related