Node.js: problem with variable scope in callback

Question

var result = { controllers: [], views: [], models: [] };
var dirs = ['controllers', 'views', 'models'];

dirs.forEach(function(dirname) {
    fs.readdir('./' + dirname, function(err, res) {
        if (err) throw err;
        result[dirname] = res;
        // #2
    });
});

// #1

In this snippet of code, having console.log(result); running at #1 (see above), empty controller, views, and models arrays just as initialized will be logged. However, I need the loop to fill the arrays with corresponding file names read via fs.

console.log(result); at #2 will log the result object filled with the desired values after the third iteration.

I believe this has something to do with the asynchronous nature of Node.js / JavaScript callbacks. Please forgive me if I'm not understanding how JavaScript variable scopes and async methods work, I'm all new to this.

thejh · Accepted Answer · 2011-08-06 21:56:23Z

2

Do it this way:

var result = { controllers: [], views: [], models: [] };
var dirs = ['controllers', 'views', 'models'];
var pending = 0;

dirs.forEach(function(dirname) {
    pending++;
    fs.readdir('./' + dirname, function(err, res) {
        pending--;
        if (err) throw err;
        result[dirname] = res;
        if (pending===0) goOn();
    });
});
function goOn() {
    // #1
}

answered Aug 6, 2011 at 21:56

thejh

45.7k18 gold badges100 silver badges109 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Dan Over a year ago

Looks good so far. I'm not fully convinced though since this solution incorporates nesting – still, this is a working and reasonable solution. Cheers!

Praveena Over a year ago

@thejh what is the guarantee that pending has right value inside callback?

Community · Accepted Answer · 2017-05-23 12:26:53Z

2

I believe this has something to do with the asynchronous nature of Node.js / JavaScript callbacks.

Yes, this is probably the reason why, when you try to output the content of your result variable at #1, it's empty. At the time of running at #1 data are simply not yet fetched because "fetching" action happens druing the execution of readdir's callback at #2. I would recommend to look at some of the resources about asynchronous paradigms stated in this answer in order to get the bigger/better picture of how callbacks and asynchronous programming works.

edited May 23, 2017 at 12:26

CommunityBot

11 silver badge

answered Aug 6, 2011 at 22:00

yojimbo87

68.8k26 gold badges127 silver badges133 bronze badges

2 Comments

bmeck Over a year ago

Be careful. Not all callbacks will always execute asynchronously.

Dan Over a year ago

Thanks, I read all of the "Control Flow in Node.js" series. Nice resource!

bmeck · Accepted Answer · 2011-08-06 21:51:48Z

0

For people confused about wording, here is the output of an example with a '1' file in each of the directories:

~/Documents/$ node test.js 
{ controllers: [], views: [], models: [] } 1
{ controllers: [], views: [ '1' ], models: [] } 2
{ controllers: [ '1' ], views: [ '1' ], models: [] } 2
{ controllers: [ '1' ],
  views: [ '1' ],
  models: [ '1' ] } 2

This is because going into the file system requires async operations to occur. So since Node does not block the values afterwards will be the same as before the async operations are done.

answered Aug 6, 2011 at 21:51

bmeck

2462 silver badges6 bronze badges

1 Comment

Dan Over a year ago

This is totally reasonable, but using fs.readdirSync() doesn't work either.

shelman · Accepted Answer · 2011-08-06 22:05:05Z

0

It does have to do with callbacks. In

fs.readdir('./' + dirname, function(err, res) {
    if (err) throw err;
    result[dirname] = res;
    // #2
});

the function passed in is a callback, that executes when the directory is fully read. Since it is asynchronous, fs.readdir() returns before the function actually executes.

So basically, after your forEach finishes, you have 3 functions waiting to be executed as callbacks (one for each directory). The code does not wait for the callbacks to happen before continuing to execute, so if #1 is reached before the directories are read, it will log your result object before the callbacks execute and modify it appropriately.

You could use fs.readdirSync instead, but ONLY IF it would not be bad for your application if this code blocked briefly / there is no danger of this code blocking indefinitely and stalling your program. If you need it to remain asynchronous, look at thejh's answer.

edited Aug 6, 2011 at 22:05

answered Aug 6, 2011 at 21:58

shelman

2,66916 silver badges17 bronze badges

1 Comment

thejh Over a year ago

Not +1ing because you really should warn about fs.readdirSync being a bad idea in many cases.

Soren · Accepted Answer · 2011-08-06 21:44:38Z

-1

I beleive it is to do with that the value if dirname changes after the return of the first function. To solve, simply copy the dirname into the scope for the second function call;

dirs.forEach(function(dirn) {
    var dirname = dirn;  // make a copy here
    fs.readdir('./' + dirname, function(err, res) {
        if (err) throw err;
        result[dirname] = res;
        // #2
    });
});

answered Aug 6, 2011 at 21:44

Soren

14.8k4 gold badges43 silver badges69 bronze badges

Collectives™ on Stack Overflow

Node.js: problem with variable scope in callback

5 Answers 5

2 Comments

2 Comments

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

2 Comments

2 Comments

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related