2

For a Node JS module I'm writing I would like to use the async function Stats.isFile() as the callback function of the Array.filter() function. Below I have a working example of what I want to achieve, but using snychronous equivalents. I can't get my head around how to wrap the async function so that i becomes usable inside the Array.filter() function.

const fs = require('fs')

exports.randomFile = function(dir = '.') {
    fs.readdir(dir, (err, fileSet) => {
        const files = fileSet.filter(isFile)
        const rnd = Math.floor(Math.random() * files.length);
        return files[rnd])
    })
}

function isFile(item) {
    return (fs.statSync(item)).isFile()
}
Improve this question
0

3 Answers 3

Reset to default
1

You can't use an async callback with .filter(). .filter() expects a synchronous result and there is no way to get a synchronous result out of an asynchronous operation. So, if you're going to use the asynchronous fs.stat(), then you will have to make the whole operation async.

Here's one way to do that. Note even randomFile() needs to communicate back it's result asynchronously. Here we use a callback for that.

const path = require('path');
const fs = require('fs');

exports.randomFile = function(dir, callback)  {
    fs.readdir(dir, (err, files) => {
        if (err) return callback(err);

        function checkRandom() {
            if (!files.length) {
                // callback with an empty string to indicate there are no files
                return callback(null, "");
            }
            const randomIndex = Math.floor(Math.random() * files.length);
            const file = files[randomIndex];
            fs.stat(path.join(dir, file), (err, stats) => {
                if (err) return callback(err);
                if (stats.isFile()) {
                    return callback(null, file);
                }
                // remove this file from the array
                files.splice(randomIndex, 1);
                // try another random one
                checkRandom();
            });
        }

        checkRandom();
    });
}

And, here's how you would use that asynchronous interface form another module.

// usage from another module:
var rf = require('./randomFile');
fs.randomFile('/temp/myapp', function(err, filename) {
   if (err) {
       console.log(err);
   } else if (!filename) {
       console.log("no files in /temp/myapp");
   } else {
       console.log("random filename is " + filename);
   }
});
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

thanks for this script which seems like it's 'production-ready'! :-) May have to add a path.join(dir, file) so that fs.stat works.
@jfix - You can't usefully use an ES6 default value on the first parameter when the second parameter (the callback) is required. You could declare it that way, but the only way to trigger using the default value would be to pass undefined as the first argument which hardly makes sense. So, I don't think an ES6 default parameter for dir makes sense in this case. It could be implemented in the old fashioned way where the code checks to see if one or two arguments were passed and assigns a default dir if only one argument was passed. But, I don't know how to get ES6 to do that for you.
1

if you go async you have to go async from the entry point on, so randomFile needs some way to "return" an async value (typically via callbacks, promises, or as a stream).

i do not know how your file structure looks like but instead of checking all entries for being a file i would select a random entry, check if it is a file, and if not try again.

this could look like this

const fs = require('fs');
const path = require('path');

exports.randomFile = (dir = '.', cb) => {
    fs.readdir(dir, (err, files) => {
        if (err) { return cb(err); }
        pickRandom(files.map((f) => path.join(dir, f)), cb);
    });
}

function pickRandom (files, cb) {
    const rnd = Math.floor(Math.random() * files.length);
    const file = files[rnd];
    fs.stat(file, (err, stats) => {
        if (err) {
            return cb(err);
        }
        if (stats.isFile()) {
            return cb(null, file);
        } else {
            return pickRandom(files, cb);
        }
    })
}
Improve this answer

6 Comments

Probably should remove the non-file from your files array before calling pickRandom() again so you more clearly zero in on finding an actual file. You also have to be prepared for the case where there are no files (only directory entries) in which can you would loop infinitely.
@squiddle This is awesome, thank you! While it solves the problem at hand nicely, how would one go about using an async function for Array.filter(), just out of curiosity. :-) Because in a next step, I intend to add filtering by file extensions, for example.
@jfriend00 that is all right, in a production implementation i would put an attempt limit counter for pickRandom to abort after X tries as well. files could be empty and it would also fail ungraceful.
@jfix since Array.filter is a synchronous api you just cannot do it. you could first gather the stats for all files, collect them in an array and then use Array.filter. or you Array.map over files, collect all stats as promises, and use an iterator which understands an array of promises like Promise.all (promisejs.org/patterns) and then use Array.filter on the result. but i see no point in trying so hard to use Array.filter
ok @squiddle thanks for confirming that it's just not possible to use Array.filter() in this context in the way I intended. It looked like the perfect tool for the job, but it's reassuring that you say it's not possible. I feel a bit better about myself now! ;-)
|
0

Using async/await, you can write your own async filter function:

const asyncFilter = async (list, action) => {
    const toFilter = [];
    await Promise.all(list.map(async (item, index) => {
        const result = await action(item);
        if (!result) toFilter.push(index);
    }));
    return list.filter((_, index) => !toFilter.includes(index));
}

In your case, you can write isFile asynchronously (returning a Promise) and the line:

const files = fileSet.filter(isFile)

Becomes:

const files = await asyncFilter(fileSet, isFile);

Don't forget to add the async keyword before the callback of fs.readdir

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.