Check if file is completely written with node.js

Question

I need a little bit help how I can handle following task in JavaScript: I have an app, that use Jimp for image processing and node-sprite-generator. That all runs in node.js context. I load a few images to Jimp, make something with the image then write it back to my file system with the nodejs filemodule. Then I would take the new created images and pasted it to node-sprite-generator. The problem is, that not all images are created/written at this time. While the code for create the spritesheet runs immediately after Jimp returns, I think Jimp processing all of the images and return a promise. The result is, that the code for creating the spritesheet get executed but the stack is not done.

I tried to test if the file is written, with fs.stat() and the propertie mtime like

 if (stat.mtime.getTime() === prev.mtime.getTime())

But then it can happens, that an error occurred, when the file is not created at this time. Also: I need a way to check if the image is written completely with handling, when the path to the image is current not available.

function resize(img) {
    Jimp.read(img.path).then(function (err, file) {
        if (err) throw err;
        file.resize(200, 200)
            .quality(70)
            .write(newPath); //newPath for simplicity
    });
}


function rec(imgObjArray) {
    if(_.isEmpty(imgObjArray)) return;  
    resize(imgObjArray.pop());
//correct mistake in function call from fn() to rec()
    rec(imgObjArray);
}

rec(imgObjArray); //imgObjArray === [img,img,img....]

//nsg() does not work because not all images are written at this time
     nsg({
            src: [
                'out/images/desktop/*.jpg'
            ],
            spritePath: 'out/images/desktop/sprite.jpg',,
            compositor: 'jimp'
        }, function (err) {
            console.log('Sprite generated!');
        })

I think first I must check if the image exist at a given path and then check if written is finished. But when I make a fn with fs.access(path[, mode], callback) and the file is not created at this time, I get an error.

If you've got a promise you need to use that to execute the next action when the first one is done. It would be helpful to see your code, or at least a portion of it where you're transitioning tasks, to provide any useful guidance on how to fix it. — Jason
– Jason, Commented Oct 12, 2016 at 16:28
I thought so too, so I make a then() function to the jimp queue which contains the fn-call to execute the sprite-sheet code. But I get the same issue. — Zantinger
– Zantinger, Commented Oct 12, 2016 at 16:32
You need to show us your code. Questions about code MUST include the relevant code in your question. We could probably help in minutes if you show us the code. — jfriend00
– jfriend00, Commented Oct 12, 2016 at 16:36

Jason · Accepted Answer · 2016-10-12 20:08:32Z

You've got a mix of synchronous and asynchronous code in here. I'll try to describe what's happening in comments:

First, your function definitions - you're firing off asynchronous actions without properly handling their completions

// I've re-spaced the code slightly and removed your comments so mine stand out
function resize(img) {
    Jimp.read(img.path).then(function (err, file) {
        // this code only executes after the file is done reading, but this
        // is an asynchronous action - it doesn't hold up execution
        if (err) throw err;
        file.resize(200, 200).quality(70).write(newPath);
        // .write() is presumably *also* an asynchronous action - if you want
        // something to happen only *after* it's been written, it needs to be in
        // a callback or promise on the write method
    });

    // I added this explicitly - after you *start* your Jimp.read, you *immediately*
    // return from this function, *before* the read is completed.  If you want
    // something to happen only *after* your read and write, you either need to
    // return the promise so you can act on it, or put the further actions in a callback
    return undefined;
}

function rec(imgObjArray) {
    if(_.isEmpty(imgObjArray)) return; 
    // resize() runs and returns *before* the file is read, resized, and written
    resize(imgObjArray.pop());
    // I don't know what fn() is, it's not defined here - presumably it's not important
    fn(imgObjArray);
}

... then, your procedural calls:

// this fires off and completes immediately, having initiated the asynchronous methods
rec(imgObjArray);

// you call this on the assumption that all of your code above has completed, but since
// it's asynchronous, that's not true, you get here with *none* of your images completed
nsg({
    src: [
        'out/images/desktop/*.jpg'
    ],
    spritePath: 'out/images/desktop/sprite.jpg',
    compositor: 'jimp'
}, function (err) {
    console.log('Sprite generated!');
});

You have two options:

If file.write() is a synchronous call, you can just return the promise and act on it:

function resize(img) {
    // by *returning* this call, we're actually returning the promise, we can act on
    // in the future
    return Jimp.read(img.path).then(function (err, file) {
        if (err) throw err;
        file.resize(200, 200).quality(70).write(newPath);
    });
}

function rec(imgObjArray) {
    if(_.isEmpty(imgObjArray)) return; 
    // the result of resize is now a promise
    return resize(imgObjArray.pop()).then(function(err) {;
        // again, assuming `fn()` is synchronous...
        fn(imgObjArray);
    });
}

// now the result of *this* call is a promise, which you can use to control
// the timing of your next call
rec(imgObjArray).then(function(err) {
    // now this will only run after all of the previous calls have executed
    nsg({
        src: [
            'out/images/desktop/*.jpg'
        ],
        spritePath: 'out/images/desktop/sprite.jpg',
        compositor: 'jimp'
    }, function (err) {
        console.log('Sprite generated!');
    });
});

... apologies if the promise syntax is incorrect, I haven't used node actively since they became ubiquitous.

There is likely a way to use promises in the same way even if your sub calls are asynchronous, I just don't have that ready.

Otherwise, you can pass a callback into your functions:

function resize(img, cb) {
    // ... you get the idea...
        file.resize(200, 300).quality(70).write(newPath, cb);
}

function rec(imgObjArray, cb) {
    // ... you get the idea...
    resize(imgObjArray.pop(), cb);
}

rec(imgObjArray, function(err, response) {
    nsg({
        src: [
            'out/images/desktop/*.jpg'
        ],
        spritePath: 'out/images/desktop/sprite.jpg',
        compositor: 'jimp'
    }, function (err) {
        console.log('Sprite generated!');
    });
});

Hope this helps!

IT WORKS! Thank you so much @Jason. PS: the fn() function is actual a rec() call. This call the function itself till the array imgObj is empty. But the most important is, that you help me out with a working solution. Thanks.
OK, so another piece of advice - recursive calls are less efficient than just looping through an array, so your rec() function should just do a simple loop rather than calling itself and popping off the elements. With your scoping as it's set up, the end result would be the same. Either way, I'm glad this helped!

Collectives™ on Stack Overflow

Check if file is completely written with node.js

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related