331

I have a simplified function that looks like this:

function(query) {
  myApi.exec('SomeCommand', function(response) {
    return response;
  });
}

Basically i want it to call myApi.exec, and return the response that is given in the callback lambda. However, the above code doesn't work and simply returns immediately.

Just for a very hackish attempt, i tried the below which didn't work, but at least you get the idea what i'm trying to achieve:

function(query) {
  var r;
  myApi.exec('SomeCommand', function(response) {
    r = response;
  });
  while (!r) {}
  return r;
}

Basically, what's a good 'node.js/event driven' way of going about this? I want my function to wait until the callback gets called, then return the value that was passed to it.

3
  • 4
    Or am i going about it entirely the wrong way here, and should i be calling another callback, rather than returning a response? Commented Feb 15, 2011 at 22:33
  • This is in my opinion the best SO explanation why the busy loop doesn't work. Commented May 20, 2017 at 16:11
  • Don't try to wait. Just call next function(callback-dependent) within at the end of callback itself Commented Aug 5, 2019 at 7:08

10 Answers 10

306

The "good node.js /event driven" way of doing this is to not wait.

Like almost everything else when working with event driven systems like node, your function should accept a callback parameter that will be invoked when then computation is complete. The caller should not wait for the value to be "returned" in the normal sense, but rather send the routine that will handle the resulting value:

function(query, callback) {
  myApi.exec('SomeCommand', function(response) {
    // other stuff here...
    // bla bla..
    callback(response); // this will "return" your value to the original caller
  });
}

So you dont use it like this:

var returnValue = myFunction(query);

But like this:

myFunction(query, function(returnValue) {
  // use the return value here instead of like a regular (non-evented) return value
});
Sign up to request clarification or add additional context in comments.

13 Comments

Ok great. What about if myApi.exec never called the callback? How would i make it so that the callback gets called after say 10 seconds with an error value saying it timed our or something?
Or better yet (added a check so the callback cant be invoked twice): jsfiddle.net/LdaFw/1
It is clear non-blocking is the standard in node/js, however there are certainly times when blocking is desired (e.g. blocking on stdin). Even node has "blocking" methods (see all the fs sync* methods). As such, I think this is still a valid question. Is there a nice way to achieve blocking in node aside from busy waiting?
A late answer to the comment by @nategood: I can think of a couple of ways; too much to explain in this comment, but google them. Remember that Node is not made to be blocked, so these are not perfect. Think of them as suggestions. Anyway, here goes: (1) Use C to implement your function and publish it to NPM in order to use it. That's what the sync methods do. (2) Use fibers, github.com/laverdet/node-fibers, (3) Use promises, for example the Q-library, (4) Use a thin layer on top of javascript, that looks blocking, but compiles to async, like maxtaco.github.com/coffee-script
It is so frustrating when people answer a question with "you shouldn't do that." If one wants to be helpful and answer a question, that is a stand up thing to do. But telling me unequivocally that I shouldn't do something is just unfriendly. There are a million different reasons why someone would want to call a routine synchronously or asynchronously. This was a question about how to do it. If you provide helpful advice about the nature of the api while providing the answer, that is helpful, but if you don't provide an answer, why bother replying. (I guess I should really head my own advice.)
|
135

One way to achieve this is to wrap the API call into a promise and then use await to wait for the result.

// Let's say this is the API function with two callbacks,
// one for success and the other for error.
function apiFunction(query, successCallback, errorCallback) {
    if (query == "bad query") {
        errorCallback("problem with the query");
    }
    successCallback("Your query was <" + query + ">");
}

// Next function wraps the above API call into a Promise
// and handles the callbacks with resolve and reject.
function apiFunctionWrapper(query) {
    return new Promise((resolve, reject) => {
        apiFunction(query,(successResponse) => {
            resolve(successResponse);
        }, (errorResponse) => {
            reject(errorResponse);
        });
    });
}

// Now you can use await to get the result from the wrapped api function
// and you can use standard try-catch to handle the errors.
async function businessLogic() {
    try {
        const result = await apiFunctionWrapper("query all users");
        console.log(result);
        
        // the next line will fail
        const result2 = await apiFunctionWrapper("bad query");
    } catch(error) {
        console.error("ERROR:" + error);
    }
}

// Call the main function.
businessLogic();

Output:

Your query was <query all users>
ERROR:problem with the query

4 Comments

This is a very well done example of wrapping a function with a callback so you can use it with async/await I dont often need this, so have trouble remembering how to handle this situation, I'm copying this for my personal notes/references.
Very well written example. easy to understand for beginners like me. happy to recover from async/await callback hell
Good job. This is exactly what I needed, as I got an API function call that uses callbacks and I had no idea how to "await" its result.
This is the exact use case when one wants to wait till a callback is called: create an async API around a callback mechanism. Great spot, and great description.
25

check this: https://github.com/luciotato/waitfor-ES6

your code with wait.for: (requires generators, --harmony flag)

function* (query) {
  var r = yield wait.for( myApi.exec, 'SomeCommand');
  return r;
}

Comments

12

If you don't want to use call back then you can Use "Q" module.

For example:

function getdb() {
    var deferred = Q.defer();
    MongoClient.connect(databaseUrl, function(err, db) {
        if (err) {
            console.log("Problem connecting database");
            deferred.reject(new Error(err));
        } else {
            var collection = db.collection("url");
            deferred.resolve(collection);
        }
    });
    return deferred.promise;
}


getdb().then(function(collection) {
   // This function will be called afte getdb() will be executed. 

}).fail(function(err){
    // If Error accrued. 

});

For more information refer this: https://github.com/kriskowal/q

Comments

9

It's 2020 and chances are the API has already a promise-based version that works with await. However, some interfaces, especially event emitters will require this workaround:

// doesn't wait
let value;
someEventEmitter.once((e) => { value = e.value; });
// waits
let value = await new Promise((resolve) => {
  someEventEmitter.once('event', (e) => { resolve(e.value); });
});

In this particular case it would be:

let response = await new Promise((resolve) => {
  myAPI.exec('SomeCommand', (response) => { resolve(response); });
});

Await has been in new Node.js releases for the past 3 years (since v7.6).

1 Comment

Exactly what I needed - for some reason it's easy to get lost in thinking about how to do this with callbacks when something as simple as wrapping an event emitter in a Promise is all that's required
5

Note: This answer should probably not be used in production code. It's a hack and you should know about the implications.

There is the uvrun module (updated for newer Nodejs versions here) where you can execute a single loop round of the libuv main event loop (which is the Nodejs main loop).

Your code would look like this:

function(query) {
  var r;
  myApi.exec('SomeCommand', function(response) {
    r = response;
  });
  var uvrun = require("uvrun");
  while (!r)
    uvrun.runOnce();
  return r;
}

(You might alternative use uvrun.runNoWait(). That could avoid some problems with blocking, but takes 100% CPU.)

Note that this approach kind of invalidates the whole purpose of Nodejs, i.e. to have everything async and non-blocking. Also, it could increase your callstack depth a lot, so you might end up with stack overflows. If you run such function recursively, you definitely will run into troubles.

See the other answers about how to redesign your code to do it "right".

This solution here is probably only useful when you do testing and esp. want to have synced and serial code.

Comments

5

Since node 4.8.0 you are able to use the feature of ES6 called generator. You may follow this article for deeper concepts. But basically you can use generators and promises to get this job done. I'm using bluebird to promisify and manage the generator.

Your code should be fine like the example below.

const Promise = require('bluebird');

function* getResponse(query) {
  const r = yield new Promise(resolve => myApi.exec('SomeCommand', resolve);
  return r;
}

Promise.coroutine(getResponse)()
  .then(response => console.log(response));

Comments

1

supposing you have a function:

var fetchPage(page, callback) {
   ....
   request(uri, function (error, response, body) {
        ....
        if (something_good) {
          callback(true, page+1);
        } else {
          callback(false);
        }
        .....
   });
};

you can make use of callbacks like this:

fetchPage(1, x = function(next, page) {
if (next) {
    console.log("^^^ CALLBACK -->  fetchPage: " + page);
    fetchPage(page, x);
}
});

1 Comment

I thought I understood recursion until I've read this code
0

Using async and await it is lot more easy.

router.post('/login',async (req, res, next) => {
i = await queries.checkUser(req.body);
console.log('i: '+JSON.stringify(i));
});

//User Available Check
async function checkUser(request) {
try {
    let response = await sql.query('select * from login where email = ?', 
    [request.email]);
    return response[0];

    } catch (err) {
    console.log(err);

  }

}

1 Comment

The API used in the question doesn't return a promise, so you would need to wrap it in one first … like this answer did two years ago.
-3

That defeats the purpose of non-blocking IO -- you're blocking it when it doesn't need blocking :)

You should nest your callbacks instead of forcing node.js to wait, or call another callback inside the callback where you need the result of r.

Chances are, if you need to force blocking, you're thinking about your architecture wrong.

5 Comments

I had a suspicion i had this around backwards.
Chances are, I just want to write a quick script to http.get() some URL and console.log() its contents. Why do I have to jump over backwards to do that in Node?
@DanDascalescu: And why do I have to declare type signatures to do it in static languages? And why do I have to put it in a main-method in C-like languages? And why do I have to compile it in a compiled language? What you're questioning is a fundamental design decision in Node.js. That decision has pros and cons. If you don't like it, you can use another language that fits your style better. That's why we have more than one.
@Jakob: the solutions you've listed are suboptimal indeed. That doesn't mean there aren't good ones, such as Meteor's server-side use of Node in fibers, which eliminates the callback hell problem.
@Jakob: If the best answer to "why does ecosystem X make common task Y needlessly difficult?" is "if you don't like it, don't use ecosystem X," then that is a strong sign that the designers and maintainers of ecosystem X are prioritizing their own egos above the actual usability of their ecosystem. It's been my experience that the Node community (in contrast to the Ruby, Elixir, and even PHP communities) goes out of its way to make common tasks difficult. Thank you SO MUCH for offering yourself as a living example of that antipattern.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.