Using Async queue to control DB connection request

Question

I'm buidling an app with Node anb Mongodb Native. I'm working on a db module which i can require and call in other modules so that I end up using just one connection. The module db.js started out with this code:

var _db = null;
var getDb = module.exports.getDb = function(callback) {
  if (_db) {
    console.log('_db returned');
    return callback(null, _db);
  }
  MongoClient.connect('mongodb://localhost:' + config.db.port + '/' + config.db.name, {native_parser: true}, function (err, db) {
    if (err) return callback(err);
    console.log('_db created');

    _db = db;
    callback(err, _db);
    });
};

In my other modules that need a db connection I do this

db.getDb(function (err, connection) {
  // Do something with connection
});

It works fine. But an unpleasant problem is that if my code would call getDb multiple times in a very short time span, I would end up with several copies of a connection. Like if I do my db.js requirements and getDb calls at the very beginning of all modules that need a db connection

I'm now thinking about controlling the calls to getDb by queuing them, so that only the absolute first call will create a connection and save it in _db. All later calls will get the created connection _db in return. I believe Async queue will help me with this...

The problem is that i dont understand how I write this with Async queue. The documentation is a little bit vague, and i dont find any better examples online. Maybe you can give me some hints. This is what i got so far...

var dbCalls = async.queue(function (task, callback) {
  if (_db) {
    console.log('_db returned');
    return callback(null, _db);
  }
  MongoClient.connect('mongodb://localhost:' + config.db.port + '/' + config.db.name, {native_parser: true}, function (err, db) {
    if (err) return callback(err);
    console.log('Connected to mongodb://localhost:' + config.db.port + '/' + config.db.name);

    _db = db;
    callback(null, _db);
  });
}, 1);

// I guess this .push() must be the exposed (exported) API for other modules to get a connection, but how do I return it to them,  
dbCalls.push(null, function (err) {
  console.log('finished processing foo');
});

dbCalls.push(null, function (err) {
  console.log('finished processing bar');
});

I dont understand the object passed as first argument to .push() What should i use if for? Right now its null How do I pass on the connection and possible error all the way out to the module that made the call?

robertklep · Accepted Answer · 2014-10-09 10:26:06Z

2

A quick and dirty solution without async.queue:

var _db      = null;
var _err     = null;
var _queue   = [];
var _pending = false;

var getDb = module.exports.getDb = function(callback) {
  if (_err || _db) {
    console.log('_db returned');
    return callback(_err, _db);
  } else if (_pending) { // already a connect() request pending
    _queue.push(callback);
  } else {
    _pending = true;
    _queue.push(callback);
    MongoClient.connect(..., function (err, db) {
      _err = err;
      _db = db;
      _queue.forEach(function(queuedCallback) {
        queuedCallback(err, db);
      });
    });
};

answered Oct 9, 2014 at 10:26

robertklep

205k38 gold badges421 silver badges413 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Anders Östman Over a year ago

Thanks! I appreciate this non dependent solution alot because it gives me an insight of how libraries like async (probably) work behind the scenes! In the long run, I think knowing how to solve problems with plain js will serve you best!

Collectives™ on Stack Overflow

Using Async queue to control DB connection request

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related