AWS Lambda (Node.js, v. 8.10) & Mongoose: MongoNetworkError connection to DB timed out

Question

Env.:

AWS Lambda (Node.js, v. 8.10), waitForEmptyEventLoop === false
MongoDB (Atlas)
Mongoose

Problem: sometimes (randomly) I get next error:

MongoNetworkError: connection 6 to db_host:27017 timed out
  File "/opt/nodejs/node_modules/mongodb-core/lib/connection/connection.js", line 259, col 7, in TLSSocket.<anonymous>
    new MongoNetworkError(f('connection %s to %s:%s timed out', self.id, self.host, self.port)),
  File "events.js", line 313, col 30, in Object.onceWrapper
  File "events.js", line 106, col 13, in emitNone
  File "events.js", line 208, col 7, in TLSSocket.emit
  File "net.js", line 420, col 8, in TLSSocket.Socket._onTimeout
  File "timers.js", line 482, col 11, in ontimeout
  File "timers.js", line 317, col 5, in tryOnTimeout
  File "timers.js", line 277, col 5, in Timer.listOnTimeout

Code of db connection:

const mongoose = require('mongoose');
const log = require('./log');

const options = {
  reconnectTries: 30,
  reconnectInterval: 500,
  poolSize: Number(process.env.DB_POOLSIZE) || 1,
  socketTimeoutMS: 30000,
  keepAlive: true,
  bufferCommands: false,
  bufferMaxEntries: 0,
};

let isConnected;

module.exports.connect = () => new Promise((resolve, reject) => {
  if (isConnected) {
    return resolve();
  }

  return mongoose.connect(process.env.DB_URI, options)
    .then((db) => {
      isConnected = db.connections[0].readyState;
      resolve();
    }).catch((error) => {
      log.error('DB:', error);
      reject(error);
    });
});

I've checked - isConnected cached successfully, mongodb connection cached in mongoose. All must be okay, but sometimes I get this error.

Anybody have any ideas how can I solve this issue?

Same problem here, in production/dev environments I see this error on about 1% of executions. Did you find any solution/workaround? — Rodrigo Reis
– Rodrigo Reis, Commented Jun 3, 2019 at 18:10
@RodrigoReis I use next config (mongoose): { reconnectTries: 30, reconnectInterval: 500, poolSize: 1, socketTimeoutMS: 2000000, keepAlive: true, } — Max Vynohradov
– Max Vynohradov, Commented Jun 3, 2019 at 19:31
Just increase socketTimeoutMS - for me 2000000 enough to keep connection between lambdas invocation of "warm" container — Max Vynohradov
– Max Vynohradov, Commented Jun 3, 2019 at 19:33
Another variant (smbd treat this as best practice) - create/close connection for each lambda invocation. Not bad idea if you know that lambda will invoked rarely — Max Vynohradov
– Max Vynohradov, Commented Jun 3, 2019 at 19:35
Thank you @max-vinogradov . I'll try the mentioned configurations. If it really works for you you should post as an Answer :) — Rodrigo Reis
– Rodrigo Reis, Commented Jun 3, 2019 at 20:22

Max Vynohradov · Accepted Answer · 2019-06-06 09:48:30Z

3

Just increase socketTimeoutMS - for me 2000000 enough to keep connection between lambdas invocation of "warm" container. Use next config (mongoose):

 { reconnectTries: 30, reconnectInterval: 500, poolSize: 1, socketTimeoutMS: 2000000, keepAlive: true, }

Another variant (smbd treat this as best practice) - create/close connection for each lambda invocation. Not bad idea if you know that lambda will invoked rarely

answered Jun 6, 2019 at 9:48

Max Vynohradov

1,5782 gold badges16 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

electric.Kariim · Accepted Answer · 2019-01-16 22:02:38Z

2

You need a 'keep warm' script/functionality that will keep the connection alive/warm by querying every so often.

This issue is caused by DB connections timing out or expiring since Lambda spins up a new instance of computing every time the function is executed. It doesn't keep the existing connection warm.

This is a known issue with serverless whereby DB connection persistence is intermittent.

answered Jan 16, 2019 at 22:02

electric.Kariim

1201 silver badge7 bronze badges

2 Comments

Max Vynohradov Over a year ago

I use warm-up (serverless-warm-up plugin) - it invokes lambdas each 15 minutes. It doesn't help for this issue.

Mikel David Carozzi Sanchez Over a year ago

Non VPC lambdas died at 5 mins, VPC at 10 mins. so you need to tune your EventBridge setting.

Collectives™ on Stack Overflow

AWS Lambda (Node.js, v. 8.10) & Mongoose: MongoNetworkError connection to DB timed out

2 Answers 2

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related