mongodb connection timeout error with mongoose and nodejs

Question

I desperately need some help.I am trying to upload large file(8 GB) to gridfs using mongoose and nodeJS. But as the file is very large it takes some time to upload. And after a while I get the following error:

home/user/FileUpload/node_modules/mongodb/lib/utils.js:98
    process.nextTick(function() { throw err; });
                                  ^
MongoError: connection 0 to 127.0.0.1:27017 timed out
    at Function.MongoError.create (/home/user/FileUpload/node_modules/mongodb-core/lib/error.js:29:11)
    at Socket.<anonymous> (/home/user/FileUpload/node_modules/mongodb-core/lib/connection/connection.js:186:20)
    at Object.onceWrapper (events.js:314:30)
    at emitNone (events.js:105:13)
    at Socket.emit (events.js:207:7)
    at Socket._onTimeout (net.js:402:8)
    at ontimeout (timers.js:488:11)
    at tryOnTimeout (timers.js:323:5)
    at Timer.listOnTimeout (timers.js:283:5)

I have tried to resolve this by increasing connectTimeoutMS but the error still persist. I am using MongoDB 3.4.5 mongoose 4.8.4 nodejs 8.1.4 and npm 5.0.3.

Following is app.js:

var mongoose = require('mongoose');
var schema = mongoose.schema;
mongoose.connect('mongodb://127.0.0.1/gridFS'),{
    server: {
        socketOptions: {
            socketTimeoutMS: 3000000,
            connectionTimeoutMS: 3000000,
            keepAlive:3000000
        }
    },

    replset: {
        socketOptions: {
            keepAlive: 3000000,
            connectTimeoutMS: 3000000
        }
    }
};
var conn = mongoose.connection;
var path = require('path');
var Grid = require('gridfs-stream');
var fs = require('fs');
var videoPath = path.join(__dirname, 'readFrom/bio seq test1.txt');
Grid.mongo = mongoose.mongo;
conn.once('open', function(){

    console.log('- connection open -');
    var gfs = Grid(conn.db);

    var writestream = gfs.createWriteStream({

        filename: 'bio seq test 1'
    });

    fs.createReadStream(videoPath).pipe(writestream);
    writestream.on('close', function(file){
      console.log(file.filename + 'Written to DB');
    });
});

Following is package.json file:

{
  "name": "file-upload-gridfs",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo '' &amp;&amp; exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "body-parser": "^1.16.1",
    "cookie-parser": "^1.4.3",
    "express": "^4.14.1",
    "gridfs-stream": "^1.1.1",
    "mongoose": "^4.8.4",
    "morgan": "^1.8.2",
    "multer": "1.3.0",
    "multer-gridfs-storage": "1.0.0",
    "path.join": "^1.0.0",
    "serve-favicon": "^2.4.3"
  }
}

Arno van Oordt · Accepted Answer · 2024-06-03 15:56:14Z

10

Ok. I figured out the problem using this really helpful discussion.The default socket connection time for MongoDB is 30 seconds. If any query/operation takes longer than this the connection is aborted and connection timeout error occurs. With this change I was able to upload a 32GB file to GridFS without any interruption.

https://github.com/Automattic/mongoose/issues/4789

I was passing the timeout parameter in following way.

server: {
        socketOptions: {
            socketTimeoutMS: 3000000,
            connectionTimeoutMS: 3000000,
            keepAlive:3000000
        }
    },

    replset: {
        socketOptions: {
            keepAlive: 3000000,
            connectTimeoutMS: 3000000
        }
    }
};

But it needs to be set in the following way:

const serverOptions = {
  poolSize: 100,
  socketOptions: {
    socketTimeoutMS: 6000000
  }
};

mongoose.createConnection(dbpath, {
  server: serverOptions,
  replset: serverOptions //if you are using replication
});

In my case I have used localhost.

const serverOptions = {
    poolsize:100 ,
    socketOptions:{
        socketTimeoutMS: 6000000
    }
};
var mongodbUri = 'mongodb://localhost:27017/gridFS';
mongoose.connect(mongodbUri, {
    server: serverOptions
});

Hope this will help anyone with similar issue.

edited Jun 3, 2024 at 15:56

Arno van Oordt

3,6507 gold badges43 silver badges78 bronze badges

answered Jul 19, 2017 at 21:39

jony70

2251 gold badge2 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Soorya Prakash Over a year ago

I am using Mongodb 3.2 and Native Mongodb driver 2.2.4. I executing the aggregate query with $out at the end. I have added the above config during db connection.But still I am getting the error as "MongoError: connection 489 to timed out"

Greg Wozniak Over a year ago

Now I am getting: options are deprecated, all their options are supported at the top level of the options object

Collectives™ on Stack Overflow

mongodb connection timeout error with mongoose and nodejs

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related