Lazy loading using nodejs and mongoDB as backend data

Question

Our collection has around 100 million documents. We created a simple application using nodejs and expressjs with a limit clause in the mongo query . It is sufficient for the users as of now. In the mean time we are trying to implement lazy loading so that the initial page load returns few documents and when the user scrolls we would like to load further documents. Struggling where to start and how to ahead to implement it. Appreciate your suggestions. My index.js file would look like this

router.get('/users', function(req, res) {
  var db = req.db; 
  var users = db.get('users'); 

  users.find(query, {limit: 10000}, function(e, docs){
    res.render('users', { 
      title: 'Users',
      'users': docs  
    });
  });
});

I would like to remove the limit and using skip trying to achieve this. Please post your suggestions

SarathMS · Accepted Answer · 2015-05-15 07:22:18Z

7

This should help. It uses the .skip method of the .find() cursor. I call it pagination rather than lazy loading.

var itemsPerPage = 10;

router.get('/users/:pageNum', function(req, res) {
  var db = req.db; 
  var users = db.get('users'); 
  users.find(query, {skip: (itemsPerPage * (pageNum-1)), limit: itemsPerPage},function(e, docs){
    res.render('users', { 
      title: 'Users',
      'users': docs  
    });
  });
});

answered May 15, 2015 at 7:22

SarathMS

5392 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

sasidharan Over a year ago

so from where pageNum should be passed , If we do it as pagination ?

SarathMS Over a year ago

That will be part of the request you make. http://server.tld/users/2. I haven't used expressjs a lot, so I'm not sure if that is how you define query params.

SarathMS Over a year ago

I also agree with @bidhan-a's suggestion to use Mongoosejs.

sasidharan Over a year ago

Thanks very much . I did it with javascript with 2 buttons namely previous and next. it works

Bulkan · Accepted Answer · 2015-05-15 07:07:09Z

0

You could implement paging in your route

Something like below;

// GET /users?start=20
router.get('/users', function(req, res) {
  var db = req.db; 

  var start = parseInt(req.query.start) || 0;
  var users = db.get('users'); 

  users.find(query, {start: start, limit: 10000}, function(e, docs){
    res.render('users', { 
      title: 'Users',
      'users': docs  
    });
  });
});

answered May 15, 2015 at 7:07

Bulkan

2,5921 gold badge22 silver badges32 bronze badges

Comments

Bidhan · Accepted Answer · 2015-05-15 07:05:40Z

-2

I suggest you use Mongoose to model the MongoDB objects efficiently and then make use of one of many mongoose pagination modules available.

answered May 15, 2015 at 7:05

Bidhan

10.7k3 gold badges42 silver badges51 bronze badges

Collectives™ on Stack Overflow

Lazy loading using nodejs and mongoDB as backend data

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related