MongoDB Cursor - Efficient way to count AND enumerate documents (in Node)

Question

I would like to know how many documents are going to be processed by my MongoDB cursor before I iterate over the cursor. Do I have to run the query twice, or can I determine the size of a cursor and then iterate over the cursor without running the query twice?

myCount = myCollection.find({complex query}).count(); // will get my count
myCursor = myCollection.find({complex query}); // will get my cursor for iteration

but these two commands will run the query twice, which is presumably inefficient?

What is the most efficient way to determine the number of documents that I will iterate over without running the query twice? My docs are quite large.

Does MongoDB 'know' how many docs are going to be returned before it starts returning them via the cursor? Where can I read up on Mongo internals for this info?

I am running this within node, and I'm using the standard MongoDB node driver. So, the solution needs to deal with the usual node callback mechanisms etc, etc.

Thanks

EDIT

I have amended the original question to state I'm using the node driver

EDIT2

yogesh's answer is correct, I just needed to figure out the syntax for node. I've shown the working code below in case it helps anyone

db.collection('collectionName', function(err, collection) {
    function processItem(err, item) {
         if (item === null) {
               db.close();
               callback(null, {info: "yippee"});
               return; 
         }
         cursor.nextObject(processItem); // continue looping
     }
     var cursor = collection.find({foo:1});
     cursor.count(function(err,count){
         console.log('resultCursor size='+count);
         cursor.nextObject(processItem); // Start of the loop
     });
}

I guess you have to either issue 2 separate requests, or otherwise keep the current count in some other place by yourself. Essentially cursor has to iterate to the end to count the items and return you that number. Most (if not all) databases do not keep the accurate count of the data entries and I believe MongoDB is no different. This is for performance reasons: given DBs can be replicated and are multi-thread usually, it's hard to maintain and guarantee such a number. Some databases keep an estimate number though, and in MongoDB you can get that through collection stats. — scadge
– scadge, Commented Aug 14, 2023 at 8:18

Neo-coder · Accepted Answer · 2015-05-29 13:06:39Z

3

Check cursor.next this find next document in the cursor returned by the db.collection.find() method. So your case you should write as ( tested on mongo shell )

var collectionData  = myCollection.find({complex query})
collectionData.count() // return matching documents count 
collectionData.next() // return next documents

Or you also check mongo hasnext it returns true if the cursor returned by the db.collection.find() query can iterate further to return more documents.

answered May 29, 2015 at 13:06

Neo-coder

7,8585 gold badges36 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Journeyman Over a year ago

apologies, I should have been more specific: this is running within node.js. The node driver appears to work slightly differently and I'm not sure if this approach will work with the node driver and callbacks etc.

Collectives™ on Stack Overflow

MongoDB Cursor - Efficient way to count AND enumerate documents (in Node)

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related