MongoDB NodeJS Native Driver(mongodb) vs Mongo Shell Performance

Question

I have 10000 records in MongoDB in table1.

Data is as below:

"_id" : ObjectId("5d5e500cb89312272cfe51fc"),
"cities" : [ 
    {
        "cityid" : "5d5d2205cdd42d1cf0a92b33",
        "value" : "XYZ"
    }, 
    {
        "cityid" : "5d5d2214cdd42d1cf0a92b34",
        "value" : "Rowcliffe"
    }, 
],

Query is as below:

      {
        $unwind: "$cities"
      },
      { "$addFields": { "cities.cityid": { "$toObjectId": "$cities.cityid" } } },
      {
        $lookup: {
          from: "cities",
          localField: "cities.cityid",
          foreignField: "_id",
          as: "docs"
        }
      },

So, here i lookup cityid in another table with lookup query in Robo3T and mongo shell. All works fine.

I am getting result in 0.08 sec for 10000 records.

Now, same query m running in nodejs with mongodb native driver, here m getting result in 15 sec.

I'm not getting why this huge difference between this.I don't know what i am doing wrong in nodejs.I have written the same query in nodejs with mongodb native driver.

Please let me know what i m doing wrong.

Why this nodejs mongodb native driver performance is so poor?

In robo3t add a default limit to the query. Try to put DBQuery.shellBatchSize = 500000; to see the same performance of node.js — Manuel Spigolon
– Manuel Spigolon, Commented Sep 4, 2019 at 13:56
@ManuelSpigolon correct.! after changing shellBatchsize to 500000, i am getting same time as node js 15 sec. thats mean mongodb joining is that much slow for just 10000 records..!! — chirag
– chirag, Commented Sep 5, 2019 at 6:05

Manuel Spigolon · Accepted Answer · 2019-09-05 06:41:16Z

1

In Robo3T there is a default query limit so it takes less time because when it fetches the limit its exit.

To avoid it you need to add to your query execution this snipped:

// change the limit size, default 50
DBQuery.shellBatchSize = 500000;

answered Sep 5, 2019 at 6:41

Manuel Spigolon

13.4k7 gold badges60 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

chirag Over a year ago

thanks for answer. but why i am getting so much delay in lookup for just 10000 records..?? its simple aggregation query with indexing..

Manuel Spigolon Over a year ago

In this case you could check the explain of the query, in this way you will understand if you are using or not indexes or you are doing a tablescan

chirag Over a year ago

I have tried explain(), and its showing that it uses indexes....still it is taking too much time

Manuel Spigolon Over a year ago

I think the lookup it is heavy since the "join" in a no-sql is not recommended, you could think to remove that part and add some code to fetch the data that you need: 2 query and a join via code through a Map

chirag Over a year ago

but i think aggregation is faster than MapReduce, so how it will give result fatser than this??

|

Collectives™ on Stack Overflow

MongoDB NodeJS Native Driver(mongodb) vs Mongo Shell Performance

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related