NodeJS Performance with working with arrays

Question

I want to retrieve a user's chats with corresponding users from a different collection in NodeJS and MongoDB.
The nature of NodeJS gives me a bad feeling that running the following code will block or decrease performance of my app. I can duplicate some data but I want to learn more about NodeJS.

Please let me know whether my code is ok and will not decrease performance.

Here I fetch 20 chats. I also need their corresponding users. then I get the userIds and perform another query against the User collection. Now I have both but I should merge them using Array.map.

I don't use $lookup because my collections are sharded.

$lookup

Performs a left outer join to an unsharded collection in the same database to filter in documents from the "joined" collection for processing. To each input document, the $lookup stage adds a new array field whose elements are the matching documents from the "joined" collection. The $lookup stage passes these reshaped documents to the next stage. https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/#mongodb-pipeline-pipe.-lookup

let chats = await Chat.find({ active: true }).limit(20);
/*
[
  {_id: ..., userId: 1, title: 'Chat A'},
  ...
]
*/


const userIds = chats.map(item => item.userId);
/*
[1, ...]
*/


const users = await User.find({ _id: { $in: userIds }});
/*
[
  {_id: 1, fullName: 'Jack'},
  ...
]
*/


chats = chats.map(item => {
  item.user = users.find(user => user._id === item.userId);

  return item;
});
/*
[
  {
    _id: ...,
    userId: 1,
    user: {_id: 1, fullName: 'Jack'},    // <-------- added
    title: 'Chat A'
  },
  ...
]
*/

NeNaD · Accepted Answer · 2023-04-04 14:09:04Z

2

This is NOT how you should do it. MongoDB has something called Aggregation Framework and $lookup pipeline that will do that for you automatically with only 1 MongoDB query.

But since you are using Mongoose, this query become even more simpler since you can use populate() method of the Mongoose. So your whole code can be replaced with one line like this:

const chats = await Chat.find({ active: true }).populate('userId').limit(20);

console.log(chats);

Note: If your collections are sharded, in my opinion you already implemented the logic in best possible way.

edited Apr 4, 2023 at 14:09

answered Mar 18, 2022 at 16:10

NeNaD

20.8k11 gold badges64 silver badges118 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jacqueline Over a year ago

My collections are sharded no chance for $lookup.

NeNaD Over a year ago

I see. You should add that to the question.

tuphamdev96 · Accepted Answer · 2022-03-18 16:15:47Z

1

You are using async/await, so your code will wait a response from every time use await

// Wait to finish here
let chats = await Chat.find({ active: true }).limit(20);
/*
[
  {_id: ..., userId: 1, title: 'Chat A'},
  ...
]
*/

// Wait to finish here too
const users = await User.find({ _id: { $in: userIds }});
/*
[
  {_id: 1, fullName: 'Jack'},
  ...
]
*/

So if you has too many data and you don't have any index on your collection it will be too long to finish those query. At this case you should create ref in your collection Chat to collection User with chat.userId = user._id

Then when you call query chat, you populate field userId so you don't have to map const userIds = chats.map(item => item.userId); and chats = chats.map...

Sample for chat schema

const { Schema, model } = require("mongoose");

const chatSchema = new Schema({
  active: Boolean,
  userId: {
    type: "ObjectId",
    ref: "User",
  },
  title: String,
  message: String
  // another property
});

const userSchema = new Schema({
  username: String,
  email: String
  // another property
})


// query for chat

const chatModel = new model('chat', chatSchema)
let chats = await chatModel.find({ active: true }).populate('userId').limit(20);

/*
[
  {
    _id: ...,
    userId: {_id: 1, fullName: 'Jack'},    // <-------- already have
    title: 'Chat A'
  },
  ...
]
*/

answered Mar 18, 2022 at 16:15

tuphamdev96

3441 silver badge4 bronze badges

1 Comment

Jacqueline Over a year ago

The populate method does the same behind the scene but with less code exposed to programmers. Please read: medium.com/cameoeng/…

Collectives™ on Stack Overflow

NodeJS Performance with working with arrays

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related