Using nested document structure in mongodb

Question

I am planning to use a nested document structure for my MongoDB Schema design as I don't want to go for flat schema design as In my case I will need to fetch my result in one query only.

Since MongoDB has a size limit for a document. MongoDB Limits and Threshold

A MongoDB document has a size limit of 16MB ( an amount of data). If your subcollection can growth without limits go flat.

I don't need to fetch my nested data but only be needing my nested data for filtering and querying purpose.

I want to know whether I will still be bound by MongoDB size limits even if I use my embedded data only for querying and filter purpose and never for fetching of nested data because as per my understanding, in this case, MongoDB won't load the complete document in memory but only the selected fields?

Nested schema design example

{
    clinicName: "XYZ Hopital",
    clinicAddress: "ABC place.",
    "doctorsWorking":{
        "doctorId1":{
            "doctorJoined": ISODate("2017-03-15T10:47:47.647Z")
        },
        "doctorId2":{
            "doctorJoined": ISODate("2017-04-15T10:47:47.647Z")
        },
        "doctorId3":{
            "doctorJoined": ISODate("2017-05-15T10:47:47.647Z")
        },
        ...
        ...
        //upto 30000-40000 more records suppose
    }
}

love gupta · Accepted Answer · 2017-04-06 06:21:26Z

2

I don't think your understanding is correct when you say "because as per my understanding, in this case, MongoDB won't load the complete document in memory but only the selected fields?".

If we see MongoDB Doc. then it reads

The maximum BSON document size is 16 megabytes. The maximum document size helps ensure that a single document cannot use excessive amount of RAM or, during transmission, excessive amount of bandwidth. To store documents larger than the maximum size, MongoDB provides the GridFS API.

So the clear limit is 16 MB on document size. Mongo should stop you from saving such a document which is greater than this size.

If I agree with your understanding for a while then let's say that it allows to save any size of document but more than 16 MB in RAM is not allowed. But on other hand, while storing the data it won't know what queries will be run on this data. So ultimately you will be inserting such big documents which can't be used later. (because while inserting we don't tell the query pattern, we can even try to fetch the full document in a single shot later).

If the limit is on transmission (hypothetically assuming) then there are lot of ways (via code) software developers can bring data into RAM in clusters and they won't cross 16 MB limit ever (that's how they do IO ops. on large files). They will make fun of this limit and just leave it useless. I hope MongoDB creators knew it and didn't want it to happen.

Also if limit is on transmission then there won't be any need of separate collection. We can put everything in a single collections and just write smart queries and can fetch data. If fetched data is crossing 16 MB then fetch it in parts and forget the limit. But it doesn't go this way.

So the limit must be on document size else it can create so many issues.

In my opinion if you just need "doctorsWorking" data for filtering or querying purpose (and if you also think that "doctorsWorking" will cause document to cross 16 MB limit) then it's good to keep it in a separate collection.

Ultimately all things depend on query and data pattern. If a doctor can serve in multiple hospitals in shifts then it will be great to keep doctors in separate collection.

edited Apr 6, 2017 at 6:21

answered Apr 6, 2017 at 6:00

love gupta

5395 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Robins Gupta Over a year ago

Thanks for taking your time to answer. I won't not be loading the whole document but only a part of it while updating like

db.Clinic.update({"_id" : ObjectId("58e5e3ec54ccefdd4d85f638")}, { $set: { "doctorsWorking.doctor5": {             "doctorJoined": ISODate("2017-05-15T10:47:47.647Z")         } }}

Using this method it will only update specific field provided. What I want to know is that will it load the complete document in memory even using this method or only a part of it which is getting updated? docs.mongodb.com/manual/reference/operator/update/set

Vince Bowdren Over a year ago

@RobinsGupta MongoDB will know that the entire document is too large, and stop the update.

love gupta Over a year ago

@RobinsGupta It won't allow any document more than 16 MB. Either by any update or by insert or by upsert.

Collectives™ on Stack Overflow

Using nested document structure in mongodb

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related