MongoDB 'unwind' nested objects

Question

I have a collection in which each document looks something like this:

{
_id: 'dev_id:datetime_hour',
data: {
    0: { 
        0: {
            voltage_a: float, 
            voltage_b: float, 
            voltage_c: float, 
            current_a: float, 
            current_b: float, 
            current_c: float, 
            current_n: float, 
            active_power_a: float, 
            active_power_b: float, 
            active_power_c: float, 
            total_active_power: float   
        },
        1: {
            voltage_a: float, 
            voltage_b: float, 
            voltage_c: float, 
            current_a: float, 
            current_b: float, 
            current_c: float, 
            current_n: float, 
            active_power_a: float, 
            active_power_b: float, 
            active_power_c: float, 
            total_active_power: float   
        },
        2: {
            voltage_a: float, 
            voltage_b: float, 
            voltage_c: float, 
            current_a: float, 
            current_b: float, 
            current_c: float, 
            current_n: float, 
            active_power_a: float, 
            active_power_b: float, 
            active_power_c: float, 
            total_active_power: float   
        },
        59: {
            voltage_a: float, 
            voltage_b: float, 
            voltage_c: float, 
            current_a: float, 
            current_b: float, 
            current_c: float, 
            current_n: float, 
            active_power_a: float, 
            active_power_b: float, 
            active_power_c: float, 
            total_active_power: float   
        } 
    },
    1: { 
        0: {
            voltage_a: float, 
            voltage_b: float, 
            voltage_c: float, 
            current_a: float, 
            current_b: float, 
            current_c: float, 
            current_n: float, 
            active_power_a: float, 
            active_power_b: float, 
            active_power_c: float, 
            total_active_power: float   
        },
        1: {
            voltage_a: float, 
            voltage_b: float, 
            voltage_c: float, 
            current_a: float, 
            current_b: float, 
            current_c: float, 
            current_n: float, 
            active_power_a: float, 
            active_power_b: float, 
            active_power_c: float, 
            total_active_power: float   
        },
        2: {
            voltage_a: float, 
            voltage_b: float, 
            voltage_c: float, 
            current_a: float, 
            current_b: float, 
            current_c: float, 
            current_n: float, 
            active_power_a: float, 
            active_power_b: float, 
            active_power_c: float, 
            total_active_power: float   
        },
        59: {
            voltage_a: float, 
            voltage_b: float, 
            voltage_c: float, 
            current_a: float, 
            current_b: float, 
            current_c: float, 
            current_n: float, 
            active_power_a: float, 
            active_power_b: float, 
            active_power_c: float, 
            total_active_power: float   
        } 
    }
}

I've simplified it here but the fundamental idea is: sensor data is stored every second but is bundled together by the hour. The 'data' field indexes these by the minute and each minute indexes by the second. Therefore a full hours worth of data would yeild 3600 entries in the nested data field. For example to get the sensor data for the first minute and third second I could access the object directly: data.1.3.

This type of schema was recommended for storing time series data by MongoDB.

Stage 1 of my aggregation pipeline looks like this:

db.raw_electric.aggregate(
  [
    // Stage 1
    {
      $match: {
        _id: { $regex: /^r10a:/ },
        datehour: {$gte: ISODate("2016-09-21T17:00:00"), $lte: ISODate("2016-09-21T19:00:00")}
      }
    }

  ]

);

Is it possible to 'unwind' the document - similar to how you'd unwind an array so that I can expose each of the nested layers of an object?

You can't 'unwind' a document using the $unwind operator, it's only applied to arrays. — chridam
– chridam, Commented Sep 23, 2016 at 10:30
I'm aware... that's why I'm looking for an equivalent/alternative for this case. — Abdul Maye
– Abdul Maye, Commented Sep 23, 2016 at 10:31

Sachin · Accepted Answer · 2016-09-23 11:10:18Z

2

You should have created you schema in following way:

{
    _id: 'dev_id:datetime_hour',
    data: [{
        name: '0',
        info: [{}]
    }]
}

i.e.your data should be an array of objects.and from that array you can fetch any object using its index.

{
    _id: 'dev_id:datetime_hour',
    data: [{
            name: '0',
            info: [{
                voltage_a: float,
                voltage_b: float,
                voltage_c: float,
                current_a: float,
                current_b: float,
                current_c: float,
                current_n: float,
                active_power_a: float,
                active_power_b: float,
                active_power_c: float,
                total_active_power: float
            }, {
                voltage_a: float,
                voltage_b: float,
                voltage_c: float,
                current_a: float,
                current_b: float,
                current_c: float,
                current_n: float,
                active_power_a: float,
                active_power_b: float,
                active_power_c: float,
                total_active_power: float
            }, {
                voltage_a: float,
                voltage_b: float,
                voltage_c: float,
                current_a: float,
                current_b: float,
                current_c: float,
                current_n: float,
                active_power_a: float,
                active_power_b: float,
                active_power_c: float,
                total_active_power: float
            }, {
                voltage_a: float,
                voltage_b: float,
                voltage_c: float,
                current_a: float,
                current_b: float,
                current_c: float,
                current_n: float,
                active_power_a: float,
                active_power_b: float,
                active_power_c: float,
                total_active_power: float
            }],
            {
                name: '1',
                info: [{
                        voltage_a: float,
                        voltage_b: float,
                        voltage_c: float,
                        current_a: float,
                        current_b: float,
                        current_c: float,
                        current_n: float,
                        active_power_a: float,
                        active_power_b: float,
                        active_power_c: float,
                        total_active_power: float
                    }, {
                        voltage_a: float,
                        voltage_b: float,
                        voltage_c: float,
                        current_a: float,
                        current_b: float,
                        current_c: float,
                        current_n: float,
                        active_power_a: float,
                        active_power_b: float,
                        active_power_c: float,
                        total_active_power: float
                    }
                }]
        }
    }]
}

answered Sep 23, 2016 at 11:10

Sachin

2,93219 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Sachin Over a year ago

@AbdulMaye. glad to hear that It helped.. Don't forget to mark answer if it work fine. cheers.

Abdul Maye Over a year ago

So it seems this may simplify the query process but how would I go about actually updating these records? Say I wanted to append the the info array for the object that has name:1. I'm using updateOne with upsert: true.

Sachin Over a year ago

Use this query to update or push an object inside the info array: db.getCollection('test').update({"_id" : ObjectId("57e89f3c6be4d87da7824d28"),"data.name":"0"}, {$addToSet:{'data.$.info':{v1:'v1',v2:'v2'}}})

Abdul Maye Over a year ago

OK so it seems there's no simple way to do what I want. I've tried and using the positional operator $ means I cannot do upserts as my mongo driver throws an error when I try to do that and it failed to find a match in the first place... I think I'm going to have to go with a change in schema - a more flat structure...

Parth Vyas · Accepted Answer · 2017-02-17 10:26:51Z

Seems like you have multiple nested objects. In your case if you consider the total arrays that will be unwinded to get 1 hour data, it will be : 1*60*60 = 3600.

Also, multiple nesting unnecessarily adds to the complexity in retrieving and updating data.

You need a more flat structure, which can be attained as follows :-

Create separate document for each minute. The structure would something be like -

{
 _id: ObjectId('');
 hour: 1,
 minute: 1
 seconds: [
  {
   item: 0,
   voltage_a: float,
   voltage_b: float,
   voltage_c: float,
   current_a: float,
   current_b: float,
   current_c: float,
   current_n: float,
   active_power_a: float,
   active_power_b: float,
   active_power_c: float,
   total_active_power: float
  },
  {
   item: 1,
   voltage_a: float,
   voltage_b: float,
   voltage_c: float,
   current_a: float,
   current_b: float,
   current_c: float,
   current_n: float,
   active_power_a: float,
   active_power_b: float,
   active_power_c: float,
   total_active_power: float
  }
 ]
},
{
 _id: ObjectId('');
 hour: 1,
 minute: 2
 seconds: [
  {
   item: 0,
   voltage_a: float,
   voltage_b: float,
   voltage_c: float,
   current_a: float,
   current_b: float,
   current_c: float,
   current_n: float,
   active_power_a: float,
   active_power_b: float,
   active_power_c: float,
   total_active_power: float
  },
  {
   item: 1,
   voltage_a: float,
   voltage_b: float,
   voltage_c: float,
   current_a: float,
   current_b: float,
   current_c: float,
   current_n: float,
   active_power_a: float,
   active_power_b: float,
   active_power_c: float,
   total_active_power: float
  }
 ]
}

This structure might subsequently increase the number of documents. But will reduce the complexity involved in querying. Also, with efficient indexing the performance can be maintained. (For large data, you should also look into sharding).

Collectives™ on Stack Overflow

MongoDB 'unwind' nested objects

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related