Node.js/Javascript searching in arrays of JSON objects

Question

I am fairly new to Javascript and JSON objects and taking my first foray into Node.Js.

I have a problem for which my solution is working perfectly but it seems very much like a hack and very inefficient. I'm wondering if there's a better way to do this.

I have two arrays of JSON objects coming from a MongoDB.

Activities Array

[{"_id":"538599d44ec097f59aa9f49d", "type":"A", "value":0.5},
 {"_id":"538599d44ec097f59aa9f49e", "type":"A", "value":0.25},
 {"_id":"538599d44ec097f59aa9f49f", "type":"B", "value":1.0},
 {"_id":"538599d44ec097f59aa9f4a0", "type":"B", "value":0.5}]

Transaction Counts Array

[{"_id":"538599d44ec097f59aa9f4a0", "total":3},
 {"_id":"538599d44ec097f59aa9f49e", "total":6,
 {"_id":"538599d44ec097f59aa9f49d", "total":2,]

Now basically I want to get all of the info for EVERY item out of the activities array and lookup the matching total from the transaction counts array. The transaction counts array won't have an entry for that _id if the total is 0.

Desired results:

    [{"_id":"538599d44ec097f59aa9f49d", "type":"A", "value":0.5, "total":2},
     {"_id":"538599d44ec097f59aa9f49e", "type":"A", "value":0.25, "total":6},
     {"_id":"538599d44ec097f59aa9f49f", "type":"B", "value":1.0, "total":0},
     {"_id":"538599d44ec097f59aa9f4a0", "type":"B", "value":0.5, "total":3}]

I would do this in 5 seconds in SQL using a left join from activities to transactions but can't figure out how to emulate this efficiently in MongoDb so doing it in the application instead. This is how I am currently doing it:

    var results = [];
    for (i=0; i<data.activities.length;i++) {
        var item_count = 0;
        for (j=0; j<data.txns.length;j++) {
            if (data.activities[i]._id === data.txns[j]._id) {
                item_count = data.txns[j].total;
            }
        }
        var new_json = data.activities[i];
        new_json.item_count = item_count;
        results.push(new_json);
    }

So this successfully gives me the results variable containing an array of JSON objects, each now containing the activity details and that activity's corresponding transaction total.

The cheaper way to do this seems to be to turn the nesting inside out, iterate over the transactions array first, and for each transaction object find the corresponding activity details. However, it still seems 'wrong' to be nesting for loops like this for something that seems like it should be a simple lookup from one set of data to another by id.

Thanks in advance.

Aaron

@elmigranto thanks I'm aware of that but am trying to learn Mongo as I go as well. So far I both love and hate it! — Alpaus
– Alpaus, Commented Jun 5, 2014 at 5:45
Then you should not try to recreate relational DB in Mongo, otherwise you are going to suffer and end up using Node to do what DB usually does (which is not good for obv. reasons). I'm not sure what your data is about, but left join and same _ids in different collections kind of tell that you should rethink the way you structure it. — Aleksei Zabrodskii
– Aleksei Zabrodskii, Commented Jun 5, 2014 at 7:36

elclanrs · Accepted Answer · 2014-06-05 05:40:21Z

1

Here's a different, perhaps more readable way to do what you want:

var extend = function(x, y) {
  Object.keys(y).forEach(function(k){x[k] = y[k]});
  return x;
};

var result = activities.map(function(a) {
  return extend(a, transactions.filter(function(t) {
    return t._id == a._id;
  })[0]);
});

The transaction counts array won't have an entry for that _id if the total is 0.

By the looks of your desired output, I'm not sure I understand what you mean there, but you could add a simple test inside the filter:

if (!t.total) return;

edited Jun 5, 2014 at 5:40

answered Jun 5, 2014 at 5:34

elclanrs

94.2k21 gold badges137 silver badges171 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Alpaus Over a year ago

Thanks @elclanrs. Basically the transactions object is the result of an aggregate count of the activity ids within the transaction collection. So if there are no transactions for a particular activity, it has no entry in my transactions object but I still need to recognise that activity as having 0 transactions so can't just discard it because it's not there. Are you able to explain what the above solution is doing? It's way over my head! :-)

Matt Stevens · Accepted Answer · 2014-06-14 05:34:50Z

Purely from an algorithm efficiency pov, you could benefit, first by not completing the inner loop when you have found the correct _id key, second by not repeatedly iterating over objects you know have already been matched. This will reduce the algorithm's run time by a factor of 4.

For example (marked with comments as my js is rusty, this is pseudocode);

 var results = [];
    for (i=0; i<data.activities.length;i++) {
        var item_count = 0;
        for (j=0; j<data.txns.length;j++) {
            if (data.activities[i]._id === data.txns[j]._id) {
                item_count = data.txns[j].total;
                // data.txns.remove(i);
                // break;
            }
        }
        var new_json = data.activities[i];
        new_json.item_count = item_count;
        results.push(new_json);
    }

Collectives™ on Stack Overflow

Node.js/Javascript searching in arrays of JSON objects

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related