mongodb delete nested object without knowledge of object nodes

Question

For the below document, I am trying to delete the node which contains id = 123

 {
       '_id': "1234567890",
        "image" : {
            "unknown-node-1" : {
                "id" : 123
            },
            "unknown-node-2" : {
                "id" : 124
            }
        }
    }

Result should be as below.

{
    '_id': "1234567890",
    "image" : {        
        "unknown-node-2" : {
            "id" : 124
        }
    }
}

The below query achieves the result. But i have to know the unknown-node-1 in advance. How can I achieve the results without pre-knowledge of node, but only info that I have is image.*.id = 123

(* means unknown node)

Is it possible in mongo? or should I do these find on my app code.

db.test.update({'_id': "1234567890"}, {$unset: {'image.unknown-node-1': ""}})

If you don't know the subdocument name in advance, you can't do it with the document structure you have right now. You could do it if you put all "unknown-node-N" docs into an array. — Dmytro Shevchenko
– Dmytro Shevchenko, Commented Nov 23, 2015 at 14:09

Abdullah Rasheed · Accepted Answer · 2015-11-27 14:08:56Z

4

Faiz,

There is no operator to help match and project a single key value pair without knowing the key. You'll have to write post processing code to scan each one of the documents to find the node with the id and then perform your removal.

If you have the liberty of changing your schema, you'll have more flexibilty. With a document design like this:

 {
       '_id': "1234567890",
        "image" : [
            {"id" : 123, "name":"unknown-node-1"},
            {"id" : 124, "name":"unknown-node-2"},
            {"id" : 125, "name":"unknown-node-3"}
        ]
  }

You could remove documents from the array like this:

db.collectionName.update(
  {'_id': "1234567890"},
  { $pull: { image: { id: 123} } }
)

This would result in:

{
    '_id': "1234567890",
    "image" : [
            {"id" : 124, "name":"unknown-node-2"},
            {"id" : 125, "name":"unknown-node-3"}
        ]
}

edited Nov 27, 2015 at 14:08

answered Nov 23, 2015 at 14:15

Abdullah Rasheed

3,7624 gold badges35 silver badges51 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Faiz Mohamed Haneef Over a year ago

had the liberty to change the design and went ahead with that option after looking at Chridam's answer on map reduce.

Abdullah Rasheed Over a year ago

Yes, Chridam always seems to know how to provide a solution to the hardest situations when it comes to Mongodb, i'm inspired by alot of his answers. I'm glad I was able to help out!

R_AD Over a year ago

Just wanted to call out saying this is a fantastic answer. I am new to Mongo and was diving down a rabbit hole that had dynamic keys in my documents. I feel like this re-wired my brain a bit and now I am seeing a much easy path forward with my document model. Thank you @AbdullahRasheed!

chridam · Accepted Answer · 2015-11-23 14:52:49Z

With your current schema, you will need a mechanism to get a list of the dynamic keys that you need to assemble the query before doing the update and one way of doing this would be with MapReduce. Take for instance the following map-reduce operation which will populate a separate collection with all the keys as the _id values:

mr = db.runCommand({
    "mapreduce": "test",
    "map" : function() {
        for (var key in this.image) { emit(key, null); }
    },
    "reduce" : function(key, stuff) { return null; }, 
    "out": "test_keys"
})

To get a list of all the dynamic keys, run distinct on the resulting collection:

> db[mr.result].distinct("_id")
[ "unknown-node-1", "unknown-node-2" ]

Now given the list above, you can assemble your query by creating an object that will have its properties set within a loop. Normally if you knew the keys beforehand, your query will have this structure:

var query = {
        "image.unknown-node-1.id": 123
    },
    update = {
        "$unset": {
            "image.unknown-node-1": ""
        }
    };
db.test.update(query, update);

But since the nodes are dynamic, you will have to iterate the list returned from the mapReduce operation and for each element, create the query and update parameters as above to update the collection. The list could be huge so for maximum efficiency and if your MongoDB server is 2.6 or newer, it would be better to take advantage of using a write commands Bulk API that allow for the execution of bulk update operations which are simply abstractions on top of the server to make it easy to build bulk operations and thus get perfomance gains with your update over large collections. These bulk operations come mainly in two flavours:

Ordered bulk operations. These operations execute all the operation in order and error out on the first write error.
Unordered bulk operations. These operations execute all the operations in parallel and aggregates up all the errors. Unordered bulk operations do not guarantee order of execution.

Note, for older servers than 2.6 the API will downconvert the operations. However it's not possible to downconvert 100% so there might be some edge cases where it cannot correctly report the right numbers.

In your case, you could implement the Bulk API update operation like this:

mr = db.runCommand({
    "mapreduce": "test",
    "map" : function() {
        for (var key in this.image) { emit(key, null); }
    },
    "reduce" : function(key, stuff) { return null; }, 
    "out": "test_keys"
})
// Get the dynamic keys
var dynamic_keys = db[mr.result].distinct("_id");

// Get the collection and bulk api artefacts
var bulk = db.test.initializeUnorderedBulkOp(), // Initialize the Unordered Batch
    counter = 0;    

// Execute the each command, triggers for each key
dynamic_keys.forEach(function(key) {            
    // Create the query and update documents
    var query = {},
        update = {
            "$unset": {}
        };

    query["image."+ key +".id"] = 123;
    update["$unset"]["image." + key] = ";"

    bulk.find(query).update(update);            
    counter++;

    if (counter % 100 == 0 ) {
        bulk.execute() {  
        // re-initialise batch operation           
        bulk = db.test.initializeUnorderedBulkOp();
    }
});             

if (counter % 100 != 0) { bulk.execute(); }

upvoted as the solution logically would get my question answered (havent tried to implement you answer). but your answer did inspire me to change the mongdb design. thank you.

Collectives™ on Stack Overflow

mongodb delete nested object without knowledge of object nodes

2 Answers 2

3 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related