removing duplicate array values from mongodb

Question

In mongodb I have collection where arrays has duplicate entries like

{
    "_id": ObjectId("57cf3cdd5f20a3b0ba009777"),
    "Chat": 6,
    "string": [
        "1348157031 Riyadh",
        " 548275320 Mohammad Sumon",
        " 1348157031 Riyadh",
        " 548275320 Mohammad Sumon",
        " 1348157031 Riyadh",
        " 1348157031 Riyadh"
    ]
}

I need to remove duplicate arrays and keep only unique array values like below.

{
    "_id": ObjectId("57cf3cdd5f20a3b0ba009777"),
    "Chat": 6,
    "string": [
        "1348157031 Riyadh",
        " 548275320 Mohammad Sumon",
    ]
}

what would be the best way to do this

thanks

Do you want to modify existing document or want to apply this to new documents only? — Nitin Verma
– Nitin Verma, Commented Sep 8, 2016 at 19:58
Possible duplicate of How to remove duplicate entries from an array? — dyouberg
– dyouberg, Commented Sep 8, 2016 at 20:04
I understand this is kind of duplicate but would be grateful if anyone could help — Sumon
– Sumon, Commented Sep 8, 2016 at 20:20
You probably need to do this client side like this (which is listed in the duplicate link) stackoverflow.com/questions/8405331/… — dyouberg
– dyouberg, Commented Sep 8, 2016 at 20:27

MohamadHossein · Accepted Answer · 2020-02-24 10:50:50Z

2

db.getCollection('Test').aggregate([{
    $unwind: '$string'},
    {
        $group: {
            _id: '$_id', 
            string: {
                $addToSet: '$string'
            }, 
            Chat: {
                $first: '$Chat'
            }
        }
    }
    ]);

O/P: here you are getting 2 "1348157031 Riyadh" because there is an extra space which defines itself as an different entity.

{
    "_id" : ObjectId("57cf3cdd5f20a3b0ba009777"),
    "string" : [ 
        " 1348157031 Riyadh", 
        " 548275320 Mohammad Sumon", 
        "1348157031 Riyadh"
    ],
    "Chat" : 6
}

edited Feb 24, 2020 at 10:50

MohamadHossein

6407 silver badges23 bronze badges

answered Sep 9, 2016 at 7:17

Shantanu Madane

6155 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Sumon Over a year ago

Thanks Shantanu, can I use any code to trim space from first element from this array.

Shantanu Madane Over a year ago

If its bymistake then i would suggest you to modify it directly in db, and always trim and save at the client side from the next time

Sumon Over a year ago

@ Shantany I am able to aggregate but how to permanently delete duplicate element from db

Shantanu Madane Over a year ago

Follow the link this could help stackoverflow.com/questions/18804404/…

Dennis Golomazov · Accepted Answer · 2018-09-24 18:06:25Z

Mongo 3.4+ has $addFields aggregation stage, which allows you to avoid explicitly listing all the other fields to keep:

collection.aggregate([
    {"$addFields": {
        "string": {"$setUnion": ["$string", []]}
    }}
])

Just for reference, here is another (more lengthy) way that uses $replaceRoot and also doesn't require listing all possible fields:

collection.aggregate([
    {'$unwind': {
        'path': '$string',
        // output the document even if its list of books is empty
        'preserveNullAndEmptyArrays': true
    }},
    {'$group': {
        '_id': '$_id',
        'string': {'$addToSet': '$string'},
        // arbitrary name that doesn't exist on any document
        '_other_fields': {'$first': '$$ROOT'},
    }},
    {
      // the field, in the resulting document, has the value from the last document merged for the field. (c) docs
      // so the new deduped array value will be used
      '$replaceRoot': {'newRoot': {'$mergeObjects': ['$_other_fields', "$$ROOT"]}}
    },
    {'$project': {'_other_fields': 0}}
])

Collectives™ on Stack Overflow

removing duplicate array values from mongodb

2 Answers 2

4 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related