Remove duplicates from array without $unwind

Question

Using query aggregation I want to create a new array by a filter of another array, so that the filtered result will be done by a specific field of the preliminary array. In this case, I want to filter by the field "fieldName".

I will always want to filter out the last occur

example: I have one document:

{
"fullyQualifiedName" : "MongoDB",
"items" : [ 
    {
        "fieldName" : "_id",
        "fieldCount" : 7,
        "confidence_level" : 1,
        "fieldClassifications" : [ 
            "LineageGuid"
        ],
    }, 
    {
        "fieldName" : "_id",
        "fieldCount" : 7,
        "fieldClassifications" : [ 
            {
                "classificationName" : "LineageGuid",
            }
        ]
    }, 
    {
        "fieldName" : "details",
        "fieldCount" : 7,
    }, 
    {
        "fieldName" : "state",
        "fieldCount" : 7,
    }
]

}

I want to create a new array like:

"items" : [ 
    {
        "fieldName" : "_id",
        "fieldCount" : 7,
        "confidence_level" : 1,
        "fieldClassifications" : [ 
            "LineageGuid"
        ],
    }, 
    {
        "fieldName" : "details",
        "fieldCount" : 7,
    }, 
    {
        "fieldName" : "state",
        "fieldCount" : 7,
    }
]

The simple solution is to $unwind and $group again but I can't do it because of performance issue. I am using MongoDB 3.4

Is there anything now working in the anwer? If "yes" then please let know the issue. — Ashh
– Ashh, Commented Nov 8, 2018 at 11:40

Ashh · Accepted Answer · 2018-11-04 17:46:14Z

1

You can use below aggregation

db.collection.aggregate([
  { "$addFields": {
    "items": {
      "$map": {
        "input": {
          "$setUnion": [
            { "$map": {
              "input": "$items",
              "in": { "$indexOfArray": ["$items.fieldName", "$$this.fieldName"] }
            }}
          ]
        },
        "as": "i",
        "in": {
          "fieldName": { "$arrayElemAt": ["$items.fieldName", "$$i"] },
          "fieldCount": { "$arrayElemAt": ["$items.fieldCount", "$$i"] },
          "confidence_level": { "$arrayElemAt": ["$items.confidence_level", "$$i"] },
          "fieldClassifications": { "$arrayElemAt": ["$items.fieldClassifications", "$$i"] }
        }
      }
    }
  }}
])

answered Nov 4, 2018 at 17:46

Ashh

46.6k16 gold badges111 silver badges137 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Remove duplicates from array without $unwind

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related