Aggregate multiple arrays into one huge array with MongoDB

Question

I got a collection with documents such as those:

{
    "_id": 0,
    "pictures": [
        {
            "path": "/path/to/first/picture.jpg",
            "web": "true"
        },
        {
            "path": "/path/to/second/picture.jpg",
            "web": "true",
        }
    ],
    "logos": [
        {
            "path": "/path/to/first/logo.jpg",
            "web": "true"
        },
        {
            "path": "/path/to/second/logo.jpg",
            "web": "false",
        }
    ]
}
{
    "_id": 1,
    "pictures": [
        {
            "path": "/a/bit/weird/path/to/picture.jpg",
            "web": "false"
        },
        {
            "path": "/another/weird/path/to/picture.jpg",
            "web": "false",
        }
    ],
    "logos": [
        {
            "path": "/path/to/another/logo.jpg",
            "web": "false"
        },
        {
            "path": "/a/last/path/to/logo.jpg",
            "web": "true",
        }
    ]
}

What I'm trying to get as result is this one with the conditions below.

{ 
    "web_images": [
        {
            "path": "/path/to/first/picture.jpg",
            "web": "true"
        },
        {
            "path": "/path/to/second/picture.jpg",
            "web": "true",
        },
        {
            "path": "/path/to/first/logo.jpg",
            "web": "true"
        },
        {
            "path": "/a/last/path/to/logo.jpg",
            "web": "true",
        }
    ]
}

The only real condition on that is to get all paths, that have the string at the field "web" set on true. (I know it could also be bool, but it doesn't matter in this case) The main problem is to concat the arrays "logos" and "pictures" into one array. I read some solutions using "$unwind". But in those cases, they are trying to concatenate one array.

This worked for me, but in my real case I got five different arrays containing those objects. And they have to result all in one array.

My solution so far looks as following (but doesn't work):

db.products.aggregate( [
    { $unwind: "$pictures" },
    { $unwind: "$logos" },
    { $group: { _id: null, pics: { $push: { $or: ["$pictures", "$logos"] } } } }
    { $project: { _id: 0, pictures: "$pics" } }
] );

As you can see there is still the condition checking on the "web"-flag missing. Also the output brings absolutely nothing. Not even an error message.

EDIT

I forgot an important detail on this question.

As I described in the upper part there are five different arrays that can appear in each document. It is also possible that there aren't any of them in each document. So it can also look like this:

{
    "_id": 1,
    "pictures": [
        {
            "path": "/a/bit/weird/path/to/picture.jpg",
            "web": "false"
        },
        {
            "path": "/another/weird/path/to/picture.jpg",
            "web": "false",
        }
    ]
}

That could be the difference between my problem the solved one, which my was marked as duplicate to. As @Styvane told me to do it gave me just tons of documents such as this:

{
    "_id" : ObjectId("57f5026aaf39013d0c9186af"),
    "web_images": null
}

It is possible that I get this error because of those documents which have some arrays missing.

For the condition, just wrap the $setUnion expression in $filter: db.collection.aggregate([{"$project": { "web_images": { "$filter": { "input": { "$setUnion": [ "$pictures", "$logos" ] }, "as": "p", "cond": { "$eq": [ "$$p.web", "true" ] } } } } }]) — Sede
– Sede, Commented Mar 15, 2017 at 14:41
What does the $$ in "$eq": [ "$$p.web", "true"] do? And it doesn't work for me. I just get thousands of documents with an id and "web_images": null — Max
– Max, Commented Mar 15, 2017 at 14:54
p is a variable name for each element in the concatenated array. To access that variable in the cond expression, MongoDB required that you prefix it with $$ and use the "dot notation" to access the subdocument field. I hope I answered your question. — Sede
– Sede, Commented Mar 15, 2017 at 14:59
yep, thanks for that, but it still doesn't want to give me my result in one document. I still get tons of useless documents. { "_id": ObjectId("someHash"), "web_images": null } — Max
– Max, Commented Mar 15, 2017 at 15:01
Oops I could have forgotten one important detail: It is also possible that a document doesn't contain one or both fields. It's possible that it only contains "pictures", only "logos" or even none of both. Could that be a big problem? — Max
– Max, Commented Mar 15, 2017 at 15:05

Sede · Accepted Answer · 2017-03-15 15:35:43Z

2

You can use the $filter and the $setUnion/$concatArrays operators to concatenate and filter your documents. Also you need to use the $ifNull operator to replace the missing field with empty array.

db.collection.aggregate([
    { "$project": { 
        "web_images": { 
            "$filter": { 
                "input": { 
                    "$setUnion": [ 
                        { "$ifNull": [ "$pictures", [] ] },
                        { "$ifNull": [ "$logos", [] ] }
                    ]
                }, 
                "as": "p", 
                "cond": { "$eq": [ "$$p.web", "true" ] } 
            } 
        } 
    }},
    { "$match": { "web_images.0": { "$exists": true } } }
])

answered Mar 15, 2017 at 15:35

Sede

61.5k20 gold badges158 silver badges162 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Max Over a year ago

Thank you so much! I added a $unwind and then a $group to the pipeline at the "web_images"-field. This results in one single document with every element that matches the criterias in just one array. That was exactly what I was looking for!

Collectives™ on Stack Overflow

Aggregate multiple arrays into one huge array with MongoDB

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related