Getting Unique results for Objects inside List in Elasticsearch

Question

I have mapping like this

  "custom_metadata": {
    "properties": {
      "key": {
        "type": "keyword"
      },
      "value": {
        "type": "keyword"
      }
    }
  }

The ingested data looks like this

// data in document 1
"custom_metadata": [
    {
      "value": "NPL",
      "key": "schema"
    },
    {
      "value": "SAPERP",
      "key": "system"
    }
  ]

// data in document 2
"custom_metadata": [
    {
      "value": "trial",
      "key": "schema"
    },
    {
      "value": "Oracle",
      "key": "system"
    }
  ]

I want to aggregate on each key and get relevant value in search results, like this

"schema": [
  {"value": "NPL"},
  { "value": "trial",}
],
"system":[
  {"value": "SAPERP"},
  { "value": "Oracle",}
]

Note: Above output is for representation.If I get something like in ES then I can parse and get desired result on service side

What I have tried:

"custom_metadata_key": {
      "terms": {
        "field": "custom_metadata.key"
      },
      "aggregations": {
        "custom_metadata_value": {
          "terms": {
            "field": "custom_metadata.value"
          }
        }
      }
    }

Above nested agg , aggregates each key and give all the values in results.

{
          "key" : "schema",
          "doc_count" : 2,
          "custom_metadata_value" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "Oracle",
                "doc_count" : 2
              },
              {
                "key" : "NPL",
                "doc_count" : 1
              },
              {
                "key" : "SAPERP",
                "doc_count" : 1
              },
              {
                "key" : "trial",
                "doc_count" : 1
              }
            ]
          }
        }

Above output repeats for all key and gives same aggregation for all.

Sagar Patel · Accepted Answer · 2022-09-22 11:06:19Z

You need to change data type of field custom_metadata from object to nested and you can achive your desire output easily.

Mapping

{
  "mappings": {
    "properties": {
      "custom_metadata":{
        "type": "nested"
      }
    }
  }
}

Query

{
  "size": 0,
  "aggs": {
    "data": {
      "nested": {
        "path": "custom_metadata"
      },
      "aggs": {
        "custom_metadata_key": {
          "terms": {
            "field": "custom_metadata.key.keyword",
            "size": 10
          },
          "aggs": {
            "custom_metadata_value": {
              "terms": {
                "field": "custom_metadata.value.keyword",
                "size": 10
              }
            }
          }
        }
      }
    }
  }
}

Output

"aggregations": {
    "data": {
      "doc_count": 4,
      "custom_metadata_key": {
        "doc_count_error_upper_bound": 0,
        "sum_other_doc_count": 0,
        "buckets": [
          {
            "key": "schema",
            "doc_count": 2,
            "custom_metadata_value": {
              "doc_count_error_upper_bound": 0,
              "sum_other_doc_count": 0,
              "buckets": [
                {
                  "key": "NPL",
                  "doc_count": 1
                },
                {
                  "key": "trial",
                  "doc_count": 1
                }
              ]
            }
          },
          {
            "key": "system",
            "doc_count": 2,
            "custom_metadata_value": {
              "doc_count_error_upper_bound": 0,
              "sum_other_doc_count": 0,
              "buckets": [
                {
                  "key": "Oracle",
                  "doc_count": 1
                },
                {
                  "key": "SAPERP",
                  "doc_count": 1
                }
              ]
            }
          }
        ]
      }
    }
  }

Collectives™ on Stack Overflow

Getting Unique results for Objects inside List in Elasticsearch

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related