1

Would love an explanation of why this happens and how to correct it.

Here's a snippet of the source document:

{
   "created_time":1412988495000,
   "tags":{
      "items":[
         {
            "tag_type":"Placement",
            "tag_id":"id1"
         },
         {
            "tag_type":"Product",
            "tag_id":"id2"
         }
      ]
   }
}

The following terms aggregation:

  "aggs":{
       "tags":{
          "terms":{
             "script":"doc['tags'].value != null ? doc['tags.items.tag_type'].value + ':' + doc['tags.items.tag_id'].value : ''",
             "size":2000,
             "exclude":{
                "pattern":"null:null"
             }
          }
       }
    }

returns:

   "buckets":[
      {
         "key":"Placement:id1",
         "doc_count":1
      },
      {
         "key":"Placement:id2",
         "doc_count":1
      }
   ]

...when you would expect:

   "buckets":[
      {
         "key":"Placement:id1",
         "doc_count":1
      },
      {
         "key":"Product:id2",
         "doc_count":1
      }
   ]

1 Answer 1

1

I would probably go with a nested type. I don't know all the details of your setup, but here is a proof of concept, at least. I took out the "items" property because I didn't need that many layers, and just used "tags" as the nested type. It could be added back in if needed, I think.

So I set up an index with a "nested" property:

DELETE /test_index

PUT /test_index
{
   "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 0
   },
   "mappings": {
      "doc": {
         "properties": {
            "created_time": {
               "type": "date"
            },
            "tags": {
               "type": "nested",
               "properties": {
                  "tag_type": {
                     "type": "string",
                     "index": "not_analyzed"
                  },
                  "tag_id": {
                     "type": "string",
                     "index": "not_analyzed"
                  }
               }
            }
         }
      }
   }
}

Then added a couple of docs (notice that the structure differs slightly from yours):

PUT /test_index/doc/1
{
   "created_time": 1412988495000,
   "tags": [
      {
         "tag_type": "Placement",
         "tag_id": "id1"
      },
      {
         "tag_type": "Product",
         "tag_id": "id2"
      }
   ]
}

PUT /test_index/doc/2
{
   "created_time": 1412988475000,
   "tags": [
      {
         "tag_type": "Type3",
         "tag_id": "id3"
      },
      {
         "tag_type": "Type4",
         "tag_id": "id3"
      }
   ]
}

Now a scripted terms aggregation inside a nested aggregation seems to do the trick:

POST /test_index/_search?search_type=count
{
   "query": {
      "match_all": {}
   },
   "aggs": {
      "tags": {
         "nested": { "path": "tags" },
         "aggs":{
             "tag_vals": {
                 "terms": {
                     "script": "doc['tag_type'].value+':'+doc['tag_id'].value"
                 }
             }
         }
      }
   }
}
...
{
   "took": 3,
   "timed_out": false,
   "_shards": {
      "total": 1,
      "successful": 1,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "tags": {
         "doc_count": 4,
         "tag_vals": {
            "buckets": [
               {
                  "key": "Placement:id1",
                  "doc_count": 1
               },
               {
                  "key": "Product:id2",
                  "doc_count": 1
               },
               {
                  "key": "Type3:id3",
                  "doc_count": 1
               },
               {
                  "key": "Type4:id3",
                  "doc_count": 1
               }
            ]
         }
      }
   }
}

Here is the code I used:

http://sense.qbox.io/gist/4ceaf8693f85ff257c2fd0639ba62295f2e5e8c5

Sign up to request clarification or add additional context in comments.

1 Comment

Thank you, found a similar case here: stackoverflow.com/questions/27776428/…

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.