Scope 0 count terms in aggregation in ElasticSearch

Question

i am doing aggregations on "location" field in my document ,where there is also a "city" field in the same document.I am querying the document on city field and aggregating the documents on location field.

{
  "aggs": {
    "locations": {
      "terms": {
        "field": "location",
        "min_doc_count": 0
      }
    }
  },
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "city": "mumbai",
                "_cache": true
              }
            }
          ]
        }
      }
    }
  }
}

Now the count and aggregations come fine and along with the hits.but my problem is that i want to do aggregation with 'doc-count' set to 0 and the aggregation bucket returns me all the lcoations with 0 count which even falls in other city.I want to get 0 count locations only for that city.want to scope the context of 0 count location to city. I tried achieving this by nested aggregation placing location inside nested city and then doing aggs, or combining the filter aggs with terms agg but still getting the same result.Is there any way to achieve this or elasticsearch is inherently build to work like this. ES Version - 1.6

My mapping looks like this:

{
  "service": {
    "_source": {
      "enabled": true
    },
    "properties": {
      "name": {
        "type": "string",
        "index": "not_analyzed"
      },
      "location": {
        "type": "string",
        "index": "not_analyzed"
      },
      "city": {
        "type": "string",
        "index": "not_analyzed"
      }
    }
  }
}

Sample docs to index

{ "name": "a", "location": "x", "city": "mumbai" }

{ "name": "b", "location": "x", "city": "mumbai" }

{ "name": "c", "location": "y" "city": "chennai" }

yeah, just read the note on ES documentation for this, ES is build to work like this.Is there anybody out able to hack this trick... — user3775217
– user3775217, Commented Oct 2, 2015 at 16:29
Please show your current query and eventually the mapping you're using as well. — Val
– Val, Commented Oct 3, 2015 at 3:20
mappings: "service" :{ "_source" : {"enabled" : true }, "properties":{ "name" : {"type" : "string", "index" : "not_analyzed"}, "location" : {"type" : "string", "index" : "not_analyzed"}, "city" : {"type" : "string", "index" : "not_analyzed"} — user3775217
– user3775217, Commented Oct 3, 2015 at 5:30
Query : { "aggs": { "locations": { "terms": { "field": "location", "min_doc_count": 0, } } }, "query": { "filtered": { "filter": { "bool": { "must": [ { "term": { "city": "mumbai", "_cache": true } } ] } } } } } @Val — user3775217
– user3775217, Commented Oct 3, 2015 at 5:40

Val · Accepted Answer · 2015-10-06 06:43:45Z

1

You should try to sort your terms aggregation (embedded into a filter aggregation) by ascending doc count and you'll get all the terms with 0 doc count first. Note that by default, you'll only get the first 10 terms, if you have less terms with 0 doc count, you'll see them all, otherwise you might need to increase the size parameter to something higher than 10.

{
  "aggs": {
    "city_filter": {
      "filter": {
        "term": {
          "city": "mumbai"
        }
      },
      "aggs": {
        "locations": {
          "terms": {
            "field": "location",
            "min_doc_count": 0,
            "size": 20,         <----- add this if you have more than ten 0-doc-count terms
            "order": {          <----- add this to see 0-doc-count first
              "_count": "asc"
            }
          }
        }
      }
    }
  },
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "city": "mumbai",
                "_cache": true
              }
            }
          ]
        }
      }
    }
  }
}

edited Oct 6, 2015 at 6:43

answered Oct 5, 2015 at 6:01

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user3775217 Over a year ago

Thanks for replying.But i want to get aggs with 0 count that matches my filter scope of the query. Like i am filtering by city(=mumbai).I also get 0 count terms in the aggs who have city value(other than mumbai). Elastic mentioned on their site that it is supposed to work like this.Your answer gives me a way to think to restrict the size. but that will be a very messy solution with other filters added as they need to be customized everytime my filters change.Any way to go from here? @Val

Val Over a year ago

Hmm, the default behavior is that the aggregation works in the context of the filtered documents, so in your case, the aggregation buckets should only contain city terms that are matched by the filter. Can you show an example of the results you get and some sample documents you have?

user3775217 Over a year ago

added some sample docs index them and then run the same query that i shared with min_count = 0, you will still get 'chennai' in location with count= 0.

Val Over a year ago

I see what you mean now. I've modified my query to make it work the way you like.

user3775217 Over a year ago

not this is not what i am looking for, aggregations work on field data. its a hack. i need to find a way to filter field data of the index based on city and then run aggregator over the filtered field data....Thanks will share the solution if found.

Collectives™ on Stack Overflow

Scope 0 count terms in aggregation in ElasticSearch

1 Answer 1

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related