Elasticsearch - aggregate and sort based on count and uniqueness

Question

I have a problem Im not sure how to solve.

Imagine i have some tweets from different users and would like to know which words are used the most for a specific user. To make it more interesting, Id like to boost the count by how unique the word is in the context of all tweets (from all users).

So, lets say that one user has tweeted the word "stackoverflow" four times in total and that "stackoverflow" has only been used by that user. The same user has also tweeted the words "I", "for" and "yes" several times more (say 10), but those words has also been tweeted by loads of other users plenty of times.

I'd like to find a query that would boost the count/score of "stackoverflow" higher than the count/score of, lets say, "for".

The elasticsearch type in this case would contain a string with the full tweet ("tweet"), an unanalyzed array with the words used in the tweet ("tweetedWords") and the name of the user ("user").

Any ideas?

micke239 · Accepted Answer · 2016-01-26 20:12:59Z

2

After finding the right google-keyword ("elasticsearch aggregate on frequency"), I found what I was looking for: significant_terms.

{
  "query" : {
    "terms": {"user": ["{user}"]}
  },
  "size": 0,
  "aggs": {
    "tweetedWords": {
      "significant_terms": {
        "field": "tweetedWords"
      }
    }
  }
}

answered Jan 26, 2016 at 20:12

micke239

354 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Elasticsearch - aggregate and sort based on count and uniqueness

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related