Elasticsearch aggregation on part of string, not full string

Question

Basically, what I'm trying to do here is get the second-level-down categories from a hierarchically stored string. The problem is that the level of hierarchy vary and one product category could have six levels and another only four, otherwise I would have just implemented predefined levels.

I have some products with categories like so:

[
  {
    title: 'product one',
    categories: [
      'clothing/mens/shoes/boots/steel-toe'
    ]
  },
  {
    title: 'product two',
    categories: [
      'clothing/womens/tops/sweaters/open-neck'
    ]
  },
  {
    title: 'product three',
    categories: [
      'clothing/kids/shoes/sneakers/light-up'
    ]
  },
  {
    title: 'product etc.',
    categories: [
      'clothing/baby/bibs/super-hero'
    ]
  }, 
  ... more products
]

I'm trying to get aggregation buckets like so:

buckets: [
  {
    key: 'clothing/mens',
    ...
  },
  {
    key: 'clothing/womens',
    ...
  },
  {
    key: 'clothing/kids',
    ...
  },
  {
    key: 'clothing/baby',
    ...
  },
]

I've tried looking at filter prefixes, includes and excludes on terms, but I can't find anything that works. Please someone point me in the right direction.

Andrei Stefan · Accepted Answer · 2015-06-20 00:50:59Z

3

Your category field should be analyzed with a custom analyzer. Maybe you have some other plans with the category, so I'll just add a subfield used only for aggregations:

{
  "settings": {
    "analysis": {
      "filter": {
        "category_trimming": {
          "type": "pattern_capture",
          "preserve_original": false,
          "patterns": [
            "(^\\w+\/\\w+)"
          ]
        }
      },
      "analyzer": {
        "my_analyzer": {
          "tokenizer": "keyword",
          "filter": [
            "category_trimming",
            "lowercase"
          ]
        }
      }
    }
  },
  "mappings": {
    "test": {
      "properties": {
        "category": {
          "type": "string",
          "fields": {
            "just_for_aggregations": {
              "type": "string",
              "analyzer": "my_analyzer"
            }
          }
        }
      }
    }
  }
}

Test data:

POST /index/test/_bulk
{"index":{}}
{"category": "clothing/womens/tops/sweaters/open-neck"}
{"index":{}}
{"category": "clothing/mens/shoes/boots/steel-toe"}
{"index":{}}
{"category": "clothing/kids/shoes/sneakers/light-up"}
{"index":{}}
{"category": "clothing/baby/bibs/super-hero"}

The query itself:

GET /index/test/_search?search_type=count
{
  "aggs": {
    "by_category": {
      "terms": {
        "field": "category.just_for_aggregations",
        "size": 10
      }
    }
  }
}

The results:

   "aggregations": {
      "by_category": {
         "doc_count_error_upper_bound": 0,
         "sum_other_doc_count": 0,
         "buckets": [
            {
               "key": "clothing/baby",
               "doc_count": 1
            },
            {
               "key": "clothing/kids",
               "doc_count": 1
            },
            {
               "key": "clothing/mens",
               "doc_count": 1
            },
            {
               "key": "clothing/womens",
               "doc_count": 1
            }
         ]
      }
   }

answered Jun 20, 2015 at 0:50

Andrei Stefan

52.5k6 gold badges102 silver badges92 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user1828780 Over a year ago

I was just looking at those and thought there might be an easier way, but that should work. Thank you sir!

user1828780 Over a year ago

Thanks for the answer Andrei, on second thought. It looks like the pattern will only go two layers deep. Is there a way to make it so I can aggregate any layer deep? You see, in one situation I might need to get only 'level1/level2' deep and on another I might need 'level1/level2/level3' or even 'level1/level2/level3/level4' deep.

Andrei Stefan Over a year ago

If you want any (and all) "paths" then take a look at the path hierarchy tokenizer.

Collectives™ on Stack Overflow

Elasticsearch aggregation on part of string, not full string

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related