ElasticSearch array data match multiple properties in nested element with AND condition

Question

I'm facing a problem where I have two documents each containing an array of objects. I like to search for one document containing two properties for a nested object (matching both at the same time in the same object) but I always get both documents.

I created the documents with:

POST /respondereval/_doc
{
  "resp_id": "1236",
  "responses": [
     {"key": "meta","text":"abc"},
     {"key": "property 1", "text": "yes"},
     {"key": "property 2", "text": "yes"},
  ]
}

POST /respondereval/_doc
{
  "resp_id": "1237",
  "responses": [
     {"key": "meta","text":"abc"},
     {"key": "property 1", "text": "no"},
     {"key": "property 2", "text": "yes"},
  ]
}

I defined an index for them to prevent ES to flat out the objects like this:

PUT /respondereval
{
  "mappings" : {
    "properties": {
      "responses" : {
        "type": "nested"
      }
    }
  }
}

I now like to search for the first document (resp_id 1236) with the following query:

GET /respondereval/_search
{
  "query": {
    "nested": {
      "path": "responses",
      "query": {
        "bool": {
          "must": [
            { "match": { "responses.key": "property 1" } },
            { "match": { "responses.text": "yes" } }
          ]
        }
      }
    }
  }
}

This should only return one element which matches both conditions at the same time.

Unfortunatly, it always returns both documents. I assume it's because at some point, ES still flattens the values in the nested objects arrays into something like this (simplified):

resp_id 1236: "key":["gender", "property 1", "property 2"], "text:["abc", "yes", "yes"]
resp_id 1237: "key":["gender", "property 1", "property 2"], "text:["abc", "no", "yes"]

which both contain the property1 and yes.

What is the correct way to solve this so that only documents are returned which contains an element in the objects array which matches both conditions ("key": "property 1" AND "text": "yes") at the same time?

Gibbs · Accepted Answer · 2020-08-03 18:00:17Z

2

The problem is with your mapping. You have text mapping which uses standard analyser by default.

Standard analyzer creates tokens on whitespaces. So

property 1 will be tokenised as

{
    "tokens": [
        {
            "token": "property",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0
        },
        {
            "token": "1",
            "start_offset": 9,
            "end_offset": 10,
            "type": "<NUM>",
            "position": 1
        }
    ]
}

Similarly property 2 also.

Hence both the documents are returned.

And when you search for yes, it matched from second text in the second document. property 1 matches property analysed token of second key in the document.

To make it work: - use keyword variation

{
  "query": {
    "nested": {
      "path": "responses",
      "query": {
        "bool": {
          "must": [
            { "match": { "responses.key.keyword": "property 1" } },
            { "match": { "responses.text.keyword": "yes" } }
          ]
        }
      }
    }
  }
}

It would be proper:

{
  "query": {
    "nested": {
      "path": "responses",
      "query": {
        "bool": {
          "must": [
            { "match_phrase": { "responses.key": "property 1" } },//phrase queries
            { "match": { "responses.text": "yes" } }
          ]
        }
      }
    }
  }
}

edited Aug 3, 2020 at 18:00

answered Aug 3, 2020 at 17:36

Gibbs

23.1k14 gold badges84 silver badges147 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Alex Over a year ago

Thanks for the response +1. Your keyword example works as expected BUT what would be the correct way to resolve the mapping problem so that I can use the responses.key without the keyword?

Alex Over a year ago

This makes sense. Thanks a lot.

Prathap Reddy · Accepted Answer · 2020-08-03 17:35:46Z

0

Have you directly tried the must query without nested.path

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "responses.key": "property 1"
          }
        },
        {
          "match": {
            "responses.text": "yes"
          }
        }
      ]
    }
  }
}

answered Aug 3, 2020 at 17:35

Prathap Reddy

1,8492 gold badges8 silver badges18 bronze badges

2 Comments

Prathap Reddy Over a year ago

I thought it's similar to this answer as per my knowledge. Isn't it similar to the above question?

Alex Over a year ago

It does not yield any results. @Gibbs answer is correct.

Collectives™ on Stack Overflow

ElasticSearch array data match multiple properties in nested element with AND condition

2 Answers 2

2 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related