elasticsearch must_not returning wrong values

Question

I'm trying to figure out what I'm doing wrong with my Elastic query. I'm trying to filter out all the docs that have "Software Engineer" as their title. my Query:

{
  "from": 0,
  "size": 20,
  "query": {
    "bool": {
      "must": [{
        "bool": {
          "must_not": [{
            "term": {
              "title.keyword": "Software Engineer"
            }
          }]
        }
      }]
    }
  }
}

in my mapping...

"title": {"type": "text"}

Then my results:

hits:[
{title: "Software Engineer"},
{title: "Engineer"},
{title: "Software Engineer"},
{title: "Software and Data Quality Manager"},
...
]

I would like to not get Software Engineer in my search results here. Any help would be greatly appreciated!

Bhavya · Accepted Answer · 2021-02-05 14:19:58Z

2

You need to use multi-fields if you want to map the title field as text and keyword type. Modify your index mapping to

{
  "mappings": {
    "properties": {
      "title": {
        "type": "text",
        "fields": {
          "keyword": { 
            "type":  "keyword"
          }
        }
      }
    }
  }
}

Index Data:

{
  "title": "Software and Data Quality Manager"
}
{
  "title": "Software Engineer"
}
{
  "title": "Engineer"
}

Search Query:

{
  "from": 0,
  "size": 20,
  "query": {
    "bool": {
      "must_not": {
        "term": {
          "title.keyword": "Software Engineer"
        }
      }
    }
  }
}

Search Result:

"hits": [
      {
        "_index": "66064654",
        "_type": "_doc",
        "_id": "2",
        "_score": 0.0,
        "_source": {
          "title": "Engineer"
        }
      },
      {
        "_index": "66064654",
        "_type": "_doc",
        "_id": "3",
        "_score": 0.0,
        "_source": {
          "title": "Software and Data Quality Manager"
        }
      }
    ]

edited Feb 5, 2021 at 14:19

answered Feb 5, 2021 at 14:14

Bhavya

16.2k3 gold badges23 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Ben Over a year ago

I've made the change in my mapping, and I'm still getting the same results. I imagine this is because there is some propagation happening under the hood where my index is being eventually updated to have this field (some 60k docs in it) Does that sound reasonable? Your results seem indisputable, so i'm trying to figure out where i'm still going wrong.

Amit Over a year ago

@Ben but it looks like you already have title.keyword field, as shown in the query posted in your question.

Junaid Over a year ago

Apparently, it appears to be the case that you have text mappings defined on the field over which you are doing search. So it would be better to confirm that first.

Kaveh · Accepted Answer · 2021-02-07 16:03:36Z

1

You query is correct and keyword by default is no operation tokenizer, so please check to be sure you are not using any type of normalizer for your keywrod type in index and search time that causing change to your text. If you use default keyword also be aware that your search will be treated case sensitive except that you use normalizer during searech and index.

answered Feb 7, 2021 at 16:03

Kaveh

1,31010 silver badges18 bronze badges

Collectives™ on Stack Overflow

elasticsearch must_not returning wrong values

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related