Remove an object from array in ElasticSearch based on a query

Question

I want to remove an object from a nested structure in elastic doc, This is how my elastic doc looks like in the index 'submissions'. Based on the conditions I want to remove an object from all the documents.

{
  "took": 21,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 11,
    "max_score": 1,
    "hits": [
      {
        "_index": "submissions",
        "_type": "_doc",
        "_id": "15_12069",
        "_score": 1,
        "_source": {
          "id": "15_12069",
          "account_id": 2,
          "survey_id": 15,
          "submission_id": 12069,
          "answers": [
            {
              "question_id": 142,     //
              "skipped": false,       //<------ remove object with question_id: 142
              "answer_txt": "product" //
            },
            {
              "question_id": 153,
              "skipped": false,
              "answer_txt": "happy"
            }
          ]
        }
      },
      {
        "_index": "submissions",
        "_type": "_doc",
        "_id": "15_12073",
        "_score": 1,
        "_source": {
          "id": "15_12073",
          "account_id": 2,
          "survey_id": 15,
          "submission_id": 12073,
          "answers": [
            {
              "question_id": 142,       //
              "skipped": false,         //<------ remove object with question_id: 142
              "answer_txt": "coherent"  //
            },
            {
              "question_id": 153,
              "skipped": false,
              "answer_txt": "cool"
            }
          ]
        }
      }
    ]
  }
}

I wanted to try updateByQuery api ( _update_by_query ) and ctx._source.remove with query

{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "must": [
              {
                "match": {
                  "account_id": 2
                }
              },
              {
                "match": {
                  "survey_id": 15
                }
              }
            ]
          }
        },
        {
          "nested": {
            "path": "answers",
            "query": {
              "bool": {
                "must": [
                  {
                    "match": {
                      "answers.question_id": 142
                    }
                  }
                ]
              }
            }
          }
        }
      ]
    }
  }
}

Any insight on this or do i have a better approach?

Bhavya · Accepted Answer · 2020-11-18 18:11:00Z

You can use Update By Query API, in the following way

Adding a working example with index data, mapping, and query

Index Mapping:

{
  "mappings": {
    "properties": {
      "answers": {
        "type": "nested"
      }
    }
  }
}

Index Data:

    {
  "id": "15_12069",
  "account_id": 2,
  "survey_id": 15,
  "submission_id": 12069,
  "answers": [
    {
      "question_id": 142, 
      "skipped": false, 
      "answer_txt": "product" 
    },
    {
      "question_id": 153,
      "skipped": false,
      "answer_txt": "happy"
    }
  ]
}
{
      "id": "15_12073",
      "account_id": 2,
      "survey_id": 16,
      "submission_id": 12073,
      "answers": [
        {
          "question_id": 142,
          "skipped": false,
          "answer_txt": "coherent"
        },
        {
          "question_id": 153,
          "skipped": false,
          "answer_txt": "cool"
        }
      ]
    }

Query:

  POST /index/_update_by_query
{
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "must": [
              {
                "match": {
                  "account_id": 2
                }
              },
              {
                "match": {
                  "survey_id": 15
                }
              }
            ]
          }
        },
        {
          "nested": {
            "path": "answers",
            "query": {
              "bool": {
                "must": [
                  {
                    "match": {
                      "answers.question_id": 142
                    }
                  }
                ]
              }
            }
          }
        }
      ]
    }
  },
  "script": {
    "source": "ctx._source.answers.removeIf(question_id -> question_id.question_id == params.remove_id);",
    "params": {
      "remove_id": 142
    }
  }
}

After performing the above query, the document which satisfies all the conditions of the query i.e. "account_id": 2 AND "survey_id": 15 AND "answers.question_id": 142, from that document object with question_id: 142 is removed.

Therefore, from the first document (as indexed above), the document containing "answers.question_id": 142 is removed, and now the document contains the below data (after running the query)

{
  "_index": "64898361",
  "_type": "_doc",
  "_id": "1",
  "_version": 8,
  "_seq_no": 13,
  "_primary_term": 1,
  "found": true,
  "_source": {
    "survey_id": 15,
    "submission_id": 12069,
    "account_id": 2,
    "answers": [
      {
        "answer_txt": "happy",
        "question_id": 153,
        "skipped": false
      }
    ],
    "id": "15_12069"
  }
}

There will be no change in the second document, as it does not satisfy all the query conditions.

Collectives™ on Stack Overflow

Remove an object from array in ElasticSearch based on a query

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related