ElasticSearch - Bulk update on a nested list object fields

Question

So I had a scenario where I had a lot of data on which I had to update one field in a nested list object. So I was using bulk update and I am able to update the fields in _source. This is how the data looks like and how I was able to change the documentType.

"_source" : {
  "accountId" : "7ef72993-a068-4c1f-89ca-f12183a78517",
  "uuid" : "abdefc35-14c8-40a9-b6cf-203b4462934a",
  "documentType" : "pdf",
  "details" : [
    {
      "name" : "abhi",
      "age" : 21
    }
  ]
}

And the way I was able to bulk update documentType is:

POST _bulk
{ "update" : {"_id" : "abdefc35-14c8-40a9-b6cf-203b4462934a",  "_index" : "model_index"} }
{ "doc" : {"documentType" : "jpg"}}

I have shown only one example here but I had to run this on 10K docs, but what now I want to do is update the details field name attribute. So one thing is that the details field is a list which always contains one element. (Not sure why it was designed like this, might be for future enhancements).

Now coming to the point where I am stuck is that I am not able to parse through that details list. What I tried doesn't help if I specify zeroth index like details[0] then it creates a new field as details[0] with that value and if I try just with the name field then it updates the whole details object with that field like this:

POST _bulk
{ "update" : {"_id" : "abdefc35-14c8-40a9-b6cf-203b4462934a",  "_index" : "model_index"} }
{ "doc" : {"details" : [{"name": "abhi"}]}}

-------------------------------------------------

"_source" : {
  "accountId" : "7ef72993-a068-4c1f-89ca-f12183a78517",
  "uuid" : "abdefc35-14c8-40a9-b6cf-203b4462934a",
  "documentType" : "jpg",
  "details" : [
    {
     "name" : "abhi"
    }
  ]
}

So now how can I update the zeroth index of details list name attribute using bulk update. Even if there is another way how I can update different documents with different names then it will be appreciated.

Jozef - Spatialized.io · Accepted Answer · 2023-04-28 00:07:50Z

2

What you need is a script within your _bulk call. But since you cannot have both doc and script in one call, you can do both replacements from within the script.

Extended (more readable) form:

POST _bulk
{
  "update": {
    "_id": "abdefc35-14c8-40a9-b6cf-203b4462934a",
    "_index": "model_index"
  }
}
{
  "script": {
    "source": """
      if (params.documentType != null) {
        ctx._source.documentType = params.documentType;
      } else if (params.name != null) {
        ctx._source.details[0].name = params.name
      }
    """,
    "params": {
      "name": "abhi_1400",
      "documentType": "jpg"
    }
  }
}

Compact (syntactically valid) form:

POST _bulk
{"update":{"_id":"abdefc35-14c8-40a9-b6cf-203b4462934a","_index":"model_index"}}
{"script":{"source":"if (params.documentType != null) {ctx._source.documentType = params.documentType} else if (params.name != null) { ctx._source.details[0].name = params.name }","params":{"name":"abhi_1400","documentType":"jpg"}}}

edited Apr 28, 2023 at 0:07

answered Dec 2, 2020 at 21:36

Jozef - Spatialized.io

17k4 gold badges29 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Abhishek Jain Over a year ago

Thanks, Joe for the quick answer. Yeah, so I was able to get it working with the script along with a for loop as details always had only zeroth index as of now.

Collectives™ on Stack Overflow

ElasticSearch - Bulk update on a nested list object fields

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related