5

I have documents like:

{
  MyProp: ["lorem", "ipsum", "dolor"]
  ... lots of stuff here ...
}

My documents can be quite big (but these MyProp fields are not), and expensive to generate from scratch.

Sometimes I need to update batches of these - it would therefore be beneficial to do a partial update (to save "indexing client" processing power and bandwidth, and thus time) and replace the MyProp values with new values.

Example of original document:

{
  MyProp: ["lorem", "ipsum", "dolor"]
  ... lots of stuff here ...
}

Example of updated document (or rather how it should look):

{
  MyProp: ["dolor", "sit"]
  ... lots of stuff here ...
}

From what I have seen, this includes scripting.

Can anyone enlighten me with the remaining bits of the puzzle?

Bounty added:

I'd like to also have some instructions of how to make these in a batch statement, if possible.

6
  • Can you show one update you'd like to make? Commented Nov 27, 2016 at 5:20
  • @Val, done! Thanks Commented Nov 28, 2016 at 0:21
  • Are you willing to update MyProp in a single document at a time or in several documents? Commented Nov 28, 2016 at 5:09
  • Most of the time, they need unique values. Commented Nov 29, 2016 at 10:22
  • What Elasticsearch version? Commented Nov 29, 2016 at 10:31

2 Answers 2

4
+400

You can use the update by query API in order to do batch updates. This works since ES 2.3 onwards, otherwise you need to install a plugin.

POST index/_update_by_query
{
  "script": {
    "inline": "ctx._source.myProp += newProp",
    "params": {
      "newProp": "sit"
    }
  },
  "query": {
    "match_all": {}
  }
}

You can of course use whatever query you want in order to select the documents on which MyProp needs to be updated. For instance, you could have a query to select documents having some specific MyProp values to be replaced.

The above will only add a new value to the existing array. If you need to completely replace the MyProp array, then you can also change the script to this:

POST index/_update_by_query
{
  "script": {
    "inline": "ctx._source.myProp = newProps",
    "params": {
      "newProps": ["dolor", "sit"]
    }
  },
  "query": {
    "match_all": {}
  }
}

Note that you also need to enable dynamic scripting in order for this to work.

UPDATE

If you simply want to update a single document you can use the partial document update API, like this:

POST test/type1/1/_update
{
    "doc" : {
        "MyProp" : ["dolor", "sit"]
    }
}

This will effectively replace the MyProp array in the specified document.

If you want to go the bulk route, you don't need scripting to achieve what you want:

POST index/type/_bulk
{ "update" : {"_id" : "1"} }
{ "doc" : {"MyProp" : ["dolor", "sit"] } }
{ "update" : {"_id" : "2"} }
{ "doc" : {"MyProp" : ["dolor", "sit"] } }
Sign up to request clarification or add additional context in comments.

8 Comments

Will this effectively clear the existing values though?
Sorry to bother you. Can you also give an example on how to do this in non-batch mode? (for clarity)
Sorry again. For the single document partial update. Are you positive this would replace the array and clear existing values?
Yes that will completely replace the array and clear the previous values.
So just issuing a bulk with partial api updates would do the thing? I'm not sure that your first example applies as I never push new values to existing values, only replacement. I understood the Docs as that they were adding new values without deleting the old ones.
|
0

Would a _bulk update work for you?

POST test/type1/_bulk
{"update":{"_id":1}}
{"script":{"inline":"ctx._source.MyProp += new_param","params":{"new_param":"bla"},"lang":"groovy"}}
{"update":{"_id":2}}
{"script":{"inline":"ctx._source.MyProp += new_param","params":{"new_param":"bla"},"lang":"groovy"}}
{"update":{"_id":3}}
{"script":{"inline":"ctx._source.MyProp += new_param","params":{"new_param":"bla"},"lang":"groovy"}}
....

And you would also need to enable inline scripting for groovy. What the above would do is to add a bla value to the listed documents in MyProp field. Of course, depending on your requirements many other changes can be performed in that script.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.