Replace document in Elasticsearch index with field condition

Question

If I have indexed a document in Elasticsearch that contains a datetime parameter, or some kind of sequence number, can I update/replace the entire document with a new version if, and only if, the value in my new document is greater than that in the currently indexed document?

Searching has shown me so far how I can affect the values of specific fields through scripting, but I'm not sure if I can use a script or operation as an update criterion, and replace the whole document if it's met.

To be more specific, we have a document object that contains a timestamp of when it was placed on the queue for processing, and since we may have multiple processors pulling things off the queue we would like to ensure that we only index documents newer than the one we already have in the index, discarding any old changes.

Assael Azran · Accepted Answer · 2019-12-17 08:27:30Z

1

Try to use the _update_by_query Api.

Update By Query

Example:

Mappings

PUT my_index
{
  "mappings": {
    "properties": {
      "user": {
        "type": "keyword"
      },
      "timestamp": {
        "type": "keyword"
      }
    }
  }
}

Indexing documents

POST my_index/_doc/1
{
  "user":"user1",
  "timestamp":1234
}

POST my_index/_doc/2
{
  "user":"user2",
  "timestamp":1235
}

Update By Query

Let's update only documents with timestamp greater than 1234.

POST /my_index/_update_by_query
{
  "script": {
    "source": "ctx._source.user='new user';", ----> updating field user
    "lang": "painless"
  },
  "query": {
    "range": {
      "timestamp": {
        "gt": 1234
      }
    }
  }
}

You can update other fields or insert new ones, just play with "source": "ctx._source.user='new user';ctx._source.timestamp=456";ctx._source.new_field=value"

Results

 {
    "_index": "my_index",
    "_type": "_doc",
    "_id": "2",
    "_score": 1,
    "_source": {
      "user": "new user",
      "timestamp": 1235
    }
  }

Hope this helps

answered Dec 17, 2019 at 8:27

Assael Azran

2,9931 gold badge12 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Assael Azran Over a year ago

@Adrian Is that what you were looking for?

Adrian Over a year ago

It's part-way there. I did find information about how to update individual properties like this using scripts. What I'm trying to do is replace the entire document - the document itself is kind of complex, having something like 40 properties with multiple levels of nesting, one of which is a list. While I could technically script that, if I can replace the entire document in one fell swoop that would be preferable.

Collectives™ on Stack Overflow

Replace document in Elasticsearch index with field condition

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related