Concurrent writes to Elasticsearch Index using plainless script

Question

This is a followup question to (Is there a way to have field level audit in elastic search?)

Step 1:

#push a sample doc
PUT my_index/_doc/1
{
  "created_at": "2025-02-24T13:00:00Z",
  "email": "[email protected]",
  "name": "dogan",
  "field_update_timestamps": {
    "email": "2025-02-24T13:00:00Z",
    "name": "2025-02-24T13:00:00Z"
  }
}

Step 2:

#update the doc with `_update` API call
POST my_index/_update/1
{
  "script": {
    "source": """
      if (!ctx._source.containsKey('field_update_timestamps')) {ctx._source.field_update_timestamps = [:]; }
      for (entry in params.entrySet()) {
        ctx._source[entry.key] = entry.value; ctx._source.field_update_timestamps[entry.key] = 
        new java.text.SimpleDateFormat('yyyy-MM-dd\'T\'HH:mm:ss\'Z\'').format(new java.util.Date());
      }
    """,
    "lang": "painless",
    "params": {
      "name": "musab"
    }
  }
}

So can we update the same record in the index concurrently? I mean if we start adding different domain like attendance in office or worked from home. {"wfh": "", "wfo": "" concurrently. Please note no 2 separate threads will put the same fields concurrently ie there will be no another thread which will be updating "wfh" or "wfo", however there can be another thread writing "email". Will there be any optimistic lock exception or data loss?

Thanks

Cc: @Musab Dugan

Musab Dogan · Accepted Answer · 2025-04-28 07:37:23Z

0

EDIT: Check Optimistic concurrency control

Elasticsearch is distributed. When documents are created, updated, or deleted, the new version of the document has to be replicated to other nodes in the cluster. Elasticsearch is also asynchronous and concurrent, meaning that these replication requests are sent in parallel, and may arrive at their destination out of sequence. Elasticsearch needs a way of ensuring that an older version of a document never overwrites a newer version.

https://www.elastic.co/docs/reference/elasticsearch/rest-apis/optimistic-concurrency-control

Is the following example what you are looking for?

edited Apr 28 at 7:37

answered Apr 23 at 9:28

Musab Dogan

3,8611 gold badge8 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Spartacus Apr 28 at 0:02

thanks for the answer, is it not the same what mentioned in the question? I am more worried if ES will update correctly and dont loose data for concurrent updates on a given record in the index?

Musab Dogan Apr 28 at 7:35

I edited my answer with optimistic-concurrency-control information. please check it.

Spartacus Apr 28 at 7:52

are you saying to get the record using search api, then use the _seq_no and _primary_term to update the index? If that is the case then there is no need to have the painless script, as I have both the stored data and the incoming payload to compare and act accordingly.

Musab Dogan Apr 29 at 8:25

Yes, use the _seq_no and _primary_term no is the on of the method to make sure data is not changed. If you no need to have the painless script it's even better

Collectives™ on Stack Overflow

Concurrent writes to Elasticsearch Index using plainless script

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related