Elasticsearch: Append new elements to nested array of objects

Question

I'm trying to add new items to a nested array of objects from a specific document. I have searched and it seems updates with a partial document doesn't support what I need, it replaces the entire array with the new elements. So I went for a scripted update, which worked as expected via REST API:

PUT /transactions
{
    "mappings": {
        "_doc": {
            "properties": {
                "userId": { "type": "keyword" },
                "transactions": {
                    "type": "object",
                    "properties": {
                        "date": { "type": "date" },
                        "amount": { "type": "double" }
                    }
                }
            }
        }
    }
}

POST /transactions/_doc/1
{
    "userId": "123",
    "transactions": [
        { "date": "2019-07-15T10:32:02Z", "amount": 122 },
        { "date": "2019-07-17T22:09:43Z", "amount": 560 }
    ]
}

POST /transactions/_doc/1/_update
{
    "script": {
        "source": "ctx._source.transactions.addAll(params.transactions)",
        "params": {
            "transactions": [
                { "date": "2019-07-14T21:10:22Z", "amount": 890 },
                { "date": "2019-07-15T15:56:18Z", "amount": 54 }
            ]
        }
    }
}

Then I took the same script to my Java application, this is what the code looks like:

List<Transaction> transactions = Collections.singletonList(new Transaction(320));

Script script = new Script(
    Script.DEFAULT_SCRIPT_TYPE, Script.DEFAULT_SCRIPT_LANG,
    "ctx._source.transactions.addAll(params.transactions);",
    Collections.singletonMap("transactions", transactions));

transportClient
    .prepareUpdate("transactions", "_doc", 1)
    .setFetchSource(false);
    .setScript(script);
    .get();

When the code above is executed, I get the following remote exception:

Caused by: java.io.IOException: can not write type [class com.example.model.Transaction]
    at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:713)
    at org.elasticsearch.common.io.stream.StreamOutput.lambda$static$9(StreamOutput.java:599)
    at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:711)
    at org.elasticsearch.common.io.stream.StreamOutput.lambda$static$11(StreamOutput.java:621)
    at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:711)
    at org.elasticsearch.common.io.stream.StreamOutput.writeMap(StreamOutput.java:494)
    at org.elasticsearch.script.Script.writeTo(Script.java:533)
    ...

If I serialize the list into a string I get this exception instead:

org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper: class_cast_exception: Cannot cast java.lang.String to java.util.Collection

Finally, my question is how can I accomplish this with the Java API (Transport Client, Elasticsearch 6.3.2)?

One alternative is to fetch the whole document, deserialize it, append a new transaction and then update the whole document, but it seems overkill and probably degrades the performance.

What version of ES are you using?

Arpan Kanthal
– Arpan Kanthal

2019-07-18 13:56:17 +00:00
Commented Jul 18, 2019 at 13:56 — Arpan Kanthal
– Arpan Kanthal, Commented Jul 18, 2019 at 13:56
As stated in the question, 6.3.2.

AndreLDM
– AndreLDM

2019-07-18 17:33:25 +00:00
Commented Jul 18, 2019 at 17:33 — AndreLDM
– AndreLDM, Commented Jul 18, 2019 at 17:33
Sorry, my bad. Slipped by me. Dows the answer help though?

Arpan Kanthal
– Arpan Kanthal

2019-07-18 17:49:18 +00:00
Commented Jul 18, 2019 at 17:49 — Arpan Kanthal
– Arpan Kanthal, Commented Jul 18, 2019 at 17:49
Yes, it worked, thank you!

AndreLDM
– AndreLDM

2019-07-18 18:09:17 +00:00
Commented Jul 18, 2019 at 18:09 — AndreLDM
– AndreLDM, Commented Jul 18, 2019 at 18:09
Awesome... Glad I could help

Arpan Kanthal
– Arpan Kanthal

2019-07-18 18:28:30 +00:00
Commented Jul 18, 2019 at 18:28 — Arpan Kanthal
– Arpan Kanthal, Commented Jul 18, 2019 at 18:28

AndreLDM · Accepted Answer · 2019-07-18 19:18:48Z

2

I just had a curiosity look at the source code, from the writeGenericValue method, it looks like ES doesn't support writes of any custom objects except the ones handled in the multiple if else statements (Object[], List, Map, ReadableInstant, BytesReference).

From the stack trace it looks like script.writeTo is trying to write Transaction objects into the output stream and failing to do so. Converting the Transaction objects to individual Maps and then sending it might solve the problem.

edited Jul 18, 2019 at 19:18

AndreLDM

2,2281 gold badge20 silver badges27 bronze badges

answered Jul 18, 2019 at 14:17

Arpan Kanthal

5032 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Elasticsearch: Append new elements to nested array of objects

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related