Elasticsearch Groovy Script Syntax for generation of nested fields

Question

The below simple ES Groovy script takes a python dictionary with order data (orderItem) and appends it to a list of orders within Elasticsearch. The list with all orderItems then resides under _source.Orders

"script": "if (ctx._source.containsKey(\"Orders\")) {ctx._source.Orders += orderItem;} else {ctx._source.Orders = [orderItem]}; "
"params":{"orderItem": orderItem}

In my use case, orders come from different shops and want them to go in to a list structure under _source.Orders.Shop5Hgk, _source.Orders.Shop86hG, _source.Orders.Shop5G60, etc. The shop names are dynamic.

No matter, what I try, ES throws exceptions complaining that Orders obviously is null.

GroovyScriptExecutionException[NullPointerException[Cannot set property 'Shop5Hgk' on null object]

So, what is the right groovy syntax to create the Orders field first, and then the field for the shop name and then append orderItems to that?

Update: Full python function with (not working)

def updateLastOrdersElasticsearch(self,data):

    es = elasticsearch.Elasticsearch(timeout=500)
    actions = []

    for shopName,orderList in data.items():
        for orderItem in orderList:
            sku = orderItem['SKU']
            action = {
                "_index": "myindex",
                "script": "if (ctx._source.containsKey(\"Orders\")) {if (ctx._source.containsKey(shopName)){ctx._source.Orders."+shopName+" += Orders;}} else {ctx._source.Orders = []; ctx._source.Orders."+shopName+" = [Orders]}; ctx._source.TimestampUpdated = TimestampUpdated",
                "_type": "items",
                '_op_type': 'update',
                "_id": sku,
                "params":{"shopName":shopName,"Orders": orderItem, "TimestampUpdated":datetime.now().isoformat()}
                }
            actions.append(action)
    return helpers.bulk(es, actions)

Val · Accepted Answer · 2015-12-30 05:42:02Z

1

I think that initially your _source.Orders field is null, i.e. not even an empty array.

Moreover, containsKey might not be the right way to go, because your _source might contain a field named Orders whose type might not be an array, i.e. it might be a dynamic object standing for an existing order, or worse, just a plain string.

I suggest you try a different approach by first checking if Orders is null and initialize it to an empty array if not. Then you can append the orderItem to the resulting array:

{
  "script" : "ctx._source.Orders = ((ctx._source.Orders ?: []) += orderItem)",
  "params" : {
    "orderItem" : orderItem
  }      
}

An alternative to this would be to simply ensure that when you index your document the first time, you make sure that the Orders field is initialized with an empty array [] and then your script could simply append orderItems to that array.

UPDATE

Based on your comments, I'm revising my answer in order to deal with the case where Orders is a dynamic object containing shop names as keys and each of those keys points to an array of orders for that shop. It's basically the same idea as earlier, just that we need to deal with one more level (i.e. the shop names).

First the script makes sure that the Orders object exists and then it makes sure that the shop array within the Orders object exists as well. All that remains to do is to append the orderItem to the shop array:

{
  "script" : "ctx._source.Orders = ctx._source.Orders ?: [shopName:'']; ctx._source.Orders[shopName] = ((ctx._source.Orders[shopName] ?: []) + orderItem); ctx._source.TimestampUpdated = TimestampUpdated",
  "params" : {
    "shopName": shopName,
    "orderItem" : orderItem,
    "TimestampUpdated":datetime.now().isoformat()
  }      
}

edited Dec 30, 2015 at 5:42

answered Dec 29, 2015 at 14:06

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Jabb Over a year ago

your solution refers to the (working) code snipped at the beginning of the question. I am in search for a solution, that allows to additionally add shop names to Orders and append the orderItems "shop names" lists.

Val Over a year ago

Ok that's a bit different than your initial question statement. So Orders is not an array but a dynamic object whose keys are shop names and each shop name holds an array of orders, right?

Val Over a year ago

Ok, can you at least ensure that Orders is initialized to an empty list at first, so that we don't need to take care of that within the script?

Jabb Over a year ago

tried to do that in my updated python code example. See above

Val Over a year ago

No, I mean when you first create your document in Elasticsearch, i.e. when there are no Orders at all. Can you make sure that Orders: {} when your create the document?

|

Collectives™ on Stack Overflow

Elasticsearch Groovy Script Syntax for generation of nested fields

1 Answer 1

7 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related