Bulk update with Python's elasticsearch client

Question

I'm attempting to do a bulk update based on state change on a document property. Create works fine but bulk is freaking out. I'm getting an error to the effect of "script or doc is missing" but everything looks good.

Here is how I am attempting the bulk update:

frequency_cleared = [
    {
        "_id": result['_id'], 
        "_type": "the-type", 
        "_index": "the-index", 
        "_source": result['_source'],
        "_op_type": 'update'
    } 
    for result in search_results['hits']['hits']
]

The reason I'm iterating over my results is that I use an if in my list comprehension but since I'm able to see the results I get back I know that isn't the issue. I can't show results and had to change property names since this is for the company I work at.

Here is the traceback:

Elasticsearch.exceptions.RequestError: 
TransportError(400, 'action_request_validation_exception',
  'Validation Failed: 1: script or doc is missing...')

The ellipses represent it showing the same error for every element in the list fails.

Steve Tjoa · Accepted Answer · 2018-04-05 16:38:39Z

27

It was difficult to tell based on the docs but I found out the issue. If you want to do a bulk update you need to wrap your source in a dictionary with the key being "doc". Here is the correct example, hope this helps!

frequency_cleared = [
    {
        '_id': result['_id'], 
        "_type": "the-type", 
        "_index": "the-index", 
        "_source": {'doc': result['_source']}, 
        '_op_type': 'update'
    } 
    for result in search_results['hits']['hits']
]

Notice the slight change is "_source" to {'doc': result['_source']}

edited Apr 5, 2018 at 16:38

Steve Tjoa

61.5k18 gold badges93 silver badges103 bronze badges

answered Feb 3, 2016 at 17:47

Obj3ctiv3_C_88

1,5183 gold badges17 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

funseiki Over a year ago

This is useful - their python docs appear to be inaccurate. There, 'doc' is not wrapped inside a '_source' object.

Collectives™ on Stack Overflow

Bulk update with Python's elasticsearch client

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related