Elasticsearch update document in python

Question

I am trying to update document in elasticsearch python,

from elasticsearch import Elasticsearch
from datetime import datetime
from scapy.all import *
import json
# es = Elasticsearch()
es = Elasticsearch(['http://localhost:9200'])

doc = getDoc("1")
    print doc
    if doc != None:
      doc['_op_type'] = 'update'
      doc['_source']['macList'].append('new')
      helpers.bulk(es, doc, stats_only=False)

But this is not working. Can someone please tell me what I am doing wrong ? thanks

This is the error:

Traceback (most recent call last):
  File "./req.py", line 48, in <module>
    helpers.bulk(es, doc, stats_only=False)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/helpers/__init__.py", line 188, in bulk
    for ok, item in streaming_bulk(client, actions, **kwargs):
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/helpers/__init__.py", line 160, in streaming_bulk
    for result in _process_bulk_chunk(client, bulk_actions, raise_on_exception, raise_on_error, **kwargs):
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/helpers/__init__.py", line 89, in _process_bulk_chunk
    raise e
elasticsearch.exceptions.RequestError: TransportError(400, u'action_request_validation_exception', u'Validation Failed: 1: index is missing;2: type is missing;3: index is missing;4: type is missing;5: index is missing;6: type is missing;7: index is missing;8: type is missing;9: index is missing;10: type is missing;11: index is missing;12: type is missing;13: index is missing;14: type is missing;')

Update I have tryed this : es.update(index='macs', doc_type='users', id="1", body=doc) and I am having this error: Traceback (most recent call last):

  File "./req.py", line 50, in <module>
    es.update(index='macs', doc_type='users', id="1", body=doc)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/__init__.py", line 460, in update
    doc_type, id, '_update'), params=params, body=body)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 329, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 109, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 108, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'action_request_validation_exception', u'Validation Failed: 1: script or doc is missing;').

It is so easy with postman but I really don't understand why it is so had with Elasticsearch (python) to update doc. Someone has an other idea to help ?

Do you get some errors?

Nicolas Heimann
– Nicolas Heimann

2016-07-06 14:41:07 +00:00
Commented Jul 6, 2016 at 14:41 — Nicolas Heimann
– Nicolas Heimann, Commented Jul 6, 2016 at 14:41

rajat · Accepted Answer · 2016-07-06 15:22:43Z

2

Since the second argument in helpers.bulk() method is should be an iterable, try enclosing it in a list.

Also looking at your code, it seems you are expecting only one document from the getDoc() function. So it would be better if you use simple es.update() method instead of using helpers.bulk()

answered Jul 6, 2016 at 15:22

rajat

8941 gold badge9 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

theCode Over a year ago

thank for your response: I have tryed this :es.update(index='macs', doc_type='users', id="1", body=doc), I got example from stackoverflow.com/questions/30598152/… be style not working

rajat Over a year ago

check if your id is correct. Elasticsearch assigns a random id to all the documents, so if you aren't manually setting it to "1", 1 cannot be a id. Further you may want to try 'doc_as_upsert'=True setting in es.update() section. Also can you post what your doc looks like or possibly the mapping

theCode Over a year ago

The problem is that elasticsearch update is not accepting unicode dict. Is there any workaround ?

rajat Over a year ago

See if you have made the same mistake as in this question [link] (stackoverflow.com/questions/30598152/…)

theCode Over a year ago

when I enter something like this body={"stanford": 1, "parsed_sents": parsed } it works fine, but when I make es.get(...), the doc is not working anymore (it is unicode dict). I thing the problem is about unicode

|

Community · Accepted Answer · 2017-05-23 12:01:44Z

2

This solution did work for me.

doc= ess.get(...) 
# modify my doc and ...   
coll = Elasticsearch()
coll.update(index='stories-test',doc_type='news',id=hit.meta.id,
                    body= {"doc": doc['_source']} )

edited May 23, 2017 at 12:01

CommunityBot

11 silver badge

answered Jul 6, 2016 at 18:40

theCode

2,3974 gold badges22 silver badges29 bronze badges

Comments

Anupam · Accepted Answer · 2017-07-14 14:26:48Z

0

(In case someone lands up here with a similar problem,) I was getting the first error mentioned above:

elasticsearch.exceptions.RequestError: TransportError(400, 
'action_request_validation_exception', 'Validation Failed: 
1: index is missing;2: type is missing;

and the following fixed it for me:

obj.to_dict(include_meta=True)

When converting the obj to dict, I added include_meta=True and it worked for me.

answered Jul 14, 2017 at 14:26

Anupam

15.7k20 gold badges73 silver badges104 bronze badges

Collectives™ on Stack Overflow

Elasticsearch update document in python

3 Answers 3

6 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

6 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related