elasticsearch.bulk()

[sirkubax]

I'm confused about elasticsearch-py bulk
I did expect that the following code would work:

from elasticsearch import Elasticsearch
es = Elasticsearch()
doc = '''\n {"host":"logsqa","path":"/logs","message":"test test","@timestamp":"2014-10-02T10:11:25.980256","tags":["multiline","mydate_0.005"]} \n'''
result = es.bulk(index="logstash-test", doc_type="test", body=doc)

Based on elasticsearch.bulk() function description it should do the work.

---

More on the topic:

My post about elasticsearch.bulk()
http://stackoverflow.com/questions/26159949/elasticsearch-python-bulk-api-elasticsearch-py

Working example witch helpers.bulk(es, actions)
http://stackoverflow.com/questions/20288770/how-to-use-bulk-api-to-store-the-keywords-in-es-by-using-python

[honzakral]

Hi sirkubax,

the bulk api (as do all the others) follows very closely the bulk api format for elasticsearch itself, so the body would have to be:

doc = '''{"index": {}}\n{"host":"logsqa","path":"/logs","message":"test test","@timestamp":"2014-10-02T10:11:25.980256","tags":["multiline","mydate_0.005"]}\n'''

for it to work. Alternatively it could be a list of those two dicts.

This is a complicated and clumsy format to work with from python, that's why I tried to create a more convenient way to work with bulk in elasticsearch.helpers.bulk (0). It simply accepts an iterator of documents, will extract any optional metadata from it (like _id, _type etc) and construct (and execute) the bulk request for you. For more info on the accepted formats see the docs for streaming_bulk above which is a helper to process the stream in iterative manner (one at a time from the point of the user, batched in chunks in the background).

Hope this helps.

0 - http://elasticsearch-py.readthedocs.org/en/master/helpers.html#elasticsearch.helpers.bulk

[sirkubax]

Thanks. That's perfect.
2 paź 2014 15:08 "Honza Král" notifications@github.com napisał(a):

Hi sirkubax,

the bulk api (as do all the others) follows very closely the bulk api
format for elasticsearch itself, so the body would have to be:

doc = '''{"index": {}}\n{"host":"logsqa","path":"/logs","message":"test test","@timestamp":"2014-10-02T10:11:25.980256","tags":["multiline","mydate_0.005"]}\n'''

for it to work. Alternatively it could be a list of those two dicts.

This is a complicated and clumsy format to work with from python, that's
why I tried to create a more convenient way to work with bulk in
elasticsearch.helpers.bulk (0). It simply accepts an iterator of
documents, will extract any optional metadata from it (like _id, _type
etc) and construct (and execute) the bulk request for you. For more info on
the accepted formats see the docs for streaming_bulk above which is a
helper to process the stream in iterative manner (one at a time from the
point of the user, batched in chunks in the background).

Hope this helps.

0 -
http://elasticsearch-py.readthedocs.org/en/master/helpers.html#elasticsearch.helpers.bulk

—
Reply to this email directly or view it on GitHub
#135 (comment)
.

[honzakral]

great, closing this issue.