Elasticsearch Bulk API using curl and text file

Question

I'm a beginner with Elasticsearch and am following an "Essential Training" in LinkedIn Learning. I'm trying to follow with bulk loading API and the instructor is using Linux, I'm on Windows. He created a text file to read in with data using "VI". I just created a text file and pasted the data and removed the ".txt". The contents of the file, called reqs, is this:

{
   "index":{
      "_index":"my-test",
      "_type":"my-type",
      "_id":"1"
   }
}{
   "col1":"val1"
}{
   "index":{
      "_index":"my-test",
      "_type":"my-type",
      "_id":"2"
   }
}{
   "col1":"val2"
}{
   "index":{
      "_index":"my-test",
      "_type":"my-type",
      "_id":"3"
   }
}{
   "col1":"val3"
}

I've tried saving it with a carriage return (new line) after the last line and without. I saved this into my elasticsearch folder (C:\elasticsearch-7.12.0) which is the same directory I'm running the following command from:

c:\elasticsearch-7.12.0>curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@reqs"; echo

When I do this, I'm getting the following error:

{"error":{"root_cause":[{"type":"parse_exception","reason":"request body is required"}],"type":"parse_exception","reason":"request body is required"},"status":400}

The application/x-ndjson format is newline-delimited JSON. I'm pretty sure the formatted JSON file will not work. Maybe ensure each document is on a separate line, no newlines for human-friendly formatting? — rud
– rud, Commented Apr 20, 2021 at 19:03

Bhavya · Accepted Answer · 2021-04-07 15:28:58Z

1

Use this below curl command

curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/index-name/_bulk?pretty' --data-binary @reqs.json

reqs.json should look like this

{"index" : {"_index" : "my-test", "_type" : "my-type", "_id" : "1"}}
{"col1" : "val1"}
{"index" : {"_index" : "my-test", "_type" : "my-type", "_id" : "2"}}
{"col1" : "val2"}
{"index" : {"_index" : "my-test", "_type" : "my-type", "_id" : "3"}}
{"col1" : "val3"}

edited Apr 7, 2021 at 15:28

answered Apr 7, 2021 at 15:12

Bhavya

16.2k3 gold badges23 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Rob Searing Over a year ago

I am SO sorry - I thought I had posted a reply (NEW to Stack Overflow). It didn't work - I got a {"error":{"root_cause":[{"type":"parse_exception","reason":"request body is required"}],"type":"parse_exception","reason":"request body is required"},"status":400}

Bhavya Over a year ago

@Rob Searing have you saved the file in .json format ? i.e instead of just reqs, you need to save it as reqs.json

Rob Searing Over a year ago

correct. I have elasticsearch and kibana both working (when I open a browser I can get localhost:9200 & :5601 - Kibana opens). I have reqs.json in the elasticsearch folder with contents: {"index" : {"_index" : "my-test", "_type" : "my-type", "_id" : "1"}} {"col1" : "val1"} {"index" : {"_index" : "my-test", "_type" : "my-type", "_id" : "2"}} {"col1" : "val2"} {"index" : {"_index" : "my-test", "_type" : "my-type", "_id" : "3"}} {"col1" : "val3"} I copy paste your curl command from the same folder I have the reqs.json file in (the elasticsearch one) and get

Rob Searing Over a year ago

c:\elasticsearch-7.12.0>curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/index-name/_bulk?pretty' --data-binary @reqs.json curl: (6) Could not resolve host: application curl: (6) Could not resolve host: 'localhost

Collectives™ on Stack Overflow

Elasticsearch Bulk API using curl and text file

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related