Export data from Elasticsearch to CSV using Logstash

Question

How can I export data from Elasticsearch to CSV using Logstash? I need to include only specific columns.

Sergey Alekseev · Accepted Answer · 2016-05-03 14:39:55Z

5

Install 2 plugins: elasticsearch input plugin and csv output plugin. Then create a configuration file. Here is a good example for this particular case.

You are ready to go now, just run:
bin/logstash -f /path/to/logstash-es-to-csv-example.conf

And check export.csv file specified in output -> csv -> path.

answered May 3, 2016 at 14:39

Sergey Alekseev

12.4k12 gold badges40 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Dominik · Accepted Answer · 2017-02-10 15:12:32Z

2

Important note: There is a known bug in csv output plugin when working with Logstash 5.x. The plugin generates a string of %{host} %{message}%{host} %{message}%{host} %{message}. There's an open issue for it: https://github.com/logstash-plugins/logstash-output-csv/issues/10

As a workaround you may:

downgrade to Logstash 2.x until this gets resolved

use the file output instead

file {
    codec => line { format => "%{field1},%{field2}"}
    path => "/path/to/data_export.csv"
}

modify the plugin's code according to the github discussion...

answered Feb 10, 2017 at 15:12

Dominik

4003 silver badges16 bronze badges

Collectives™ on Stack Overflow

Export data from Elasticsearch to CSV using Logstash

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related