ElasticSearch import data from mysql

Question

Today I have started working on Elastic Search, in my last project I have done the searching in Solr, and I was importing data from mysql and that was pretty simple.

Now I have to do same work in Elastic search, and successfully done the setup I have started working as per "https://github.com/jprante/elasticsearch-jdbc#quick-links" but getting error.

java.lang.IllegalStateException: Received message from unsupported version: [2.0.0] minimal compatible version is: [5.0.0]

I am using Elastic Search version 5.0.0 and using the "wget http://xbib.org/repository/org/xbib/elasticsearch/importer/elasticsearch-jdbc/2.3.4.0/elasticsearch-jdbc-2.3.4.0-dist.zip" which is maximum available version.

Please suggest on this.

According to the link provided, in the given matrix there you could only use the importer till the ES version of 2.3.4, where as you're using the latest. — Kulasangar
– Kulasangar, Commented Nov 21, 2016 at 9:56
Simply you wanted to import data from your mysql db to your index in ES? — Kulasangar
– Kulasangar, Commented Nov 21, 2016 at 9:58
Yes, and any data updated, deleted vai any other program to database then my ES data updated like Solr delta query doing. — Deepesh Uniyal
– Deepesh Uniyal, Commented Nov 21, 2016 at 10:00
I'm not sure how Solr delta works. But you could upload MySQL data to your index using logstash and update it using a scheduler. — Kulasangar
– Kulasangar, Commented Nov 21, 2016 at 10:02

Kulasangar · Accepted Answer · 2016-11-21 10:11:37Z

2

You can use the jdbc plugin in your logstash config in order to upload data to your index and then make sure you could keep updating it by adding a scheduler according to your need.

Your logstash config could look like this:

input {
  jdbc {
    jdbc_driver_library => "/mysql-connector-java-5.1.39-bin.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://localhost:3306/database_name"
    jdbc_user => "root"
    jdbc_password => "password"
    schedule => "* * * * *"
    statement => "select * from table1"
    type => "table1"
  }
}
output {
    elasticsearch {
        index => "testdb"
        document_type => "%{type}"   # <- use the type from each input
        hosts => "localhost:9200"
    }
}

answered Nov 21, 2016 at 10:11

Kulasangar

9,5326 gold badges65 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

ElasticSearch import data from mysql

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related