Index mongoDB with ElasticSearch

Question

I already have MongoDB and installed Elasticsearch with Mongoriver. So I set up my river:

$ curl -X PUT localhost:9200/_river/database_test/_meta -d '{
  "type": "mongodb",
  "mongodb": {
    "servers": [
      {
        "host": "127.0.0.1",
        "port": 27017
      }
    ],
    "options": {
      "secondary_read_preference": true
    },
    "db": "database_test",
    "collection": "event"
  },
  "index": {
    "name": "database_test",
    "type": "event"
  }
}'

I simply want to get events that have country:Canada so I try:

$ curl -XGET 'http://localhost:9200/database_test/_search?q=country:Canada'

And I get:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 0,
    "max_score": null,
    "hits": []
  }
}

I am searching the web and I read that I should first index my collection with Elasticsearch (lost the link). Should I index my Mongodb? What should I do to get results from an existing MongoDB collection?

Rodrigo Del C. Andrade · Accepted Answer · 2013-11-21 12:23:23Z

18

The mongodb river relies on the operations log of MongoDB to index documents, so it is a requirement that you create your mongo database as a replica set. I assume that you're missing it, so when you create the river, the initial import sees nothing to index. I am also assuming that you're on Linux and you have a handle on the shell cli tools, so try this:

Follow these steps:

Make sure that the mapper-attachments Elasticsearch plugins is also installed
Make a backup of your database with mongodump
edit mongodb.conf (usually in /etc/mongodb.conf, but varies on how you installed it) and add the line:

replSet = rs0

"rs0" is the name of the replicaset, it can be whatever you like.
restart your mongo and then log in its console. Type:

rs.initiate() rs.slaveOk()

The prompt will change to rs0:PRIMARY>

Now create your river just as you did in the question and restore your database with mongorestore. Elasticsearch should index your documents.

I recomend using this plugin: http://mobz.github.io/elasticsearch-head/ to navigate your indexes and rivers and make sure your data got indexed.

If that doesnt work, please post which versions you are using for the mongodb-river-plugin, elasticsearch and mongodb.

answered Nov 21, 2013 at 12:23

Rodrigo Del C. Andrade

5031 gold badge6 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

y2p Over a year ago

Worked like a charm. I have been searching for quite some time. Thank you!

praaveen V R Over a year ago

hi i have same problem...followed your answer but still getting same error..im using mongodb-2.6.1 ,elasticsearch "lucene_version" : "4.6" ,elasticsearch-river-mongodb-1.7.1 and elasticsearch-mapper-attachments/1.9.0

Anglepoise Over a year ago

Does the actual ES index happen after mongorestore finishes?

Rodrigo Del C. Andrade Over a year ago

If the river to the index is created, it should start while the mongorestore is still running, but I am not the author of the river and I know some things have changed with how the initial import happebs since this answer was posted, since it was for a very old version of ES, so this might not hold true for the current river version.

Collectives™ on Stack Overflow

Index mongoDB with ElasticSearch

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related