Export Data MongoDB to ElasticSearch

Question

I want to export data from MongoDB to ElasticSearch.Mongo River plugin is an option for me to first take dump of collection and then restore which works fine for me.But I do not want to use Mongo River plugin , I am using elaster to export data from MongoDB to Elasticsearch.

Elastic Search Version - 1.3.2
Node.js version - v0.11.8-pre
MongoDB Version - 2.4.x

When I execute : ./bin/elaster it says:

{ 
  [Error: MapperParsingException[object mapping for [collection] tried to parse as object, but got EOF, has a concrete value been provided to it?]]
  message: 'MapperParsingException[object mapping for [collection] tried to parse as object, but got EOF, has a concrete value been provided to it?]'
}

My Elaster Configuration is:

module.exports = {

  mongo: {
    connection: 'mongodb://127.0.0.1:27017/times'
  },

  elastic: {
    host: {
      host: '127.0.0.1'
    },
    requestTimeout: 5000
  },

  collections: [ 
    {
      name: "walldisplay",
      index: "walldisplay",
      type: "collection",
      fields: [
        "_id",
        "wat",
        "wct",
        "u",
        "i",
        "cd"
      ],
      mappings: {
        "collection": {
          "properties": {
            "wat":{
              'type':'string',
              'index': 'not_analyzed'
            },
            "wct":{
              'type':'string',
              'index': 'not_analyzed'
            },
            "u":{
              "type" : "object",
              "dynamic" : true,
              "properties":{
                "_id":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "n":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "st":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "id":{
                  'type':'string',
                  'index': 'not_analyzed'
                }
              },
              "index":"not_analyzed"
            },
            "i":{
              "type" : "nested",
              "include_in_parent" : true,
              "properties":{
                "_id":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "ti":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "st":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "n":{
                  'type':'string',
                  'index': 'not_analyzed'
                },
                "cst":{
                  'type':'string',
                  'index': 'not_analyzed'
                }
              }
            },
            "cd":{
              'type':'long',
              'index': 'not_analyzed'
            },
          }
        }
      }
    }
  ]
};

Also please check sample document in-line

{
  "_id": ObjectId("5406a47970b17246b9a293e1"),
  "cd": 1409721465,
  "i": [
    {
      "_id": ObjectId("50f693d17deed44cf000007f"),
      "st": "seo-title",
      "ti": "title",
      "n": "categoryname",
      "cst": "category-seotitle",
      "r": null,
      "c": null
    },
    {
      "_id": ObjectId("50f693d17deed44cf000007f"),
      "st": "seo-title",
      "ti": "title",
      "n": "categoryname",
      "cst": "category-seotitle",
      "r": null,
      "c": null
    },
    {
      "_id": ObjectId("50f693d17deed44cf000007f"),
      "st": "seo-title",
      "ti": "title",
      "n": "categoryname",
      "cst": "category-seotitle",
      "r": null,
      "c": null
    }
  ],
  "u": {
    "_id": ObjectId("50ce4f79edaffd69e40ee010"),
    "n": "Richa Sen",
    "st": "richasen",
    "id": "d8mzxlp9ekn323l6jg5s8tly1"
  },
  "wat": 1,
  "wct": 1
}

BatScream · Accepted Answer · 2014-09-03 11:47:49Z

1

Since you have defined "collection"as the type and your index as "walldisplay", the type in the mapping should be "collection" rather than "walldisplay".

The put mapping API allows to register specific mapping definition for a specific type

See if the following works

 ..
     mappings:{
                "collection":{
                       "properties":{
                              ...
                           }
                     }
               }

edited Sep 3, 2014 at 11:47

answered Sep 3, 2014 at 11:38

BatScream

19.7k4 gold badges54 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

coder Over a year ago

Hi , Thanks for your reply. Yes it helped me to proceed further. Now it gives me following Error { [Error: MapperParsingException[failed to parse [wct]]; nested: NumberFormatException[For input string: "i"]; ] message: 'MapperParsingException[failed to parse [wct]]; nested: NumberFormatException[For input string: "i"]; ' } As you can see in provided document WCT => integer value and "i" is an array.

BatScream Over a year ago

Check if you already have an index with the name - "walldisplay" and type "collection" with a mapping in elasticsearch?. Delete the mapping or the type itself and try loading again.

coder Over a year ago

Yes I deleted all existing mappings by curl -XDELETE 'localhost:9200*/_mapping/*' and restarted Elastic Search.But still getting same error.

Robert · Accepted Answer · 2015-04-03 08:11:24Z

0

WCT field is mapped as number type but one your document is having wct value as string that's why you are getting such error so try to change your wct value from string to integer.

edited Apr 3, 2015 at 8:11

Robert

5,31843 gold badges70 silver badges115 bronze badges

answered Apr 3, 2015 at 7:52

Aroon

114 bronze badges

Collectives™ on Stack Overflow

Export Data MongoDB to ElasticSearch

2 Answers 2

3 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Related