0

I am trying to import small documents from MongoDB to ElasticSearch but getting an error

      {
    "index": {
      "_index": "impact-fulltext",
      "_id": "t2oLkoUBwNXsTufYzszL",
      "status": 400,
      "error": {
        "type": "mapper_parsing_exception",
        "reason": "failed to parse field [_id] of type [_id] in document with id \u0027t2oLkoUBwNXsTufYzszL\u0027. Preview of field\u0027s value: \u0027605315a3b4f719d00f69f2d3\u0027",
        "caused_by": {
          "type": "mapper_parsing_exception",
          "reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
        }
      }
    }
  }

I am clueless as I also defined the _id but still getting the error.

    db.collection("article_beta")
  .find()
  .limit(100)
  .toArray((err, docs) => {
    if (err) throw err;

    esClient.bulk(
      {
        body: docs.flatMap((doc) => [
          {
            index: {
              _index: "impact-fulltext",
              _id: doc._id.$oid,
            },
          },
          doc,
        ]),
      },
      (err, resp) => {
        if (err) throw err;
        console.log(resp);
        client.close();
      }
    );
  });

1 Answer 1

2

In elasticsearch, the documents have some meta information and also the source of the documents. So, the document structure is following :

{
   "_index": "index-name",
   "_id": "document-id-stored-by-elasticsearch",
   "_source": {
      // documents itself
   }
}

So, inside the document, you can not use _id field as follow because _id field is reserved field :

POST myindex/_doc
{
  "_id": "123123",
  "field1": "value1"
}

So this request will give an error as follow :

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "failed to parse field [_id] of type [_id] in document with id '16Y6koUBnxYxC21BP-tS'. Preview of field's value: '123123'",
    "caused_by": {
      "type": "mapper_parsing_exception",
      "reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
    }
  },
  "status": 400
}

In your example, doc variable which the value coming from Mongodb have a field as _id. You need to change the name of that field as id. You can put following lines inside of your anonymous function inside the flatMap function.

db.collection("article_beta")
  .find()
  .limit(100)
  .toArray((err, docs) => {
    if (err) throw err;

    esClient.bulk(
      {
        body: docs.flatMap((doc) => {
          doc.id = doc._id  // <---- added
          delete doc._id    // <---- added
          return  [
            {
              index: {
                _index: "impact-fulltext",
                _id: doc.id.$oid,
              },
            },
            doc,
          ]
        }),
      },
      (err, resp) => {
        if (err) throw err;
        console.log(resp);
        client.close();
      }
    );
  });
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.