Disable dynamic mapping creation for only specific indexes on elasticsearch?

Question

I'm trying to disable dynamic mapping creation for only specific indexes, not for all. For some reason I can't put default mapping with 'dynamic' : 'false'. So, here left two options as I can see:

specify property 'index.mapper.dynamic' in file elasticsearch.yml.
put 'index.mapper.dynamic' at index creation time, as described here https://www.elastic.co/guide/en/kibana/current/setup.html#kibana-dynamic-mapping

First option may only accept values: true, false and strict. So there is no way to specify subset of specific indexes (like we do by pattern with property 'action.auto_create_index' https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation).

Second option just not works. I've created index

POST http://localhost:9200/test_idx/
{
    "settings" : {
        "mapper" : {
            "dynamic" : false
        }
    },
    "mappings" : {
        "test_type" : {
            "properties" : {
                "field1" : {
                    "type" : "string"
                }
            }
        }
    }
}

Then checked index settings:

GET http://localhost:9200/test_idx/_settings    
{
    "test_idx" : {
        "settings" : {
            "index" : {
                "mapper" : {
                    "dynamic" : "false"
                },
                "creation_date" : "1445440252221",
                "number_of_shards" : "1",
                "number_of_replicas" : "0",
                "version" : {
                    "created" : "1050299"
                },
                "uuid" : "5QSYSYoORNqCXtdYn51XfA"
            }
        }
    }
}

and mapping:

GET http://localhost:9200/test_idx/_mapping
{
    "test_idx" : {
        "mappings" : {
            "test_type" : {
                "properties" : {
                    "field1" : {
                        "type" : "string"
                    }
                }
            }
        }
    }
}

so far so good, let's index document with undeclared field:

POST http://localhost:9200/test_idx/test_type/1
{
    "field1" : "it's ok, field must be in mapping and in source",
    "somefield" : "but this field must be in source only, not in mapping"
}

Then I've checked mapping again:

GET http://localhost:9200/test_idx/_mapping
{
    "test_idx" : {
        "mappings" : {
            "test_type" : {
                "properties" : {
                    "field1" : {
                        "type" : "string"
                    },
                    "somefield" : {
                        "type" : "string"
                    }
                }
            }
        }
    }
}

As you can see, mapping is extended regardless of index setting "dynamic" : false. I've also tried to create index exactly as described in doc

PUT http://localhost:9200/test_idx
{
    "index.mapper.dynamic": false
}

but got the same behavior.

Maybe I've missed something?

Thanks a lot in advance!

Sorry, forgot about version, I'm working with elastic 1.5.2. — Volodymyr
– Volodymyr, Commented Oct 21, 2015 at 15:47
See the last but one comment of answer, it's actually answer — Volodymyr
– Volodymyr, Commented Oct 22, 2015 at 13:10
"settings.mapper.dynamic": false disables automatic type creation — czerasz
– czerasz, Commented Jul 2, 2016 at 19:50

Andrei Stefan · Accepted Answer · 2015-10-21 16:28:42Z

21

You're almost there: the value needs to be set to strict. And the correct usage is the following:

PUT /test_idx
{
  "mappings": {
    "test_type": {
      "dynamic":"strict",
      "properties": {
        "field1": {
          "type": "string"
        }
      }
    }
  }
}

And pushing this a bit further, if you want to forbid the creation even of new types, not only fields in that index, use this:

PUT /test_idx
{
  "mappings": {
    "_default_": {
      "dynamic": "strict"
    },
    "test_type": {
      "properties": {
        "field1": {
          "type": "string"
        }
      }
    }
  }
}

Without _default_ template:

PUT /test_idx
{
  "settings": {
    "index.mapper.dynamic": false
  },
  "mappings": {
    "test_type": {
      "dynamic": "strict",
      "properties": {
        "field1": {
          "type": "string"
        }
      }
    }
  }
}

edited Oct 21, 2015 at 16:28

answered Oct 21, 2015 at 16:07

Andrei Stefan

52.5k6 gold badges102 silver badges92 bronze badges

Sign up to request clarification or add additional context in comments.

22 Comments

Volodymyr Over a year ago

Thank you for answer. Unfortunately I have to disable dynamic mapping creation for whole index (index-level). At your example it was disabled just for type 'test_type'. And I can't use 'default' mapping for this (as I told in the beginning)

Andrei Stefan Over a year ago

The _default_ mapping needs to be used in order to disable index-level creation of fields/types.

Volodymyr Over a year ago

Unfortunately I can't put _default_ mapping. I'm trying to understand how to disable dynamic mapping creation without put any mappings at all (disable dynamic mapping by putting other mapping is a bit tricky from API point of view, IMHO)

Andrei Stefan Over a year ago

Yeah, there is a way. See my last attempt using settings and, also, "dynamic": "strict" for the mapping of a single type.

Andrei Stefan Over a year ago

No, there are two separate settings. The index level one and the mapping level one. This is from documentation: The dynamic creation of mappings for unmapped types can be completely disabled by setting index.mapper.dynamic to false. The dynamic creation of fields within a type can be completely disabled by setting the dynamic property of the type to strict. The index level setting is for creating new mappings (for new types). The type level one is for fields within that specific type. index.mapper.dynamic is not inherited to new types, because the type level setting is a different one.

|

Lordran · Accepted Answer · 2016-08-12 09:21:43Z

5

You must know about that the below part just mean that ES could'nt create a type dynamically.

"mapper" : {
        "dynamic" : false
    }

You should configure ES like this:

PUT http://localhost:9200/test_idx/_mapping/test_type
{
  "dynamic":"strict"
}

Then you cant't index other field that without mapping any more ,and get an error as follow:

mapping set to strict, dynamic introduction of [hatae] within [data] is not allowed

If you wanna store the data,but make the field can't be index,you could take the setting like this:

PUT http://localhost:9200/test_idx/_mapping/test_type
{
  "dynamic":false
}

Hope these can help the people with the same issue :).

answered Aug 12, 2016 at 9:21

Lordran

6838 silver badges19 bronze badges

Comments

avetisk · Accepted Answer · 2020-02-26 15:18:38Z

3

The answer is in the doc (7x.): https://www.elastic.co/guide/en/elasticsearch/reference/7.x/dynamic.html

The dynamic setting controls whether new fields can be added dynamically or not. It accepts three settings:

true

Newly detected fields are added to the mapping. (default)

false

Newly detected fields are ignored. These fields will not be indexed so will not be searchable but will still appear in the _source field of returned hits. These fields will not be added to the mapping, new fields must be added explicitly.

strict

If new fields are detected, an exception is thrown and the document is rejected. New fields must be explicitly added to the mapping.

PUT my_index
{
  "mappings": {
    "dynamic": "strict", 
    "properties": {
      "user": { 
        "properties": {
          "name": {
            "type": "text"
          },
          "social_networks": { 
            "dynamic": true,
            "properties": {}
          }
        }
      }
    }
  }
}

answered Feb 26, 2020 at 15:18

avetisk

12.4k4 gold badges28 silver badges38 bronze badges

2 Comments

SoH Over a year ago

so this is only possible when creating index?

avetisk Over a year ago

Well, as far as I remember you can do this as an update to: not only during creation, but afterwards too.

Rafiq · Accepted Answer · 2022-02-26 00:09:53Z

first, please be concern aboout value false or strict,they work in a different way.

using "dynamic": "false" and create documents with fields not covered by the mapping, those fields will be ignored (so they won't be stored) and wouldn't show up in _source when you GET the document.

where value strict will not allow you to create the document rather it will throw an exception

Inner objects inherit the dynamic setting from their parent object or from the mapping type. In the following example, dynamic mapping is disabled at the type level, so no new top-level fields will be added dynamically.

However, the user.social_networks object enables dynamic mapping, so you can add fields to this inner object.

https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic.html

PUT my-index-000001
{
  "mappings": {
    "dynamic": false, 
    "properties": {
      "user": { 
        "properties": {
          "name": {
            "type": "text"
          },
          "social_networks": {
            "dynamic": true, 
            "properties": {}
          }
        }
      }
    }
  }
}

if you are using node.js client

    await this.client.indices.putMapping({
      index: ElasticIndex.UserDataFactory,
      body: {
        dynamic: 'strict',
        properties: {
          ...this.schema,
        },
      },
    });

Antoine Vulcain · Accepted Answer · 2019-11-09 20:57:03Z

0

You cannot disable dynamic mapping in ES 7 anymore, what you can do if you have completely unstructured data is to disable completely the mapping for the index like this:

curl -X PUT "localhost:9200/my_index?pretty" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "enabled": false 
  }
}
'

if you are using python you can do this:

from elasticsearch import Elasticsearch

# Connect to the elastic cluster
es=Elasticsearch([{'host':'localhost','port':9200}])

request_body = {
        "mappings": {
                 "enabled": False
    }
}
es.indices.create(index = 'my_index', body = request_body)

answered Nov 9, 2019 at 20:57

Antoine Vulcain

441 silver badge5 bronze badges

3 Comments

avetisk Over a year ago

I don't know which version you are using but it seems it works just fine in 7.x (at least 7.6.0).

Antoine Vulcain Over a year ago

I am using 7.4 version, I had to disable completely the mapping because each object has a different structure

Sherwin F Over a year ago

I downvoted this as the answer is not accurate and can be misleading to others. As the answer by avetisk notes, this is still possible in 7.x (I've tested 7.7.0) and the documentation still shows it as supported. The only thing that changed is the removal of the type, so the options go directly under "mappings" now.

SoH · Accepted Answer · 2021-02-23 19:54:03Z

0

For ES 7 if you want to update an existing index:

PUT customers/_mapping
{
  "dynamic": "strict"
}

answered Feb 23, 2021 at 19:54

SoH

2,1902 gold badges24 silver badges55 bronze badges

Collectives™ on Stack Overflow

Disable dynamic mapping creation for only specific indexes on elasticsearch?

6 Answers 6

22 Comments

Comments

2 Comments

Comments

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

22 Comments

Comments

2 Comments

Comments

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related