OpenSearch - Error when creating index using Java client

Question

I'm trying to create index with given settings from json.

{
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "my_synonym_analyzer": {
            "type": "custom",
            "tokenizer": "whitespace",
            "filter": [
              "my_synonym_filter"
            ]
          }
        },
        "filter": {
          "my_synonym_filter": {
            "type": "synonym_graph",
            "synonyms": []
          }
        }
      }
    }
  }
}

My code to create index is :


import jakarta.json.stream.JsonParser;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import java.io.StringReader;
import java.util.Objects;
import lombok.extern.slf4j.Slf4j;
import org.opensearch.client.json.JsonpMapper;
import org.opensearch.client.opensearch.OpenSearchClient;
import org.opensearch.client.opensearch._types.mapping.TypeMapping;
import org.opensearch.client.opensearch.indices.CreateIndexRequest;
import org.opensearch.client.opensearch.indices.CreateIndexResponse;
import org.opensearch.client.opensearch.indices.IndexSettings;
import org.opensearch.common.io.Streams;
import org.springframework.beans.factory.annotation.Autowired;

public class IndexManager {

  @Autowired private OpenSearchClient searchClient;

  protected void create() throws IOException {

    if (this.indexExists()) {
      this.delete();
    }

    JsonpMapper mapper = searchClient._transport().jsonpMapper();
    try (Reader readerSettings =
            new InputStreamReader(
                Objects.requireNonNull(
                    this.getClass()
                        .getResourceAsStream("testindex-index-settings.json")));
        Reader readerMappings =
            new InputStreamReader(
                Objects.requireNonNull(
                    this.getClass()
                        .getResourceAsStream("testindex-index-mappings.json")));
        JsonParser settingsParser =
            mapper
                .jsonProvider()
                .createParser(new StringReader(Streams.copyToString(readerSettings)));
        JsonParser mappingsParser =
            mapper
                .jsonProvider()
                .createParser(new StringReader(Streams.copyToString(readerMappings))); ) {

      String indexName = "testindex";

      IndexSettings indexSettings = IndexSettings._DESERIALIZER.deserialize(settingsParser, mapper);

      TypeMapping typeMapping = TypeMapping._DESERIALIZER.deserialize(mappingsParser, mapper);

      CreateIndexRequest indexRequest =
          new CreateIndexRequest.Builder()
              .index(indexName)
              .settings(indexSettings)
              .mappings(typeMapping)
              .build();

      CreateIndexResponse indexResponse = searchClient.indices().create(indexRequest); // line 93

      log.info("Index {} created with response [{}]", indexName, indexResponse.toString());
    }
  }
}

When trying to create index from above code, I'm getting unknown setting error:

org.opensearch.client.opensearch._types.OpenSearchException: Request failed: [illegal_argument_exception] unknown setting [index.settings.analysis.analyzer.my_synonym_analyzer.filter] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
    at org.opensearch.client.transport.rest_client.RestClientTransport.getHighLevelResponse(RestClientTransport.java:273) ~[opensearch-java-2.4.0.jar:na]
    at org.opensearch.client.transport.rest_client.RestClientTransport.performRequest(RestClientTransport.java:143) ~[opensearch-java-2.4.0.jar:na]
    at org.opensearch.client.opensearch.indices.OpenSearchIndicesClient.create(OpenSearchIndicesClient.java:248) ~[opensearch-java-2.4.0.jar:na]
    at com.joinhobnob.crm.service.search.command.IndexManager.create(IndexManager.java:93) ~[main/:na]

But while creating the index from opensearch's dashboard console, with same settings its working fine.

curl -XPUT "http://opensearch:9200/test_index?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "my_synonym_analyzer": {
            "type": "custom",
            "tokenizer": "whitespace",
            "filter": [
              "my_synonym_filter"
            ]
          }
        },
        "filter": {
          "my_synonym_filter": {
            "type": "synonym_graph",
            "synonyms": []
          }
        }
      }
    }
  }
}'

Response: 
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "index": "test_index"
}

I've gone through the given links, but have not found the solution yet.

Elasticsearch error while mapping - unknown setting

Creating Index with OpenSearch Java Client

Can anyone help me find the solution?

The error states index.settings.analysis... while your curl body shows settings.index.analysis.... Can you verify that testindex-index-settings.json contains the right hierarchy, i.e. settings.index.analysis... ? — Val
– Val, Commented May 10, 2023 at 6:09
I've verified the hierarchy, its supposed to be that way. @Val — Sabu Shakya
– Sabu Shakya, Commented May 10, 2023 at 6:32
Can you remove the "settings" section in the testindex-index-settings.json file and just make "index" being the top-level section? — Val
– Val, Commented May 10, 2023 at 6:51

Val · Accepted Answer · 2023-05-11 16:21:41Z

1

You need to remove the settings section in your testindex-index-settings.json file

  {
    "index": {
      "analysis": {
        "analyzer": {
          "my_synonym_analyzer": {
            "type": "custom",
            "tokenizer": "whitespace",
            "filter": [
              "my_synonym_filter"
            ]
          }
        },
        "filter": {
          "my_synonym_filter": {
            "type": "synonym_graph",
            "synonyms": []
          }
        }
      }
    }
  }

answered May 11, 2023 at 16:21

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Chris March · Accepted Answer · 2023-10-19 17:48:53Z

For those of you who are trying to create something from a json file or just need to understand how the new TypeMapping works, here is some code I created to build a TypeMapping object from an existing sson Mapping:

@SneakyThrows
public static TypeMapping typeMapping() {
    var builder = new TypeMapping.Builder();
    var sourceJson = typeMappingJson();

    builder.enabled(true);
    builder.dateDetection(true);

    var properties = new LinkedHashMap<String, Property>();
    var jsonNodeIterator = new ObjectMapper().readTree(sourceJson).get("mappings").get("properties").fields();
    while (jsonNodeIterator.hasNext()) {
        var node = jsonNodeIterator.next();

        var property = new Property.Builder();
        var type = node.getValue().get("type").asText();
        var format = node.getValue().hasNonNull("date") ? node.getValue().get("date").asText(null) : null;
        var index = node.getValue().hasNonNull("date") ? node.getValue().get("index").asBoolean(false) : null;

        switch (type) {
            case "boolean" -> property.boolean_(new BooleanProperty.Builder().index(index).build());
            case "text" -> property.text(new TextProperty.Builder().index(index).build());
            case "keyword" -> property.keyword(new KeywordProperty.Builder().index(index).build());
            case "date" -> property.date(new DateProperty.Builder().format(format).build());
            case "integer" -> property.integer(new IntegerNumberProperty.Builder().index(index).build());
            case "geo_pont" -> property.geoPoint(new GeoPointProperty.Builder().ignoreMalformed(true).build());
            case "object" -> property.object(new ObjectProperty.Builder().enabled(true).build());
            default -> throw new IllegalArgumentException("No mapping created for type:" + type);
        }

        properties.put(node.getKey(), property.build());
    }

    builder.properties(properties);

    return builder.build();
}

Collectives™ on Stack Overflow

OpenSearch - Error when creating index using Java client

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related