Issues with Mapping Elastic Index fields with Java Fields

Question

Current Im working on a spring boot project to fetch the data from specific Elastic search index using Spring Data ElasticSearch (version 4.1.7). Im able to connect to elastic search and fetch all the data associated with a specific index. But there are some fields in the index which are returning null value. Where as the actual value exist in the index in elastic search .
Im having issues in retriving the specific fields due to incorrect field types. I have created a Document Class which

@Document(indexName = "netstat*")
@Setting(settingPath = "static/es-settings.json")
public class NetworkStats {
@Id
@Field(type = FieldType.Keyword)
private String id; 

@Field
private String timeStamp ;

@Field(type = FieldType.Text)
private String podName ; 

@Field(type = FieldType.Text)
private String iporHost ;

@Field(type = FieldType.Auto)
private String connectionCount ; 

@Field(type = FieldType.Text)
private String command ;    
}

Apart from the above I have getters & setters for the field as well defined.Im unabel to retrieve the values for fields connectionCount, iporHost, podName, timeStamp. Though the fields have values in the elastic index.
Sample data from ElasticSearch below

{
            "_index": "netstat-2021.11.09",
            "_type": "_doc",
            "_id": "0sJmA30BW7LXXVrK0nCg",
            "_score": 1.0,
            "_source": {
                "podname": "logindeployment-5479f7-4f2qr",
                "input": {
                    "type": "log"
                },
                "iporhost": "6200",
                "tags": [],
                "command": "OUTESTAB",
                "agent": {
                    "version": "7.15.1",
                    "id": "e8feeb4e-1f9e-41b2-bcba-38e507d176db",
                    "type": "filebeat",
                    "hostname": "BL1726L",
                    "name": "BL1726L",
                    "ephemeral_id": "c2859ece-3e95-4204-909d-af59322b9fa2"
                },
                "timestamp": "05-10-21T13:17:25.014719626",
                "ecs": {
                    "version": "1.11.0"
                },
                "connectionscount": "2",
                "@timestamp": "2021-11-09T06:31:29.104Z",
                "log": {
                    "offset": 1002,
                    "file": {
                        "path": "D:\\elastic_stack\\data\\log\\netStats.log"
                    }
                }
            }
        },

Require help on how to retrieve the data for the above mentioned fields.I guess there is some issue with field mapping. Any help on this is much appreciated.

Abacus · Accepted Answer · 2021-11-09 11:12:22Z

1

the fields in Elasticsearch seem all to have lower case names. Your properties have camel case. So you need to define the name:

@Field(type = FieldType.Text, name ="podname")
private String podName ;

If you do not define a FieldType or use FieldType.Auto, then no mapping is written and Elasticsearch will try to guess the type. You should define the correct type to make sure that the conversion is done as desired.

Especially for the timestamp field you should define a date format that matches what you have in the index.

As the index exists you should make sure that you use the same type in the entity definition that is already used in the index.

answered Nov 9, 2021 at 11:12

Abacus

19.6k7 gold badges54 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

vr3w3c9 Over a year ago

thanks a lot for the response. This FieldType mapping is it mandatory to define ? Can Elasticsearch do the mapping. Also for the date format, where can I specify the format. Please clarify

Abacus Over a year ago

Not mandatory, but highly recommended. Please check the documentation at docs.spring.io/spring-data/elasticsearch/docs/4.2.6/reference/…

Nausheen Khan · Accepted Answer · 2024-10-07 07:29:16Z

0

@Field(type = FieldType.Text, name ="iporhost ") private String iporHost ;

Above should suffice for retriving the DAO getter and setter.

Mention FieldType as its highly recommended. If FieldType is absent then elastic assume the type and that may not be correct everytime. Hence avoid putting FieldType.Auto.
Explicity mentioning the fieldType help the mapping correctly.

answered Oct 7, 2024 at 7:29

Nausheen Khan

1512 silver badges6 bronze badges

Collectives™ on Stack Overflow

Issues with Mapping Elastic Index fields with Java Fields

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related