1

I am new to Elastic Search. I am using Java, Elastic Search and Spring. My scenario is that I have a jQuery Table on a page. This table is a list of all the Users. Associated with users is a Set of extra information, we'll call this extraDataValues. This could be anything, the most important thing is that the value stored is a String value, which could be a formatted Date or DateTime String value. Here it is in JSON:

"extraDataValues": [
    {
        "id": 3,
        "extraDataValueObject": {
            "id": 12,
            "label": "Metadata Date",
            "displayable": true
        },
        "value": "01/01/2015 00:01:11"
    },
    {
        "id": 4,
        "extraDataValueObject": {
            "id": 13,
            "label": "Metadata TextBox",
            "displayable": true
        },
        "value": "zzzz"
    }
],

This brings me to the problem. Given that I have a nested extraDataValuesset in Users with extraDataObjects, how can I:

  • Perform a sort on a specific extraDataObject with either ID 12 or 13 that resides in extraDataValues. Basically, in Java I would get the Set of extraDataValues and within there I would search for all the extraDataValueObjects with ID 12, then I would see if that is a Date Time and sort it as a Date Time
  • Perform a sort on a String value if it is a Date or Date Time?
  • how can I dynamically change the mapping during runtime so I can perform such a dynamic sorting
  • How can I do this in plain Elastic Search and how can I translate this to Java?

I can figure whether the value is going to be a DateTime or a `String' in Java but I have no idea how to translate this to Elastic Search in Java.

User object return from Elastic Search after a simple query:

User:
{
    "id": 1,
     "extraDataValues": [
        {
            "id": 1,
            "extraDataValueObject": {
                "id": 10,
                "label": "Metadata Date",
                "displayable": true
            },
            "value": "01/01/2015 00:00:00"
        },
        {
            "id": 2,
            "extraDataValueObject": {
                "id": 11,
                "label": "Metadata TextBox",
                "displayable": true
            },
            "value": "aaaa"
        }
    ],
    "username": "johnDoe",
    "firstName": "John",
    "surname": "Doe",
    "email": "[email protected]",
    "fullName": "John Doe"

    "type": {
        "id": 2,
        "name": "Blogger",
        "active": true,
    },
    "club": {
        "id": 2,
        "name": "Photography",
    }
},
{
    "id": 2,
     "extraDataValues": [
        {
            "id": 3,
            "extraDataValueObject": {
                "id": 10,
                "label": "Metadata Date",
                "displayable": true
            },
            "value": "01/01/2015 00:01:11"
        },
        {
            "id": 4,
            "extraDataValueObject": {
                "id": 11,
                "label": "Metadata TextBox",
                "displayable": true
            },
            "value": "zzzz"
        }
    ],
    "username": "marySmith",
    "firstName": "Mary",
    "surname": "Mary",
    "email": "[email protected]",
    "fullName": "Mary Smith"

    "type": {
        "id": 2,
        "name": "Moderator",
        "active": true,
    },
    "club": {
        "id": 2,
        "name": "Yoga",
    }
},
{
    "id": 3,
    "extraDataValues": [
        {
            "id": 5,
            "extraDataValueObject": {
                "id": 10,
                "label": "Metadata Date",
                "displayable": true
            },
            "value": "02/02/2015 00:01:11"
        },
        {
            "id": 6,
            "extraDataValueObject": {
                "id": 11,
                "label": "Metadata TextBox",
                "displayable": true
            },
            "value": "bbbb"
        }
    ],
    "username": "joeBloggs",
    "firstName": "Joe",
    "surname": "Bloggs",
    "email": "[email protected]",
    "fullName": "Joe Bloggs"

    "type": {
        "id": 3,
        "name": "Admin",
        "active": true,
    },
    "club": {
        "id": 3,
        "name": "Cycling",
    }
}

User Mapping Document:

    {
    "User": {
       "properties": {
            "type": {
                "properties": {
                    "id": {
                        "type": "long"
                    },
                    "name": {
                        "type": "string",
                        "index":    "not_analyzed",
                        "fields": {
                            "raw_lower_case": { 
                                "type":  "string",
                                "analyzer": "case_insensitive"
                            }
                        }
                    }
                }
            },
            "fullName": {
                "type": "string",
                "index":    "not_analyzed",
                "fields": {
                    "raw_lower_case": { 
                        "type":  "string",
                        "analyzer": "case_insensitive"
                    }
                }
            },              
            "username": {
                "type": "string",
                "index":    "not_analyzed",
                "fields": {
                    "raw_lower_case": { 
                        "type":  "string",
                        "analyzer": "case_insensitive"
                    }
                }
            },
            "email": {
                "type": "string",
                "index":    "not_analyzed",
                "fields": {
                    "raw_lower_case": { 
                        "type":  "string",
                        "analyzer": "case_insensitive"
                    }
                }
            },
            "firstName": {
                "type": "string",
                "index":    "not_analyzed",
                "fields": {
                    "raw_lower_case": { 
                        "type":  "string",
                        "analyzer": "case_insensitive"
                    }
                }
            },
            "surname": {
                "type": "string",
                "index":    "not_analyzed",
                "fields": {
                    "raw_lower_case": { 
                        "type":  "string",
                        "analyzer": "case_insensitive"
                    }
                }
            },
            "id": {
                "type": "long"
            },
            "metadataFieldValues": {
                "type": "nested",
                "properties": {
                    "metadataFieldDefinition": {
                        "properties": {
                            "id": {
                                "type": "long"
                            },
                            "label": {
                                "type": "string"
                            },
                            "displayable": {
                                "type": "boolean"
                            }
                        }
                    },
                    "value": {
                        "type": "string",
                        "index":    "not_analyzed",
                        "fields": {
                            "raw_lower_case": { 
                                "type":  "string",
                                "analyzer": "case_insensitive"
                            }
                        }
                    }
                }
            },
            "club": {
                "properties": {
                    "id": {
                        "type": "long"
                    },
                    "name": {
                        "type": "string",
                        "index":    "not_analyzed",
                        "fields": {
                            "raw_lower_case": { 
                                "type":  "string",
                                "analyzer": "case_insensitive"
                            }
                        }
                    }
                }
            },      
        }
    }
}

EDIT:::

I took a look at these questions: - ElasticSeach - Sorting on dates - Elasticsearch sort by single nested document key in array

Am I heading in the right direction?

1 Answer 1

0

A property must have the same type/mapping in all documents in Elasticsearch. You cannot change it later.

Once you index extraDataValues.value as a String it will remain a String and it must be a String in all documents.

That being said, maybe you could have a look at script-based sorting. However, it seems that even in that case you cannot generate a script field of a new type.

You might have to consider a different modelling - index a date as a date type in an additional field.

By the way, there are other engines, such as SIREn which let you have different types for the same property in different documents. Disclaimer: I currently work for a company that develops SIREn.

Sign up to request clarification or add additional context in comments.

2 Comments

Hi, thank you for an answer. I checked out stackoverflow.com/questions/10415494/… and stackoverflow.com/questions/18405195/…. Do you think that is feasible for my scenario or am I heading in the wrong direction?
That would mean creating a normal date field if I understand correctly. That's what I suggested and it could probably be made to work.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.