Elasticsearch date format parsing error

Question

I am trying to specify a date format for an Elasticsearch field according to ISO 8601 as:

YYYY-MM-DDThh:mm:ssTZD

I put the field mapping like so:

"properties": {
                        "startDate": {
                            "type": "date",
                            "format": "YYYY-MM-DD'T'hh:mm:ss'TZD'"
                        }
              }

When I try to insert a document with this field's value as: "startDate": "2018-01-10T07:07:07+01:00", I get this error:

"type": "mapper_parsing_exception",
        "reason": "failed to parse [afield.startDate]",
        "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "Invalid format: \"2018-01-10T07:07:07+01:00\" is malformed at \"+01:00\""
        }

Is there something wrong in the date I am inserting? I'm following the example given in this W3C guide (https://www.w3.org/TR/NOTE-datetime):

Complete date plus hours, minutes and seconds:

YYYY-MM-DDThh:mm:ssTZD (eg 1997-07-16T19:20:30+01:00)

Was any of the answers helpful? :)

Antonio Val
– Antonio Val

2018-03-15 06:16:43 +00:00
Commented Mar 15, 2018 at 6:16 — Antonio Val
– Antonio Val, Commented Mar 15, 2018 at 6:16

syntax_fire · Accepted Answer · 2018-02-21 23:19:54Z

2

Elasticsearch datetime formats can be found here: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html

You want a datetime with no milliseconds. To do that, use the following:

 PUT my_index
 {
     "settings" : {
         "number_of_shards" : 1
     },
     "mappings" : {
         "type1" : {
           "properties": {
               "startDate": {
                   "type": "basic_date_time_no_millis",
                        "format": "yyyyMMdd'T'HHmmssZ"
                            }
                         }
                     }    
                  }
     }

Timezone should be handled in the timestamp, and can be pulled from a query like:

 GET /my_index/_search
 {
     "query": {
         "range" : {
             "timestamp" : {
                 "gte": "2018-01-10 07:07:07", 
                 "lte": "now", 
                 "time_zone": "+01:00"
             }
         }
     }
 }

answered Feb 21, 2018 at 23:19

syntax_fire

313 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Antonio Val Over a year ago

Not sure if that works because that format doesn't use hyphens. Also, having to specify each time time_zone doesn't look very convenient in this case.

syntax_fire Over a year ago

This is what is suggested as best practice from documentation. I agree, it does seem weird but it does work, I tried it prior to posting.

Antonio Val · Accepted Answer · 2018-02-22 07:04:19Z

2

For custom date formats in Elasticsearch you have to check the Joda-time documentation.

In your case you can try with yyyy-MM-dd'T'HH:mm:ssZ:

"properties": {
  "startDate": {
    "type": "date",
    "format": "yyyy-MM-dd'T'HH:mm:ssZ"
  }
}

With this you should be able to insert dates and make searches using the dates like the one you used in your example: 2018-01-10T07:07:07+01:00.

edited Feb 22, 2018 at 7:04

answered Feb 22, 2018 at 6:56

Antonio Val

3,3701 gold badge16 silver badges27 bronze badges

Comments

misterone · Accepted Answer · 2020-03-10 21:51:35Z

0

For all the people out there that have a date in the format :

2016-08-14T00:00:00+02:00

or :

yyyy-MM-ddTHH:mm:ss+HH:MM

You might consider using this format in elasticsearch :

format: "yyyy-MM-dd'T'HH:mm:ss+02:00"

answered Mar 10, 2020 at 21:51

misterone

3713 silver badges11 bronze badges

Collectives™ on Stack Overflow

Elasticsearch date format parsing error

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related