Dateutil parse bug in python returns the wrong value

Question

I have looked at many possible ways to parse python times. Using parse seems link the only method that should work. While trying to use datetime.strptime causes an error because %z does not work with python 2.7. But using parse.parse incorrectly recognizes the time zone.

I parse both Fri Nov 9 09:04:02 2012 -0500 and Fri Nov 9 09:04:02 2012 -0800 and get the exact same timestamp in unix time. 1352480642

My version of python 2.7.10
My version of dateutil 1.5

Here is my code that runs the test.

#!/usr/bin/python
import time
from dateutil import parser

def get_timestamp(time_string):
    timing = parser.parse(time_string)
    return time.mktime(timing.timetuple())

test_time1 = "Fri Nov 9 09:04:02 2012 -0500"
test_time2 = "Fri Nov 9 09:04:02 2012 -0800"
print get_timestamp(test_time1)
print get_timestamp(test_time2)

Output

1352480642.0
1352480642.0

Expected output

1352469842.0
1352480642.0

in python 3, I get 1352448242.0 for both lines! are you sure the timezone is properly parsed? — Jean-François Fabre
– Jean-François Fabre ♦, Commented Aug 15, 2016 at 22:02
Not sure at all. I am pulling these strings straight from GIT logs. I am not doing the parsing python is. — Whitecat
– Whitecat, Commented Aug 15, 2016 at 22:03
the fact is: values are different depending on python versions! — Jean-François Fabre
– Jean-François Fabre ♦, Commented Aug 15, 2016 at 22:03
Actually the value is different based on your time zone, as the timezone is being ignored in the string and I believe python is just taking your time zone. As I can see you are probably somewhere in Europe 9 hours ahead because of the time difference you get. — Whitecat
– Whitecat, Commented Aug 15, 2016 at 22:04

Paul · Accepted Answer · 2016-08-15 22:48:15Z

1

This has nothing to do with the parser, you'll see the same behavior just from mktime() alone, since datetime.timetuple() doesn't have any time zone offset information, and mktime() is the inverse of localtime. You can correct this by converting it to localtime before calling timetuple():

from time import mktime
from datetime import datetime
from dateutil import tz

dt_base = datetime(2012, 11, 9, 9, 4, 2)

dt_est = dt_base.replace(tzinfo=tz.tzoffset('EST', -5 * 3600))
dt_pst = dt_base.replace(tzinfo=tz.tzoffset('PST', -8 * 3600))

def print_mktime(dt):
    print(mktime(dt.timetuple()))

# Run in UTC
print_mktime(dt_est)   # 1352469842.0
print_mktime(dt_pst)   # 1352469842.0

# Convert to local time zone first first
print_mktime(dt_est.astimezone(tz.tzlocal())) # 1352469842.0
print_mktime(dt_pst.astimezone(tz.tzlocal())) # 1352480642.0

Note that there is a chart on the documentation for time() (python 2.x docs) that tells you how to convert between these representations:

From                        To                           Use
---------------------------------------------------------------------------
seconds since the epoch   | struct_time in UTC        |  gmtime()
seconds since the epoch   | struct_time in local time |  localtime()
struct_time in UTC        | seconds since the epoch   |  calendar.timegm()
struct_time in local time | seconds since the epoch   |  mktime()

My personal preference would be to convert the parsed date to UTC, in which case calendar.timegm() would be the appropriate function:

from calendar import timegm
def print_timegm(dt):
    print(timegm(dt.timetuple()))

print_timegm(dt_est.astimezone(tz.tzutc())) # 1352469842.0
print_timegm(dt_pst.astimezone(tz.tzutc())) # 1352480642.0

edited Aug 15, 2016 at 22:48

answered Aug 15, 2016 at 22:42

Paul

10.9k14 gold badges57 silver badges94 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Whitecat Over a year ago

I like the idea of converting to local time first. But the data I am working with has date strings like this Fri Nov 9 09:04:02 2012 -0800. Is there anyway I can do this without selecting out each int individually to do datetime(2012, 11, 9, 9, 4, 2)

Paul Over a year ago

@Whitecat I strongly suggest using UTC, not localtime, to avoid DST related issues. I don't understand your objection about selecting out ints. I just manually created the output of the dateutil parser to show you it has nothing to do with the parser. You just need to change the time zone of the output of the parser like I did in the examples before calling timetuple.

Whitecat Over a year ago

My objection to selecting ints manually is I have to write that parser, to identify the different parts of the string. Also DO I have to create a different timetuple for every time zone? As the time zones are any one of multiple different time zones.

Paul Over a year ago

@Whitecat I'm not recommending that you use date literals here. parse returns a timezone aware datetime object. You can use astimezone(tz.tzutc) to convert that to a datetime object in UTC, which you can then convert to a timetuple, which can then be fed to calendar.timegm. The only reason I didn't use dateutil.parser.parse in my example is because your issue has nothing to do with parsing dates.

Collectives™ on Stack Overflow

Dateutil parse bug in python returns the wrong value

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related