Converting DateTime stamps into unix with Python

Question

I can't seem to get a time stamp into a unix timestamp format and assume I'm missing some points. Looking for some help.

I have a column within a df that is a datetime object in the format: YYYY-MM-DD HH:MM:SS and I need a new column with the unix stamp.

I'm bringing the csv into python with:

gps = filepath here   
dateparse= lambda x: pd.datetime.strptime(x, '%Y%m%d %H:%M:%S')
gps_dat = pd.read_csv(gps, parse_dates=['date_stamp'],date_parser=dateparse)

Anytime I try to change this column into a unix stamp I'm getting an error about wrong format or "datetime.datetime" has no attribute "datetime" with this:

gps_dat['unix']=datetime.datetime(gps_dat['date_stamp'])

Should I be using

calendar.timegm(tuple)

I'm still learning so any help would be greatly appreciated!

you probably imported datetime.datetime instead of just datetime, so when you call datetime.datetime you're really calling datetime.datetime.datetime, which isn't defined. Sorry to sound a bit like Xzibit, but... :) — michele b
– michele b, Commented May 9, 2016 at 13:45
I think that definitely fixed part of the problem thanks! When I use the: gps_dat['unix']=datetime.datetime(gps_dat['date_stamp']) Now I get error "cannot convert the series to <class 'int>" — Yolo_chicken
– Yolo_chicken, Commented May 9, 2016 at 13:49
gps_dat['date_stamp'] is a column within my dataframe: 2015-04-16 15:23:20 — Yolo_chicken
– Yolo_chicken, Commented May 9, 2016 at 14:20
I mean, is it already a datetime object, or is it a string? If it's already a datetime object, you just need to use gps_dat['unix'] = gps_dat['date_stamp']. If it's a string instead, use gps_dat['unix'] = datetime.strptime(gps_dat['date_stamp'], '%Y-%m-%d %H:%M:%S') — michele b
– michele b, Commented May 9, 2016 at 14:26

MaxU - stand with Ukraine · Accepted Answer · 2016-05-09 16:29:36Z

1

UPDATE: if you want to convert datetime column to UNIX timestamp:

gps_dat['unix']=gps_dat['date_stamp'].astype(np.int64) // 10**9

NOTE: but it must be of datetime type, not string/object

Old answer: parsing from UNIX timestamp to datetime

try to change your parser function like this:

dateparse= lambda x: pd.to_datetime(x, unit='s')

this will instruct to_datetime() that you're using UNIX timestamp format

edited May 9, 2016 at 16:29

answered May 9, 2016 at 15:57

MaxU - stand with Ukraine

212k37 gold badges402 silver badges436 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Yolo_chicken Over a year ago

When I parse with this function I'm still getting: 2015-04-21 23:32:25 format.

Yolo_chicken Over a year ago

THIS WORKED!! Thanks to everyone for their input. It's very appreciated.

jezrael · Accepted Answer · 2016-05-09 16:52:57Z

1

You can use faster solution with list comprehension:

print gps_dat
  nam  code date1          date_stamp
0   a     1   1/1 2012-10-08 18:15:05
1   b     3   3/4 2012-10-08 18:15:05

gps_dat['unix'] = [t.value // 10 ** 9 for t in gps_dat['date_stamp']]
print gps_dat
  nam  code date1          date_stamp        unix
0   a     1   1/1 2012-10-08 18:15:05  1349720105
1   b     3   3/4 2012-10-08 18:15:05  1349720105

Timings:

In [46]: %timeit gps_dat['date_stamp'].astype(np.int64) // 10**9
1000 loops, best of 3: 204 µs per loop

In [47]: %timeit [t.value // 10 ** 9 for t in gps_dat['date_stamp']]
The slowest run took 4.99 times longer than the fastest. This could mean that an intermediate result is being cached 
10000 loops, best of 3: 24.2 µs per loop

edited May 9, 2016 at 16:52

answered May 9, 2016 at 15:55

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Comments

Community · Accepted Answer · 2017-05-23 12:23:51Z

0

Edit: I've never used Pandas, but it looks like this is the kind of function that you should call:

gps_dat['unix'] = gps_dat.apply(lambda row: time.mktime(row['date_stamp'].timetuple()), axis=col_number)

where col_number is the index of your date_stamp column (assuming it was correctly parsed as a datetime).

Original answer (when I didn't know Pandas was involved):

Replace that

gps_dat['unix']=datetime.datetime(gps_dat['date_stamp'])

line with

gps_dat['unix'] = time.mktime(gps_dat['date_stamp'].timetuple())

And add

import time

to your imports. Note that some considerations apply to time zones, so according to your requirements you might want to add some logic to, say, convert to UTC.

edited May 23, 2017 at 12:23

CommunityBot

11 silver badge

answered May 9, 2016 at 14:53

michele b

1,8652 gold badges20 silver badges28 bronze badges

3 Comments

Yolo_chicken Over a year ago

I got: " 'Series object has no attribute 'timetuple' "

michele b Over a year ago

ok so that's a pandas.Series, not a datetime. I added a pandas tag so that people familiar with the library can help. You might want to check what gps_dat['date_stamp'].data contains by printing it out

Yolo_chicken Over a year ago

Using this I'm getting a "unhashable type:list" error. Thanks for all your help, by the way.

Collectives™ on Stack Overflow

Converting DateTime stamps into unix with Python

3 Answers 3

2 Comments

Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related