Turning Numpy array values to datetime values with List comprehensions Python

Question

I am trying to write a list comprehension where it goes through the ts array and then turn it into readable timestamps. However the dates list comprehension is faulty how would I be able to fix it and get the Expected Output below?

from datetime import datetime
import numpy as np 

ts = np.array([1628997394, 1628997444, 1628997602, 1629006977, 1629007021])
# if you encounter a "year is out of range" error the timestamp
# may be in milliseconds, try `ts /= 1000` in that case
dates=[x for x in ts if ts > 0 datetime.utcfromtimestamp(x).strftime('%Y-%m-%d %H:%M:%S')]

Error:

    dates=[x for x in ts if ts > 0 datetime.utcfromtimestamp(x).strftime('%Y-%m-%d %H:%M:%S')]
                                          ^
SyntaxError: invalid syntax

Expected Output:

[2021-08-15 03:16:34 , 2021-08-15 03:17:24, 2021-08-15 03:20:02 , 2021-08-15 05:56:17 , 2021-08-15 05:57:01]

Mahdi F. · Accepted Answer · 2021-09-05 16:21:33Z

3

You iterate on ts as x for checking in if you should check x NOT check ts and end of for and if when checking condition you need write datetime.utcfromtimestamp(x).strftime('%Y-%m-%d %H:%M:%S') in the first of list that you want create.

try this:

from datetime import datetime
import numpy as np 

ts = np.array([1628997394, 1628997444, 1628997602, 1629006977, 1629007021])

dates=[datetime.utcfromtimestamp(x).strftime('%Y-%m-%d %H:%M:%S') for x in ts if x>0]

Output:

['2021-08-15 03:16:34',
 '2021-08-15 03:17:24',
 '2021-08-15 03:20:02',
 '2021-08-15 05:56:17',
 '2021-08-15 05:57:01']

For checking runtime two solution you can use %timeit and see when you can do your job with list don't use numpy or pandas because use an additional library is not good. (using list 10 times faster than using pandas for this short array)

Runtime:

ts = np.array([1628997394, 1628997444, 1628997602, 1629006977, 1629007021])

%timeit dates=[datetime.utcfromtimestamp(x).strftime('%Y-%m-%d %H:%M:%S') for x in ts if x>0]
# 27.1 µs ± 899 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
%timeit pd.to_datetime(ts, unit='s', errors='coerce').dropna().astype(str).to_numpy()
# 708 µs ± 128 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

edited Sep 5, 2021 at 16:21

answered Sep 5, 2021 at 6:36

Mahdi F.

24.1k5 gold badges25 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mahdi F. Over a year ago

@georgehere I add runtime checking two existing answers and you can see use list is 10 time faster than using pandas.

U13-Forward · Accepted Answer · 2021-09-05 06:40:50Z

2

It could be done more efficiently with pandas:

>>> import pandas as pd
>>> ts = np.array([1628997394, 1628997444, 1628997602, 1629006977, 1629007021])
>>> pd.to_datetime(ts, unit='s', errors='coerce').dropna().astype(str).to_numpy()
array(['2021-08-15 03:16:34', '2021-08-15 03:17:24',
       '2021-08-15 03:20:02', '2021-08-15 05:56:17',
       '2021-08-15 05:57:01'], dtype=object)
>>>

answered Sep 5, 2021 at 6:40

U13-Forward

71.8k15 gold badges100 silver badges125 bronze badges

Comments

Samsul Islam · Accepted Answer · 2021-10-10 06:39:21Z

1

import numpy as np 

import pandas as pd

ts = np.array([1628997394, 1628997444, 1628997602, 1629006977, 1629007021])

ts1 = pd.to_datetime(ts, unit='s', errors= 'coerce')

ts1

solution:

DatetimeIndex(['2021-08-15 03:16:34', '2021-08-15 03:17:24',
               '2021-08-15 03:20:02', '2021-08-15 05:56:17',
               '2021-08-15 05:57:01'],
              dtype='datetime64[ns]', freq=None)

edited Oct 10, 2021 at 6:39

Samsul Islam

2,6192 gold badges20 silver badges24 bronze badges

answered Sep 5, 2021 at 10:23

Yasmin

212 bronze badges

Collectives™ on Stack Overflow

Turning Numpy array values to datetime values with List comprehensions Python

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related