Converting Pandas Timestamp to python native Datetime not working

Question

I am trying to convert a DateTimeIndex with Pandas Timestamps into an Index with Python-native Datetime objects.

Here is my reproducible example:

import pandas as pd
import numpy as np

length = 5
dateIndex = pd.date_range(start='1/1/2018', periods=length, freq="H")

df = pd.DataFrame(np.random.randint(0,length,size=(length, 6)), columns=["Open","High","Low","Close","Volume","OpenInterest"], index=dateIndex)

df.index[0]
## returns Timestamp('2018-01-01 00:00:00', freq='H')

df.index.to_pydatetime()
## returns
array([datetime.datetime(2018, 1, 1, 0, 0),
       datetime.datetime(2018, 1, 1, 1, 0),
       datetime.datetime(2018, 1, 1, 2, 0),
       datetime.datetime(2018, 1, 1, 3, 0),
       datetime.datetime(2018, 1, 1, 4, 0)], dtype=object)

BUT:

df.index = df.index.to_pydatetime()
type(df.index[0])
## returns pandas._libs.tslibs.timestamps.Timestamp

Why am I not seeing a native DatetimeObject?

Iñigo González · Accepted Answer · 2020-09-16 11:33:17Z

Pandas uses its own timestamp object for their indexes (makes sense because the native python datetime does not implement allt the interface pandas needs).

You can check it with this example:

>>> df.index.map(lambda x: f'{x} as {type(x)}')
Index(['2018-01-01 00:00:00 as <class 'pandas._libs.tslibs.timestamps.Timestamp'>',
       '2018-01-01 01:00:00 as <class 'pandas._libs.tslibs.timestamps.Timestamp'>',
       '2018-01-01 02:00:00 as <class 'pandas._libs.tslibs.timestamps.Timestamp'>',
       '2018-01-01 03:00:00 as <class 'pandas._libs.tslibs.timestamps.Timestamp'>',
       '2018-01-01 04:00:00 as <class 'pandas._libs.tslibs.timestamps.Timestamp'>'],
      dtype='object')

So, what happens when try to set the index with pydatetime objects like this?

df.index = df.index.to_pydatetime()

First python calls df.index.to_pydatetime()
And you have an ndarray of datetime objects on the right hand side of the assignment.
Then python calls df.index.equals ( returned ndarray ) which performs the type conversion.

Collectives™ on Stack Overflow

Converting Pandas Timestamp to python native Datetime not working

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related