How to get numpy data dtype when reading csv?

Question

I have a dataframe containing numpy array.

I saved it to a csv file.

After loading the csv file, I found that the column containing the numpy array has dtype string.

How to convert it to numpy array using read_csv?

import pandas as pd
import numpy as np

df = pd.DataFrame(columns = ['name', 'sex'])
df.loc[len(df), :] = ['Sam', 'M']
df.loc[len(df), :] = ['Mary', 'F']
df.loc[len(df), :] = ['Ann', 'F']

#insert np.array
df['data'] = ''
df['data'][0] = np.array([2,5,7])
df['data'][1] = np.array([6,4,8])
df['data'][2] = np.array([9,2,1])

#save to csv file
df.to_csv('data.csv', index =False)
#load csv file
df2 = pd.read_csv('data.csv')#data column becomes string, how to change it to np.array?

How do you want to save the NumPy array in the CSV file, between other fields? — 9769953
– 9769953, Commented Aug 6, 2018 at 6:55
Use numpy.fromstring(text, sep=' ') Define seperator as well. — Upasana Mittal
– Upasana Mittal, Commented Aug 6, 2018 at 6:58

shivsn · Accepted Answer · 2018-08-07 04:57:01Z

1

Its a workaround:

In [114]: df2['data'] = df2.data.str.split(' ',expand=True).replace('\[|\]','',regex=True).astype(int).values.tolist()

In [115]: df2['data'] = [np.array(i) for i in df2.data]

In [116]: df2.loc[0,'data']
Out[116]: array([2, 5, 7])

answered Aug 7, 2018 at 4:57

shivsn

7,9141 gold badge28 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sreekiran A R · Accepted Answer · 2018-08-06 07:06:42Z

0

Pandas has only 7 datatypes: Object, float, int, bool, datetime, timedelta and category. So list, string, array etc. is treated as object datatype only. You can read more about it in http://pbpython.com/pandas_dtypes.html You can use astype function to convert between these datatypes only.

edited Aug 6, 2018 at 7:06

answered Aug 6, 2018 at 7:01

Sreekiran A R

3,4253 gold badges25 silver badges45 bronze badges

Collectives™ on Stack Overflow

How to get numpy data dtype when reading csv?

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related