saving and loading list values?

Question

I have a dataframe where one of the columns contains a list of values:

example: type(df['col_list'].values[0]) = list

I saved this dataframe as csv file (df.to_csv('my_file.csv'))

When I load the dataframe (df = pd.read_csv('my_file.csv')) the column which contains list of values change to string type: type(df['col_list'].values[0]) = str

When converting to list (list(df['col_list'].values[0]) I'm getting list of characters instead of list of values.

How can I save/load dataframe which one of it's columns contains list of values ?

I'm able to save list column as it is in csv. My pandas version: '1.2.3'. What version are you on? — Mayank Porwal
– Mayank Porwal, Commented Feb 20, 2022 at 5:35

Anurag Dhadse · Accepted Answer · 2022-02-20 05:30:40Z

2

Use JSON or HDF file format instead of CSV. CSV file format is really inconvenient for storing a list or a collection of objects.

answered Feb 20, 2022 at 5:30

Anurag Dhadse

1,8732 gold badges15 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

0xSingularity · Accepted Answer · 2022-02-20 05:31:05Z

1

This is due to the table being saved as CSV and serializing the values of the list. The csv format is unable to save the list object as it is. Try saving in another format df.to_pickle('test.df'). You can then read this back into a dataframe with read_pickle

Comments

Raymond Kwok · Accepted Answer · 2022-02-20 05:32:24Z

0

I think Anurag's suggestion is very good. But just in case you want to keep things the way it is, this will do the job

import json
df['col_list'] = df['col_list'].apply(json.loads)

This would work better if you had converted col_list into JSON text before pd.to_csv by

df['col_list'] = df['col_list'].apply(json.dumps)

answered Feb 20, 2022 at 5:32

Raymond Kwok

2,5512 gold badges11 silver badges11 bronze badges

Comments

Krishna · Accepted Answer · 2023-08-01 03:42:11Z

0

Consider saving it as an excel file, if that's an option.

df.to_excel('my_file.xlsx',index=False)

answered Aug 1, 2023 at 3:42

Krishna

4251 gold badge4 silver badges11 bronze badges

Collectives™ on Stack Overflow

saving and loading list values?

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related