Save multiple dataFrames in a loop using to_pickle

Question

hi i have 4 pandas dataframe: df1, df2 ,df3, df4. What i like to do is iterate (using a for loop) the save of this dataframe using to_pickle. what i did is this:

out = 'mypath\\myfolder\\'

r = [ orders, adobe, mails , sells]
for i in r:
    i.to_pickle( out +   '\\i.pkl')

The command is fine but it does not save every database with his name but overwriting the same databse i.pkl (i think because is not correct my code) It seem it can't rename every database with his name (e.g. for orders inside the for loop orders is saved with the name i.pkl and so on with the orders dataframe involved) What i expect is to have 4 dataframe saved with the name inserted in the object r (so : orders.pkl, adobe.pkl ,mails.pkl, sells.pkl)

How can i do this?

cs95 · Accepted Answer · 2019-12-16 09:27:33Z

5

You can't stringify the variable name (this is not something you generally do), but you can do something simple:

import os

out = 'mypath\\myfolder\\'

df_list = [df1, df2, df3, df4]
for i, df in enumerate(df_list, 1):
    df.to_pickle(os.path.join(out, f'\\df{i}.pkl')

If you want to provide custom names for your files, here is my suggestion: use a dictionary.

df_map = {'orders': df1, 'adobe': df2, 'mails': df3, 'sells': df4}
for name, df in df_map.items():
    df.to_pickle(os.path.join(out, f'\\{name}.pkl')

edited Dec 16, 2019 at 9:27

answered Dec 16, 2019 at 9:08

cs95

406k106 gold badges744 silver badges797 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Parsifal Over a year ago

thank you, what if my df list would contains different names? (e.g: df_lis=[ orders, adobe, mails , sells] )

cs95 Over a year ago

@lucapellerossapelles There is no clean way to do it unless you use a dictionary (see edit).

AnsFourtyTwo Over a year ago

This is true, as DataFrames do not have a name attribute. You must handle the names on your own. A dictionairy is the right choice here. See also this question

Collectives™ on Stack Overflow

Save multiple dataFrames in a loop using to_pickle

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related