Recursive method to read multiple csv files in Pandas

Question

I would like to map each file to its unique dataframe. Something like:

df1 = pd.read_csv(file1, ...)
df2 = pd.read_csv(file2, ...)
...
dfn = pd.read_csv(filen, ...)

for this I did the following:

files = glob.glob("*.csv")
for i in range(len(files)):
    df_i = pd.read_csv(files[i],...)

I get no error. However, I cannot access any of the dataframes. When I type df_1 I get "undefined". What's going on?

RJT · Accepted Answer · 2014-07-11 14:01:30Z

1

What you are doing is assigning ds_i to a new DataFrame over and over again.

A possible solution would be to create a list of DataFrames:

for i in range(len(files)):
    dfList = list(pd.read_csv(files[i],...))

A better solution is to use a list comprehension:

dfList = [pd.read_csv(files[i]) for i in range(len(files))]

An even better solution is to drop the range:

dfList = [pd.read_csv(file) for file in files]

edited Jul 11, 2014 at 14:01

answered Jul 11, 2014 at 13:50

RJT

3842 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Rohit Over a year ago

but I am indexing over i, does not make it unique? The above solution does not work. It creates a list of length 1.

RJT Over a year ago

df_i is just the text df_i. df[i] would be indexing over i, but you don't use indexes when assigning values in python: you can either append() to an existing list, or use the list() method.

RJT Over a year ago

updated the answer with a more pythonic answer using list comprehension.

Collectives™ on Stack Overflow

Recursive method to read multiple csv files in Pandas

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related