Reading different tables from a directory into an array of separate dataframes

Question

I have several CSV files (tables) in a directory (all tables have different schemas) and want to loop over the files and read each table into a separate dataframe.

Is there any way to do this in Python/Pandas - to read the different tables into a dataframe array? How multiple tables (with different schema) be imported into a multiple separate data frames?

Guys its not really a duplicate since the other q says '# Assemble all of the data files into a single DataFrame & add a year field' - which is not what this question asks... — user7289
– user7289, Commented Aug 8, 2013 at 13:07

richie · Accepted Answer · 2013-08-08 13:30:59Z

2

try this;

import os
import pandas as pd
import glob
os.chdir("E:/") # change this to the directory where your csv files are stored
csv_files = {} # we store the dataframes in a dictionary
for file in glob.glob("*.csv"): 
    csv_files[file] = pd.read_csv(file)

for dataframe in csv_files.values():
    print dataframe

edited Aug 8, 2013 at 13:30

answered Aug 8, 2013 at 12:21

richie

18.7k19 gold badges53 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

user7289 Over a year ago

Thanks for this. How does this work exactly? It seems to read a variable into the df and then overwrite it? I am probably not reading it correctly (while looping over the CSV files in the dir).

richie Over a year ago

I've just removed df = 'data' + str(i). It dint make sense there. Correct. The df is being overwritten.

Andy Hayden Over a year ago

I think this is a good solution. Would be nicer without the df variable (and the printing). Some explanation of glob.glob would make this excellent :)

user7289 Over a year ago

Thanks for this @Richie - I get error 'Error tokenizing data. C error: Expected 24 fields in line 6927, saw 26' - how do I overcome this?

Collectives™ on Stack Overflow

Reading different tables from a directory into an array of separate dataframes

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related