How can I use read_csv for each csv files even if it is empty ? Python Pandas

Question

I have this piece of code to try to execute this program for each csv file:

import pandas as pd
import os

directory_path_csv = r'CSV_Files'

for file in os.listdir(directory_path_csv):
    if file.endswith(".csv"):
        filename = file
        df = pd.read_csv(filename, usecols= ['date','time', 'toto','tata'], parse_dates=[['date', 'time']])

But I have this error: FileNotFoundError: [Errno 2] File File1.csv does not exist: 'File.csv' and don't understand why ? In the csv folder, I can have csv file with no data just the column's names.

Thanks for your time !

Your file is in the folder directory_path_csv so you need to do os.path.join(directory_path_csv, file) — user5386938
– user5386938, Commented Oct 20, 2020 at 17:10
@JustinEzequiel I can read the csv files, but when the program execute a certain one with just one line (columns names) since it cannot do the rest, the program stops without do the rest — Zebra125
– Zebra125, Commented Oct 20, 2020 at 17:28
Your error message (FileNotFoundError: [Errno 2] File File1.csv does not exist: 'File.csv') does not match your claims. — user5386938
– user5386938, Commented Oct 20, 2020 at 17:38

zabop · Accepted Answer · 2020-10-20 18:03:37Z

1

# necessary imports
import pandas as pd
import glob

Lets say you have some non-empty csvs:

for x in range(10):
    pd.DataFrame(['exampleDf'+repr(x)]).to_csv('nonempty'+repr(x)+'.csv')

and an empty one:

# this is a shell command:
touch empty.csv

Put all csv filenames to a list

csvs=glob.glob('*csv')

You can loop through them catching EmptyDataErrors:

for csv in csvs:
    try:
        pd.read_csv(csv)
    except pd.errors.EmptyDataError:
        pass # or do whatever you want with empty csvs

EDIT: if csv has header but no data

If some csvs have headers but no data, some has data as well. Recreate situation:

for x in range(10):
    pd.DataFrame({'header':[1,2]}).to_csv('nonempty'+repr(x)+'.csv')
for x in range(10):
    pd.DataFrame({'header':[]}).to_csv('empty'+repr(x)+'.csv')

Then can do:

csvs=glob.glob('*csv')
for csv in csvs:
    df = pd.read_csv(csv)
    if len(df.index)>0:
        # then df is non-empty, do whatever with it
        print(df)
    else:
        # then df is empty
        print(df)

edited Oct 20, 2020 at 18:03

answered Oct 20, 2020 at 17:35

zabop

8,1124 gold badges56 silver badges112 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Zebra125 Over a year ago

thanks, I put read_csv et my treatments in the section try but when the program read another csv file, the first line is the name of the column of my df so it passed in the section try not in the except section

zabop Over a year ago

So your csv is not entirely empty, it has column names, but no data?

Zebra125 Over a year ago

exactly ! Sorry for the misunderstanding

Collectives™ on Stack Overflow

How can I use read_csv for each csv files even if it is empty ? Python Pandas

1 Answer 1

EDIT: if csv has header but no data

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

EDIT: if csv has header but no data

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related