how to NOT read_csv if csv is empty

Question

Using Python 2.7 and Pandas

I have to parse through my directory and plot a bunch of CSVs. If the CSV is empty, the script breaks and produces the error message:

pandas.io.common.EmptyDataError: No columns to parse from file

If I have my file paths stored in

file_paths=[]

how do I read through each one and only plot the non empty CSVs? If I have an empty dataframe defined as df=[] I attempt the following code

for i in range(0,len(file_paths)):
   if pd.read_csv(file_paths[i] == ""):
      print "empty"
   else df.append(pd.read_csv(file_paths[i],header=None))

iD_Sgh · Accepted Answer · 2023-02-01 11:01:30Z

41

I would just catch the appropriate exception, as a catch all is not recommended in python:

import pandas.io.common

for i in range(0,len(file_paths)):
   try:
      pd.read_csv(file_paths[i])
   except pandas.errors.EmptyDataError:
      print file_paths[i], " is empty"

edited Feb 1, 2023 at 11:01

iD_Sgh

1201 silver badge6 bronze badges

answered Feb 9, 2017 at 17:38

Zeugma

32.3k9 gold badges73 silver badges85 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

NandaKishore.Rao Over a year ago

This seems to have changed since this solution was posted. Now need to use pandas.errors instead of pandas.io.common

Nikhil VJ · Accepted Answer · 2018-07-08 09:43:02Z

32

Note, as of pandas 0.22.0 (that I can be sure of) , the exception raised for empty csv is pandas.errors.EmptyDataError. And if you're importing pandas like import pandas as pd, then use pd instead of pandas.

If your csv filenames are in an array manyfiles, then

import pandas as pd
for filename in manyfiles:
    try:
        df = pd.read_csv(filename)

    except pd.errors.EmptyDataError:
        print('Note: filename.csv was empty. Skipping.')
        continue # will skip the rest of the block and move to next file

    # operations on df

I'm not sure if pandas.io.common.EmptyDataError is still valid or not. Can't find it in reference docs. And I also would advise against the catch-all except: as you won't be able to know if it's something else causing the issue.

answered Jul 8, 2018 at 9:43

Nikhil VJ

6,2527 gold badges38 silver badges62 bronze badges

1 Comment

Geoff D Over a year ago

Pandas version 1.0 moved error handling to pandas.errors.EmptyDataError.

Community · Accepted Answer · 2017-05-23 12:09:05Z

6

You can use the in built try and except syntax to skip over files that return you an error, as follows:

Described here: Try/Except in Python: How do you properly ignore Exceptions?

for i in range(0,len(file_paths)):
   try:
       pd.read_csv(file_paths[i])
       ### Do Some Stuff
   except:
       continue
       # or pass

This will attempt to read each file, and if unsuccessful continue to the next file.

edited May 23, 2017 at 12:09

CommunityBot

11 silver badge

answered Feb 9, 2017 at 17:35

Chuck

3,8827 gold badges45 silver badges82 bronze badges

2 Comments

Chuck Over a year ago

You are welcome: I am glad this solved your problem. In the interests of the dissemination of proper standards: @Boud is right though - his answer is less general.

kuanb Over a year ago

I think such a broad except is not a best practice, the answer by @Boud is more appropriate.

Collectives™ on Stack Overflow

how to NOT read_csv if csv is empty

3 Answers 3

1 Comment

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related