How do I remove blank rows from a CSV file in python?

Question

Good evening! I'm currently working on a python 3 script to create a CSV file with specified columns of data. I'm really close to finalizing it, but I've run into a problem I just can't get past.

Basically, I'm trying to add two values (floats) together from two columns and append them to another column; however, when the program runs into blank lines (strings), everything goes kaput. As far as I know converting the whole csv file to a float isn't possible, so I've decided to just delete these blank lines... how do I do that?

Also, if anyone has any suggestions for a cleaner way to do this I'd be more than glad to hear it!

My code is as follows:

#! python3
# automatedReport.py - Reads and writes a new CSV file with
# Campaign Name, Group Name, Raised from Apr 1st-Apr 30th Total,
# Donation from Apr 1st-Apr 30th Total, and Campaign Total Apr 1st-Apr 30th.

import csv, os, ctypes
MessageBox = ctypes.windll.user32.MessageBoxW
total = 0

# Find out whether or not campaign_monthly_report is present.
if os.path.isfile('campaign_monthly_report.csv'):
    os.makedirs('automatedReport', exist_ok=True)
    print('Organizing campaign_monthly_report.csv...')

    #Read the CSV file.
    with open('campaign_monthly_report.csv', 'r') as csv_file:
        csv_reader = csv.DictReader(csv_file)

        #Write out the CSV file.
        with open(os.path.join('automatedReport', 'automated_campaign_monthly_report.csv'), 'w', newline='') as new_file:
            fieldnames = ['Campaign Name','Group Name','Raised from Apr 1st-Apr 30th Total','Donation from Apr 1st-Apr 30th Total', 'Campaign Total Apr 1st-Apr 30th']

            csv_writer = csv.DictWriter(new_file, fieldnames=fieldnames)

            csv_writer.writeheader()

            #Sloopy code, I know. I'm a bit new to this.
            for line in csv_reader:
                del line['Contact Name']
                del line['Contact Phone']
                del line['Contact Email']
                del line['Sign Ups']
                del line['Active Members']
                del line['% Active Members']
                del line['Raised upto Mar 31st 1st Time']
                del line['Raised upto Mar 31st Everyday']
                del line['Raised upto Mar 31st Total']
                del line['Raised upto Mar 31st Target']
                del line['Donation upto Mar 31st Group']
                del line['Donation upto Mar 31st PF']
                del line['Donation upto Mar 31st Total']
                del line['Additional $ Applied upto Mar 31st']
                del line['Raised from Apr 1st-Apr 30th 1st Time']
                del line['Raised from Apr 1st-Apr 30th Everyday']
                del line['Raised from Apr 1st-Apr 30th Target']
                del line['Donation from Apr 1st-Apr 30th Group']
                del line['Donation from Apr 1st-Apr 30th PF']
                del line['Additional $ Applied from Apr 1st-Apr 30th Total']
                del line['Date Joined']

                total = float(line['Raised from Apr 1st-Apr 30th Total']) + float(line['Donation from Apr 1st-Apr 30th Total'])
                csv_writer.writerow(line)
                print (total)

            MessageBox(None, 'Process Complete. Locate ouput in the automatedReport folder.', ' Success!', 0)
else:
    MessageBox(None, 'campaign_monthly_report not found!', ' Error!', 0)

Error message:

Traceback (most recent call last):
File "C:\Users\Mende\Desktop\Automated Campaign\automatedReport.py", line 51, in <module>
total = float(line['Raised from Apr 1st-Apr 30th Total']) + float(line['Donation from Apr 1st-Apr 30th Total'])
ValueError: could not convert string to float: >>>

Ever heard of Pandas ? pandas.pydata.org/pandas-docs/stable/generated/… — Mohamed AL ANI
– Mohamed AL ANI, Commented May 31, 2018 at 17:08
I took a look through it and it seemed to be what I was looking for, but I was hoping there was a way to do this without downloading an external library. No chance that there is a way? — Gabriel Mendez
– Gabriel Mendez, Commented May 31, 2018 at 17:14
Do you have restrictions on your libraries ? Otherwise it's simple : pip install pandas And it can do what you want in like 2 or 3 lines of codes. In any case, I don't know the packages you are using so I cant help you with that. — Mohamed AL ANI
– Mohamed AL ANI, Commented May 31, 2018 at 17:20
Please define "kaput" and show us the error message and line number it occurs on. — Larry Lustig
– Larry Lustig, Commented May 31, 2018 at 17:22

Serge Ballesta · Accepted Answer · 2018-05-31 18:38:04Z

1

The csv module only expects its file to be an iterator that returns a new line on each iteration. It it trivial to define an iterator that will filter out blank lines:

def no_blank(fd):
    try:
        while True:
            line = next(fd)
            if len(line.strip()) != 0:
                yield line
    except:
        return

You can just use it to filter blank lines out of the original file object:

...
#Read the CSV file.
with open('campaign_monthly_report.csv', 'r') as csv_file:
    csv_reader = csv.DictReader(no_blank(csv_file))
    ...

answered May 31, 2018 at 18:38

Serge Ballesta

150k13 gold badges137 silver badges267 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How do I remove blank rows from a CSV file in python?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related