How to add a blank row at the end to multiple CSV files with Python

Question

I'm using Python 3 & I am having trouble appending lines from multiple csv files into multiple rows for the Master_Total.csv file. I suspect that it is due to not having a "pre-existing" blank row for each csv file. If this is true how do I add a new blank row to each TOTAL.csv file?

TOTAL.csv file:

GND, 0.1V, 1.0V, REFKelvin,
0.000000, 0.100436, 1.003407,  150318.406250,
[no empty row]

enviro.csv file:

temp [C], pressure [kPa], humidity [%RH]
23.870001, 85.903000, 33.75244
[empty row]

When I run my script I get this:

Master_Total.csv

GND, 0.1V, 1.0V, REFKelvin,
0.000000, 0.100436, 1.003407,  150318.4062500.000000, 0.100764, 1.005011, 100.3399580.000019, 0.100252, 1.002642, 100.214996...

Master_enviro.csv

temp [C], pressure [kPa], humidity [%RH]
23.870001, 85.903000, 33.752441
23.760000, 85.914001, 32.997131
24.040001, 85.879997, 33.134460
...

Code:

import shutil, glob, csv, os, sys



path = r'directory'
Enviro_Files = glob.glob(path +"**/*enviro.csv")
Total_Files = glob.glob(path +"**/*TOTAL.csv")

with open('directory_desktop/Master_enviro.csv', 'wb') as outfile1:
    for i, filename1 in enumerate(Enviro_Files):
        with open(filename1, 'rb') as inputfile1:
            if i != 0:
                inputfile1.readline()
            shutil.copyfileobj(inputfile1, outfile1)
            print(filename1 + " has been imported.")
with open('directory_desktop/Master_TOTAL.csv', 'wb') as outfile2:
    for h, filename2 in enumerate(Total_Files):
        with open(filename2, 'rb') as inputfile2:
            if h != 0:
                inputfile2.readline()
            shutil.copyfileobj(inputfile2, outfile2)
            print(fname2 + " has been imported.")

Martin Evans · Accepted Answer · 2018-01-24 12:16:49Z

1

If you make use of Python's CSV library, you can easily test to ensure a given row has values in it, that way it does not matter if there are empty lines are not, they will be skipped over when writing the master files:

import csv
import glob

def merge_csvs(target_filename, csv_list):
    with open(target_filename, 'w', newline='') as f_master_target:
        csv_master_target = csv.writer(f_master_target)
        write_header = True

        for csv_filename in csv_list:
            with open(csv_filename, 'r', newline='') as f_single:
                csv_single = csv.reader(f_single)
                header = next(csv_single)

                if write_header:
                    csv_master_target.writerow(header)
                    write_header = False

                for row in csv_single:
                    if row:
                        csv_master_target.writerow(row)


path = 'directory'
Enviro_Files = glob.glob(path + "**/*enviro.csv")
Total_Files = glob.glob(path + "**/*TOTAL.csv")

merge_csvs('Master_enviro.csv', Enviro_Files)
merge_csvs('Master_TOTAL.csv', Total_Files)

answered Jan 24, 2018 at 12:16

Martin Evans

46.9k17 gold badges88 silver badges104 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

jjNN Over a year ago

It looks like it would work, but when I run the script it almost does nothing. It'll create the Master files with no data. It seems like the script just stops at line 8 "for csv_filename in csv_list:" I'm not sure why as I don't have much Python experience. I do not get an error, but it will not "print(path)" after that command line.

Martin Evans Over a year ago

path needs to be correctly set. It would need to have a tailing / on it to work correctly. If you did print(Enviro_Files) you will probably get []

Martin Evans Over a year ago

You're welcome! Don't forget to click on the grey tick under the up/down buttons to select the answer as the accepted solution.

Collectives™ on Stack Overflow

How to add a blank row at the end to multiple CSV files with Python

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related