Python, empty file after csv writer.. again

Question

My python program loops through a bunch of csv-files, read them, and write specific columns in the file to another csv file. While the program runs, i can see the files being written in the correct manner, but once the program is finished, all the files i've just written become empty.

The solution to all the other similar threads seems to be closing the file you write to properly, but i cant seem to figure out what im doing wrong. Anyone?

import os
import csv

def ensure_dir(f):
    d = os.path.dirname(f)
    if not os.path.exists(d):
        os.makedirs(d)

readpath = os.path.join("d:\\", "project")
savepath=os.path.join("d:\\", "save")
ensure_dir(savepath)
contents_1=os.listdir(readpath)
for i in contents_1[1:len(contents_1)]:
    readpath_2=os.path.join(readpath, i)
    if os.path.isdir(readpath_2)== True :
        contents_2=os.listdir(readpath_2)
        for i in contents_2:
            readpath_3=os.path.join(readpath_2, i)
            if os.path.isfile(readpath_3)== True :
                savefile=savepath + "\\" + i
                savefile = open(savefile, 'wb')
                writer = csv.writer(savefile, delimiter=';')
                readfile=open(readpath_3, 'rb')
                reader = csv.reader(readfile, delimiter=';')
                try:
                    for row in reader:
                        writer.writerow([row[0], row[3]])
                except:
                    print(i)
                finally:
                    savefile.close()
                    readfile.close()

You should use try..finally or the with-Statement for a clean file access without these problems. — schlamar
– schlamar, Commented Oct 10, 2011 at 19:03
Not that either of these is your problem, but adding == True to a conditional is redundant (and a bit silly), and using + "\\" + i to build a path instead of using os.path.join breaks compatibility with non-Windows OSes. — Wooble
– Wooble, Commented Oct 10, 2011 at 19:10
@ms4py adding a finally doesnt seem to solve the problem. I was hoping there would be a solution without having to use the with-statement. — Drublic
– Drublic, Commented Oct 10, 2011 at 19:16
You have an except: print(i) in there. Is it printing anything? Also, have you made sure that the files you are reading from contain data? — Steven Rumbalski
– Steven Rumbalski, Commented Oct 10, 2011 at 19:17

Steven Rumbalski · Accepted Answer · 2012-10-09 16:27:39Z

2

savefile=savepath + "\\" + i is the error. If both "d:\\project\a\x.csv" and "d:\\project\b\x.csv" exist, then you will write to savepath + "\\" + i more than once. If the second path as an empty "x.csv", then it would overwrite the result with an empty file.

Try this instead:

import os
import csv

def ensure_dir(f):
    d = os.path.dirname(f)
    if not os.path.exists(d):
        os.makedirs(d)

readpath = os.path.join("d:\\", "project")
savepath = os.path.join("d:\\", "save")

ensure_dir(savepath)

for dname in os.listdir(readpath)[1:]:
    readpath_2 = os.path.join(dname, fname)
    if not os.path.isdir(readpath_2):
        continue
    for fname in os.listdir(readpath_2)
        fullfname = os.path.join(readpath_2, fname)
        if not os.path.isfile(fullfname):
            continue
        savefile = open(savepath + "\\" + dname + "_" + fname, wb)
        writer = csv.writer(savefile, delimiter=';')
        readfile=open(fullfname, 'rb')
        reader = csv.reader(readfile, delimiter=';')
        try:
            for row in reader:
                writer.writerow([row[0], row[3]])
        except:
            print(i)
        finally:
            savefile.close()
            readfile.close()

This code could be greatly improved with os.walk

edited Oct 9, 2012 at 16:27

answered Oct 10, 2011 at 20:37

Steven Rumbalski

45.7k10 gold badges96 silver badges125 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Drublic Over a year ago

Thank you, works like a charm. And thanks for the tip on os.walk.

Zack Bloom · Accepted Answer · 2011-10-10 19:17:53Z

0

Quoting from the python documentation:

If csvfile is a file object, it must be opened with the ‘b’ flag on platforms where that makes a difference.

Change the 'w' and 'r' flags to 'wb' and 'rb'.

answered Oct 10, 2011 at 19:17

Zack Bloom

8,4372 gold badges23 silver badges27 bronze badges

Comments

John Machin · Accepted Answer · 2011-10-10 20:41:52Z

0

(1) Your outer loop AND your inner loop both use i as the loop variable. This has no hope of (a) being understood by a human (b) working properly.

(2) except: print(i) ... What??? I'd suggest you remove the try/except and fix any bugs that you come across.

answered Oct 10, 2011 at 20:41

John Machin

83.2k12 gold badges147 silver badges193 bronze badges

Collectives™ on Stack Overflow

Python, empty file after csv writer.. again

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related