Python CSV function outputs blank CSV when not enough rows?

Question

I have a function which takes a list of custom objects, conforms some values then writes them to a CSV file. Something really strange is happening in that when the list only contains a few objects, the resulting CSV file is always blank. When the list is longer, the function works fine. Is it some kind of weird anomaly with the temporary file perhaps?

I have to point out that this function returns the temporary file to a web server allowing the user to download the CSV. The web server function is below the main function.

def makeCSV(things):
    from tempfile import NamedTemporaryFile
    # make the csv headers from an object
    headers = [h for h in dir(things[0]) if not h.startswith('_')]

    # this just pretties up the object and returns it as a dict
    def cleanVals(item):
        new_item = {}
        for h in headers:
            try:
                new_item[h] = getattr(item, h)
            except:
                new_item[h] = ''
            if isinstance(new_item[h], list):
                if new_item[h]:
                    new_item[h] = [z.__str__() for z in new_item[h]]
                    new_item[h] = ', '.join(new_item[h])
                else:
                    new_item[h] = ''
            new_item[h] = new_item[h].__str__()
        return new_item

    things = map(cleanVals, things)

    f = NamedTemporaryFile(delete=True)
    dw = csv.DictWriter(f,sorted(headers),restval='',extrasaction='ignore')
    dw.writer.writerow(dw.fieldnames)
    for t in things:
        try:
            dw.writerow(t)
            # I can always see the dicts here...
            print t
        except Exception as e:
            # and there are no exceptions
            print e
    return f

Web server function:

    f = makeCSV(search_results)
    response = FileResponse(f.name)
    response.headers['Content-Disposition'] = (
            "attachment; filename=export_%s.csv" % collection)
    return response

Any help or advice greatly appreciated!

You create the file, you write into it, but you don't flush/close it. Therefore there may be data still not written to the disk. — eumiro
– eumiro, Commented Nov 16, 2012 at 7:44
dw.writer.writerow - are you sure that's not causing an error? Is there any particular reason you're using a temporary file, and not a StringIO object or similar, and just returning that? — Jon Clements
– Jon Clements, Commented Nov 16, 2012 at 8:03
I noticed that in cleanVals() you have bare execpt:. These should be avoid because they can make debugging problems even more difficult. Anyway, also consider what would happen if it were passed an item the doesn't have any of the header attributes. — martineau
– martineau, Commented Nov 16, 2012 at 8:23
@eumiro The idea of the temp file is that it closes in fact deletes itself — MFB
– MFB, Commented Nov 16, 2012 at 8:46

Armin Rigo · Accepted Answer · 2012-11-16 13:23:50Z

1

Summarizing eumiro's answer: the file needs to be flushed. Call f.flush() at the end of makeCSV().

answered Nov 16, 2012 at 13:23

Armin Rigo

13.1k41 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python CSV function outputs blank CSV when not enough rows?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related