csv writer leaves empty output file when the python script crashes

Question

I have a script that runs some analysis on files, then write the results to a CSV file. In pseudo code:

while (there are still files to analyze): 
  do_analysis
  writer.writerow({... lots of columns names and the analysis results data... })

If the script completes, the data is indeed in the csv file. But if the script crashes - the files are empty. But the writer.writerow() line is called after each file analysis is complete, so I was expecting to get data in the csv file for all files that were analysed before the crash.

What am I doing wrong?

Can you show a minimal, complete, verifiable example rather than pseudo-code? Otherwise, it's going to be near impossible to replicate your problem — C.Nivs
– C.Nivs, Commented Jul 13, 2020 at 21:45

HackLab · Accepted Answer · 2020-07-13 21:48:36Z

1

writerow does not instantly write to the file, most of the writing is done when closing the file. You can enforce the writing to the file with

file.flush()

do this whenever you want to save your writings to the file.

For further information https://www.tutorialspoint.com/python/file_flush.htm

answered Jul 13, 2020 at 21:48

HackLab

5393 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Michael Stahl Over a year ago

Thanks! This did the job. Note for newbies like me: The csv file was opened like this: with open(csvfilename, mode='w', newline='') as csv_file: Therefore I use: csv_file.flush()

Collectives™ on Stack Overflow

csv writer leaves empty output file when the python script crashes

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related