6

I am trying to add some random data into text file and I am successful in that but I am facing problem with the header line. I want to add header line once and then every time I run my script , it should add just data into file and ignore the header line if exists. I tried something like this but I fail. I try to look at this example code in SO python csv, writing headers only once, but couldn't implement properly. If somebody help me to correct my code. I will be thankful. 

import random
import csv
import os.path
from time import gmtime, strftime

filename = '/home/robdata/collection1.dat'

file_exists = os.path.isfile(filename)

v = random.randint(0, 100)

with open(filename, "a") as csvfile:
    headers = ['DATE', 'value']
    writer = csv.DictWriter(csvfile, delimiter=',', lineterminator='\n',fieldnames=headers)
    if not file_exists:
        writer.writeheader()  # file doesn't exist yet, write a header

    writer.writerow({'DATE': strftime("%Y-%m-%d %H:%M:%S", gmtime()), 'value': v})

it insert data but without adding any header line. I want to include headers on the first run of script, and next time when I run script several times , it should only add data and not header line. thank a lot for any idea or help.

Improve this question
15
  • Could you edit into the question a description of what your code is doing incorrectly? Commented Mar 23, 2017 at 13:38
  • @glibdud I add below in my question. I am able to insert data into file but without any headers. Commented Mar 23, 2017 at 13:42
  • The first time you run it (before the file exists), you should get an AttributeError. Take a look at the differences between your csv.writer object and what was used in the question you linked. Commented Mar 23, 2017 at 13:47
  • @glibdud I tried that example also which I mention. I change my question according to that example. as i am new to programming , might it is very simple mistake I am making here. Commented Mar 23, 2017 at 13:56
  • 1
    That won't do it, will it? Because the file still exists. Notice that your code asks whether the file exists, not whether it's empty. Commented Mar 23, 2017 at 14:16

5 Answers 5

Reset to default
9

A slightly simpler alternative to Mr Evans approach would be to use the following test in place of the test for existence:

fileEmpty = os.stat('collection1.dat').st_size == 0

This obviates the need to do a seek, etc.

EDIT: Complete code:

import random
import csv
import os.path
from time import gmtime, strftime

filename = '/home/robdata/collection1.dat'

fileEmpty = os.stat(filename).st_size == 0

v = random.randint(0, 100)

with open(filename, "a") as csvfile:
    headers = ['DATE', 'value']
    writer = csv.DictWriter(csvfile, delimiter=',', lineterminator='\n',fieldnames=headers)
    if fileEmpty:
        writer.writeheader()  # file doesn't exist yet, write a header

    writer.writerow({'DATE': strftime("%Y-%m-%d %H:%M:%S", gmtime()), 'value': v})
Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

thank you so much for your help. the above mention code in the question works perfectly fine according to my requirements. thanx for the help.
would you like to paste the answer so that I accept, because you first edit my question code and it works fine. I ask you because you edit it first.
You're most welcome. If this answer is the one that helped you then the etiquette on SO is for you to mark it 'accepted'. When you have more points you will be allowed to up-vote.
Sure, I can do that.
I mean the code in my question was edited by you and it works perfectly fine. If you just paste that code, I will accept your answer
|
5

Rather than testing if the file exists, you could instead check to see if has zero length by seeking to the end and asking for the file location (which is of course still needed to append).

This approach will then also write the header for the case when the file is empty but still exists. It also avoids needing to import os to make a separate os.stat() to determine the file size:

import random
import csv
from time import gmtime, strftime
    
headers = ['DATE', 'value']
v = random.randint(0, 100)
    
with open('collection1.dat', 'a', newline='') as f_output:
    csv_output = csv.DictWriter(f_output, fieldnames=headers)
    f_output.seek(0, 2)
    
    if f_output.tell() == 0:
        csv_output.writeheader()

    csv_output.writerow({'DATE': strftime("%Y-%m-%d %H:%M:%S", gmtime()), 'value': v})

writeheader() is then used to write the header only when the file is empty.

Note: If Python 2.x is still in use, you should always open the file in binary mode e.g. ab as follows:

with open('collection1.dat', 'ab') as f_output:
Improve this answer

2 Comments

I like this solution rather than doing other checks for if the file exists, then if it is 0, etc.
Great solution.
1

From the documentation of Python CSV module, writeheader can be used only with DictWriter. [https://docs.python.org/2/library/csv.html#csv.DictWriter.writeheader][Documentation of writeheader]

Hence, the code should not be using writeheader. You can try this. 

if not file_exists:
    writer.writerow(headers)
else:
    writer.writerow([strftime("%Y-%m-%d %H:%M:%S", gmtime()), v, w, z])
Improve this answer

1 Comment

I edit my question. but I tried this idea before and i got this error. ValueError: dict contains fields not in fieldnames: '2017-03-23 13:59:39', 94
1

Looks like you edited your question while we were posting answer. Please add any changes as edit or mention what you have changed. Anyway, while dict fields are different, concept is same.

On top of dictwriter, you need to pass row data as values to the dict keys for each column.

I added a loop for 5 writes delayed by 5 sec as sample.
See below.

Working Code: 

import random
import csv
import os.path
from time import gmtime, strftime, sleep

filename = 'collection1.csv'

file_exists = os.path.isfile(filename)

v = random.randint(0, 100)
w = random.randint(0, 100)
z = random.randint(0, 100)
with open(filename, "a") as csvfile:
    for i in range(5):        
        headers = ['timestamp','val1', 'val2', 'val3']
        writer = csv.DictWriter(csvfile, delimiter=',', lineterminator='\n',fieldnames=headers)
        if not file_exists:
            writer.writeheader()
        else:
            mytime = strftime("%Y-%m-%d %H:%M:%S", gmtime())
            writer.writerow({'timestamp':mytime, 'val1': v,'val2': w, 'val3':z})

        sleep(5)

collection1.csv 

timestamp,val1,val2,val3
2017-03-23 14:07:20,83,60,11
2017-03-23 14:07:25,83,60,11
2017-03-23 14:07:30,83,60,11
2017-03-23 14:07:47,55,44,88
2017-03-23 14:07:52,55,44,88
2017-03-23 14:07:57,55,44,88
2017-03-23 14:08:02,55,44,88
2017-03-23 14:08:07,55,44,88
Improve this answer

6 Comments

I tried your code but I get the same results. wait I will examine the code again. I will come back soon to update . thank you very much for your help.
Just copy/paste my code as it is. It's working on my end. And this is your old code (modified).Let me know.
@accept my apology that I didn't inform and added that I edit my code. next time I will be careful.
your code works fine. it also create in the run the headers and then in the subsequent run, it adds the data. thank you so much for your help.
@rob , Glad to help. If it resolves your question, do you mind accepting the answer so that loop is closed.
|
1

In case that you are appending pandas DataFrame to a csv file, and want to keep the header only at the first time you write out, here is what worked for me, much simpler than solutions before:

start = 100
with open('path_to_your_csv', "r",encoding = 'UTF-8') as myfile:
    reader = csv.DictReader(myfile)
    for i,line in itertools.islice(enumerate(reader) , start, 105): # iterate through row 100:105
    df # this is the data frame you want to export
    if i == start:
        df.iloc[0:0].to_csv(recent,sep = ",") # keep only header if this is the first you export
    df.to_csv(total,sep = ",",header =False)                                  
    del df
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.