Python readline ignoring header in csv file

Question

I've written some code that is supposed to be aware of if a CSV file exists and append to it if it's the right format, or erase it and create a new file.

The part where it checks if the format is correct (by checking the CSV headers to see if they are equal) is not working, because for some weird reason the readline() function is ignoring the headers (which should be the first line of the CSV file.

Do note that I don't want to use extra dependencies such as Pandas.

    import os, csv

    path = 'intent_outputs.csv'
    response = {'query':'eggs', 'header':'eggs', 'fulfillmentText':'are tasty', 'buttontext':'VISIT PAGE', 'buttonurl':'eggs.com', 'contextbuttontext':'eggs', 'contextbuttonurl':'eggs.com'}
    output = open(path, 'a')
    csv_columns = ['Query', 'Header', 'Text', 'Button Text', 'Button URL', 'Context Button Text', 'Context Button URL']
    writer = csv.DictWriter(output, fieldnames=csv_columns)
    if not os.path.exists(path):
        print("DOES NOT EXIST")
        output = open(path, 'w')
    else:
        print("EXISTS")
        output = open(path, 'r')
    if(output.readline() != ",".join(csv_columns)):
        print("NOT EQUAL")
        try:
            output = open(path, 'w')
            writer.writeheader()
        except IOError:
            print("IOError")
    else:
        print("EQUAL")
        output = open(path, 'a')
    try:
        row = {'Query':response['query'], 'Header':response['header'], 'Text':response['fulfillmentText'], 'Button Text':response['buttontext'], 'Button URL':response['buttonurl'], 'Context Button Text':response['contextbuttontext'], 'Context Button URL':response['contextbuttonurl']}
        writer.writerow(row)
    except IOError:
        print("I/O error")

Here's ~~Johnny~~ intent_outputs.csv:

Query,Header,Text,Button Text,Button URL,Context Button Text,Context Button URL
eggs,eggs,are tasty,VISIT PAGE,eggs.com,eggs,eggs.com

I want to read the first line (starting with "Query....") but it's being actively ignored.

You seem to be opening the file with output = open(path, 'a') and later output = open(path, 'w') without closing the file in between; this seems like a breathing ground for unintended behavior. Are you sure that this is a good way to go? — Ralf
– Ralf, Commented Jul 23, 2019 at 11:05
I used those tags because in the former I have to provide an instance of open() to initialize csv.DictWriter() (the 'a' flag is to prevent overwriting and permit appending to the file if all of the if-else checks pass), and in the latter the 'w' tag is so that I can erase the file and overwrite new headers. I am not sure it is a great idea, though...... — Mayukh Nair
– Mayukh Nair, Commented Jul 23, 2019 at 11:07
The problem Ralf pointed to is not the flags, but the fact you open the file twice without closing it in-between. You should probably tweak the logic to use the with open(my_file, flag) as foo context manager. — Leporello
– Leporello, Commented Jul 23, 2019 at 12:09

suitsense · Accepted Answer · 2019-07-23 11:11:42Z

1

readline() output includes newline \n character at the end. So output.readline() returns

Query,Header,Text,Button Text,Button URL,Context Button Text,Context Button URL\n

Maybe that is why this condition if(output.readline() != ",".join(csv_columns)): returns True.

answered Jul 23, 2019 at 11:11

suitsense

483 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Mayukh Nair Over a year ago

Uh, no? Just tested and it hasn't been returning True for me.

suitsense Over a year ago

Oh, sorry... Can you post the output logs or stack trace?

Mayukh Nair Over a year ago

There's no logging or stack trace as such. The program runs without countering an exception. It's constantly ignoring the first line of the CSV though, which is weird.

Mayukh Nair · Accepted Answer · 2019-07-23 12:15:41Z

Ended up solving this myself. So I rewrote the function to not rely on the opened files, and apart from some extra logic bugs in the code above, the script below works perfectly as intended.

@Suitsense's answer made me aware of the \n that readline() adds at the end, so thank you!

    path = 'intent_outputs.csv'
    response = {'query':'eggs', 'header':'eggs', 'fulfillmentText':'are tasty', 'buttontext':'VISIT PAGE', 'buttonurl':'eggs.com', 'contextbuttontext':'eggs', 'contextbuttonurl':'eggs.com'}
    csv_columns = ['Query', 'Header', 'Text', 'Button Text', 'Button URL', 'Context Button Text', 'Context Button URL']
    output = ""
    writer = ""
    try:
        output = open(path,'x')
    except FileExistsError:
        output = open(path,'r')
    if(open(path,'r').readline() != ",".join(csv_columns)+"\n"):
        try:
            output = open(path, 'w')
            writer = csv.DictWriter(output, fieldnames=csv_columns)
            writer.writeheader()
        except IOError:
            print("IOError: Couldn't write header")
    else:
        output = open(path, 'a')
        writer = csv.DictWriter(output, fieldnames=csv_columns)
    row = {'Query':response['query'], 'Header':response['header'], 'Text':response['fulfillmentText'], 'Button Text':response['buttontext'], 'Button URL':response['buttonurl'], 'Context Button Text':response['contextbuttontext'], 'Context Button URL':response['contextbuttonurl']}
    writer.writerow(row)

Collectives™ on Stack Overflow

Python readline ignoring header in csv file

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related