Issue with handling the reader object in python csv module

Question

The goal I am trying to accomplish is reading in only the particular data I want from a large csv file. To do this, I have a main menu that I use as a handler for data acquisition and then a separate menu for exiting or continuing. My issue arises when I attempt to read in more data after looping through the file once. The issue being that I have reached the end of the file and for some reason the for loop is not handling the StopIterator error correctly. Any suggestions? Thanks in advance!

fname = open(current_file, 'r')
reader = csv.reader(fname)
for row in reader:
    list_header = row
    break
def main_menu():
    i=0
    menu = {}
    print reader
    for hdr in list_header:
        menu[str(i)]=hdr
        i+=1;
    options=menu.keys() #creates a list out of the keys
    options.sort()
    for entry in options: 
        print entry, menu[entry]
    selection=raw_input("Please Select:") 
    data=[]
    for row in reader:
        a=0
        for block in row:
            if a==list_header.index(menu[selection]):
                data.append(block)
            a+=1
    print 'Saving '+menu[selection]+' values into an array.'+'\n'
    return data

def continue_menu():
    menu_1={}
    menu_1['0']='Continue'
    menu_1['1']='Exit'
    options=menu_1.keys()
    options.sort()
    for entry in options:
        print entry, menu_1[entry]
    selection=raw_input('Please Select:')
    if float(selection)==0:
        print 'As you wish'+'\n'
        proceed=True
    else:
        proceed=False
    return proceed
proceed=True    

while proceed:
    data1=main_menu()
    proceed=continue_menu()

What actually went wrong and where in the source did it go wrong? — tdelaney
– tdelaney, Commented Mar 22, 2016 at 3:02
Are you calling main_menu twice? If so, for row in reader has no data to read the second time around and its code block is skipped and an empty list is returned. If you don't give us more details about the problem there is no way for us to help. — tdelaney
– tdelaney, Commented Mar 22, 2016 at 3:23
Ahhhh.. The while loop in the bottom? And yes, that is what was happening. I had flags and if statements in and narrowed it down to the second iteration of the while loop. So you're saying the fact that main_menu gets called before the continue_menu does the reader object will be returned empty on the next iteration of the while loop? — Cauchy
– Cauchy, Commented Mar 22, 2016 at 4:44
Not quite. continue_menu doesn't touch the reader or its underlying file object so from an iteration point of view, it makes no difference. Once you've gone through the for loop, the fanme file pointer is at the end of the file. You need to reset that pointer. I'll put an example as an aswer. — tdelaney
– tdelaney, Commented Mar 22, 2016 at 16:07

tdelaney · Accepted Answer · 2016-03-22 16:14:50Z

1

csv.reader reads lines from a file object and splits them into a rows. When you hit the end of file, StopIteration is raised, the for loop catches that exception and the loop stops. Now the file pointer is at the end of the file. If you try to iterate through it a second time, its already at the end and raises StopIteration immediately. Notice in the example that nothing is printed the second time through the loop

>>> import csv
>>> fname=open('a.csv')
>>> reader = csv.reader(f)
>>> for row in reader:
...     print(row)
... 
['1', '2', '3']
['4', '5', '6']
['7', '8', '9']

>>> for row in reader:
...     print(row)
... 
>>>

One solution is to just rewind the file pointer to the start of the file. Now, the loop works again

>>> fname.seek(0,0)
0
>>> for row in reader:
...     print(row)
... 
['1', '2', '3']
['4', '5', '6']
['7', '8', '9']

Another more commonly used solution is to open the file just before the iteration. By using a while the file is closed immediately after use and the next time the loop is run, the file is opened and iterated again.

>>> with open('a.csv') as fname:
...     for row in csv.reader(fname):
...         print(row)
... 
['1', '2', '3']
['4', '5', '6']
['7', '8', '9']

answered Mar 22, 2016 at 16:14

tdelaney

77.9k6 gold badges91 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Cauchy Over a year ago

Awesome, just ended fname.seek(0,0) before the return statement in main_menu and everything works perfectly. Thanks @tdelaney

Collectives™ on Stack Overflow

Issue with handling the reader object in python csv module

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related