Python: Iterating through a csv in a while loop

Question

I have a csv file that I'm trying to iterate through and have it do something only when a timestamp in it falls within a specified range. I'm using a while loop to do this where I'm adding 30 seconds each time through, but for some reason it only iterates through the csv the first time it does the loop and not each addition time (although I know it is successfully completing the while loop itself). I'm not sure what the problem is though.

try:
    csvfilename = sys.argv[1]
except:
    csvfilename = "myCSVfile.csv"


csvfile = csv.DictReader(open(csvfilename,'rb'),delimiter=',')

startTime = 1402617311
endTime = 1419544361
while startTime < endTime:
    for line in csvfile:
        if (startTime + 30) > timeToEpoch(line['EnterTime']) >= startTime:
            output = calcFunction(line['EnterN'],line['ExitN'],line['user'])
    startTime = startTime + 30

Per request here are a few lines of the csv file printed using the csv module (formatted a bit for readability, also the differences in the names and my example are because I changed them slightly):

{
'Time Exit': '17:02:04', 
'Duration': '0:02:34', 
'Date': '12/7/14', 
'ExitN': '12/7/14 17:02', 
'PlayerSpace': '1', 
'EnterN': '12/7/14 16:59', 
'Time Enter': '16:59:30'
}
{
'Time Exit': '17:08:18', 
'Duration': '0:08:46', 
'Date': '12/7/14', 
'ExitN': '12/7/14 17:08', 
'PlayerSpace': '2', 
'EnterN': '12/7/14 16:59', 
'Time Enter': '16:59:32'
}
{
'Time Exit': '17:06:49', 
'Duration': '0:04:20', 
'Date': '12/7/14', 
'ExitN': '12/7/14 17:06', 
'PlayerSpace': '3', 
'EnterN': '12/7/14 17:02', 
'Time Enter': '17:02:29'
}

Show us a relevant sample of the CSV file. Use the csv module. — user1907906
– user1907906, Commented May 8, 2015 at 16:29
Hey Ziyao Wei - that definitely seems to get me closer but when I try csvfile.seek(0) I get AttributeError: DictReader instance has no attribute 'seek' (I use csv.DictReader to read my csv file) — Mike
– Mike, Commented May 8, 2015 at 16:42
@Mike: The seek function need to be called on the file object rather than the reader - you can post some mode code if you still have problems (show how you define csvfile would be cool). — zw324
– zw324, Commented May 8, 2015 at 16:45

Andrea Corbellini · Accepted Answer · 2015-05-08 16:51:15Z

2

When you iterate the csvfile the first time, you exhaust it, so that it'll never yield more values.

One solution to your problem may be copying all the lines to a list, which can be iterated multiple times:

startTime = 1402617311
endTime = 1419544361
csv_lines = list(csvfile)   # <-
while startTime < endTime:
    for line in csv_lines:  # <-
        if (startTime + 30) > timeToEpoch(line['EnterTime']) >= endTime:
            output = calcFunction(line['EnterN'],line['ExitN'],line['user'])
    startTime = startTime + 30

answered May 8, 2015 at 16:51

Andrea Corbellini

17.9k3 gold badges58 silver badges71 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Mike Over a year ago

This got me really close - I made some tweaks which I'll show above because this made it iterate through item in the csv which made for a really long wait time

Andrea Corbellini Over a year ago

@Mike: you can also create (and accept) your own answer.

Mike Over a year ago

Good call - I added mine as well for clarity - but wanted to make sure that I gave you credit for getting me there! Thanks

Mike · Accepted Answer · 2015-05-08 17:16:20Z

So this was based on Andrea's suggestion I moved it to a list(), which worked great, but what I really needed to do was to check the condition every 30 seconds, which made it iterate through everything every 30 seconds (making it really slow) so I adapted it and slightly making it much quicker since all my times are sequential this works:

line = list(csv_data) #Convert into a list

lineCount = line[n] # Done so I can iterate line by line

while (startTime + 30) > timeToEpoch(lineCount['EnterN']) >= startTime:
    print "time on"
    pc = playerCount(lineCount['EnterN'],lineCount['ExitN'],lineCount['Player'])
    n += 1
    lineCount = line[n]

Note: sorry there were some small errors in my original code with endTime/startTime which didn't affect people's answers, but I changed in my final answer (some of this was because I renamed variables in an attempt to reduce confusion)

Collectives™ on Stack Overflow

Python: Iterating through a csv in a while loop

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related