Why does counting lines of csvfile in Python's csv.DictReader break my code?

Question

This feels like a stupid mistake, but I am unable to spot it. When I execute this code:

with open('test.csv') as csvfile:
    # row_count = sum(1 for row in csvfile)
    reader = csv.DictReader(csvfile, delimiter = ';')
    path = os.getcwd()      # ← gets current directory

    for row in reader:
        header = row['FirstName'] + ' ' + row['Surname'] + ' – #' + row['RecruitmentID'].zfill(5)
        print('Processing: ' + ' / ' + header)

what I get is:

Processing:  / AAA aaa – #00005
Processing:  / BBB bbb – #00008
Processing:  / CCC ccc – #00160
[Finished in 0.3s]

But when I uncomment # row_count = sum(1 for row in csvfile) I only get [Finished in 0.3s] in the console.

Even using another method: numberofrows = len(list(reader)) instead breaks it the same way. What am I doing wrong?

while doing row_count = sum(1 for row in csvfile) you go to the end of your file , this way the csv reader won't fine anything else, try doing csvfile.seek(0), that should take you back at the beginning of the file, before creating the csv reader — Lohmar ASHAR
– Lohmar ASHAR, Commented Jul 30, 2019 at 12:05
@LohmarASHAR oh, this was it! Thanks, I am new to working with streams and files… — MrVocabulary
– MrVocabulary, Commented Jul 30, 2019 at 12:12
Offtopic: when you ask a question and write some example code, make it as minimal as possible. Importing pypdftk or path = os.getcwd() makes other users read too much code (and think about it) that has nothing to do with your question. Thank you. — eumiro
– eumiro, Commented Jul 30, 2019 at 12:16
@eumiro perhaps you are right, but I wasn't completely unaware which part was causing this behavior. Editer for clarity. — MrVocabulary
– MrVocabulary, Commented Jul 30, 2019 at 12:23

Thomas · Accepted Answer · 2019-07-30 12:12:12Z

2

The following will read the entire CSV file:

row_count = sum(1 for row in csvfile)

When you read a file in Python (or any other language) you are moving the "position" in the file. When you finish reading the file, the position is remembered as the end of the file. There's just nothing left to read.

Either you can close and re-open the file, or you can "rewind"/"seek" the position back to the start of the file.

This is done with the following code:

csvfile.seek(0, os.SEEK_SET)

answered Jul 30, 2019 at 12:12

Thomas

7212 gold badges10 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

MrVocabulary Over a year ago

Thanks for explaining this to me. I don't see how csvfile.seek(0, os.SEEK_SET) is any different from csvfile.seek(0) in terms of my program's behavior.

Thomas Over a year ago

You're right, they're the same thing. From the official documentation about file I/O: "To change the file object’s position, use f.seek(offset, from_what)." from_what is optional but has values 0 ("from the start of the file"), 1 ("from the current position"), and 2 ("from the end of the file"). Just giving csvfile.seek(0) defaults to 0. os.SEEK_SET == 0 as well.

Simpl · Accepted Answer · 2019-07-30 12:07:04Z

1

row_count = sum(1 for row in csvfile) will read the whole file. When calling for row in reader: you are already at EOF.

answered Jul 30, 2019 at 12:07

Simpl

2,0561 gold badge12 silver badges25 bronze badges

Collectives™ on Stack Overflow

Why does counting lines of csvfile in Python's csv.DictReader break my code?

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related