Python read file in while loop

Question

I am currently in some truble regarding python and reading files. I have to open a file in a while loop and do some stuff with the values of the file. The results are written into a new file. This new file is then read in the next run of the while loop. But in this second run I get no values out of this file... Here is a code snippet, that hopefully clarifies what I mean.

while convergence == 0:
    run += 1
    prevrun = run-1

    if os.path.isfile("./Output/temp/EmissionMat%d.txt" %prevrun) == True:
        matfile = open("./Output/temp/EmissionMat%d.txt" %prevrun, "r") 
        EmissionMat = Aux_Functions.EmissionMat(matfile)
        matfile.close()
    else:
        matfile = open("./Input/EmissionMat.txt", "r") 
        EmissionMat = Aux_Functions.EmissionMat(matfile)
        matfile.close()

    # now some valid operations, which produce a matrix

    emissionmat_file = open("./output/temp/EmissionMat%d.txt" %run, "w")
    emissionmat_file.flush()
    emissionmat_file.write(str(matrix))

    emissionmat_file.close()

Solved it!

matfile.seek(0)

This resets the pointer to the begining of the file and allows me to read the file in the next run correctly.

why do you emissionmat_file.flush() before writing, and what is the content of matrix — njzk2
– njzk2, Commented Jun 6, 2013 at 8:38
@njzk2 I do emissionmat_file.flush() to get the content to my file otherwise i get no values in my file. The content of matrix is a numpy array with 40*40 values. But do not mind the write stuff. The resulting file looked as it is desired. I just simplified it here. — swot
– swot, Commented Jun 6, 2013 at 8:46
do you have a typo in your code? "./Output" and "./output" is not the same... pro tip: use a constant/variable for this... — Daren Thomas
– Daren Thomas, Commented Jun 6, 2013 at 8:49

MatthieuBizien · Accepted Answer · 2013-06-06 09:03:38Z

1

Why to write to a file and then read it ? Moreover you use flush, so you are doing potentially long io. I would do

with open(originalpath) as f:
    mat = f.read()
while condition :
    run += 1
    write_mat_run(mat, run)
    mat = func(mat)

write_mat_run may be done in another thread. You should check io exceptions.

BTW this will probably solve your bug, or at least make it clear.

answered Jun 6, 2013 at 9:03

MatthieuBizien

1,7251 gold badge10 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

swot Over a year ago

This is part of an expectation maximization algorithm so i need the values from the previous run for the current run.

swot Over a year ago

I solved it (see question post) I will post it as a "real" answer if the system allows me to. But thanks for your help!

MatthieuBizien Over a year ago

With this pattern you use the previous run for the current one. It may be really more efficient because IO is a bottleneck and can kill you performances and less bug prone since you don't have to rely on the filesystem. Even if you solved your bug I suggest you to use this pattern.

swot Over a year ago

Ok, I will try it. I honestly have not thought of the prefomance issue, so thanks.

swot Over a year ago

Thanks a lot for this advice. I changed it the way you suggested it an it really improves the speed of the computation and also solves another error I got. :)

tom · Accepted Answer · 2013-06-06 09:03:08Z

0

I can see nothing wrong with your code. The following concrete example worked on my Linux machine:

import os

run = 0
while run < 10:
    run += 1
    prevrun = run-1

    if os.path.isfile("output%d.txt" %prevrun):
        matfile = open("output%d.txt" %prevrun, "r")
        data = matfile.readlines()
        matfile.close()
    else:
        matfile = open("input.txt", "r")
        data = matfile.readlines()
        matfile.close()

    data = [ s[:-1] + "!\n" for s in data ]

    emissionmat_file = open("output%d.txt" %run, "w")
    emissionmat_file.writelines(data)
    emissionmat_file.close()

It adds an exclamation mark to each line in the file input.txt.

answered Jun 6, 2013 at 9:03

tom

23.2k7 gold badges47 silver badges38 bronze badges

1 Comment

swot Over a year ago

As you can see in the question post I solved the problem. I will put it in a "real" answer as soon as the system allows me to do this. Thanks for your help.

swot · Accepted Answer · 2013-06-07 08:30:02Z

0

I solved it

before closing the file I do

matfile.seek(0)

This solved my problem. This methods sets the pointer of the reader to the beginning.

answered Jun 7, 2013 at 8:30

swot

2781 gold badge3 silver badges10 bronze badges

Collectives™ on Stack Overflow

Python read file in while loop

3 Answers 3

5 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

5 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related