using while loop to read files

Question

I have 200 files, from which I wanna choose the second column. I wanna store the second column of each file in a list called "colv". I wanna have colv[0]=[second column of the first file. colv[1] be the second column of the second file and so on. I write this code, but it does not work, in this code colv[0] is the first number of the second column of the first file. Can anyone help me how to fix this issue:

colv = []
i = 1
colvar = "step7_1.colvar"
while os.path.isfile(colvar):
    with open(colvar, "r") as f_in:
        line = next(f_in)
        for line in f_in:
            a = line.split()[1]
            colv.append(a)
    i+=1
    colvar = "step7_%d.colvar" %i

Are you sure there's a file step7_2.colvar? The loop stops as soon as it gets to a missing file. — Barmar
– Barmar, Commented Aug 15, 2022 at 21:25
Do you want to skip the missing files and look for the rest? Use a for loop instead of while loop. — Barmar
– Barmar, Commented Aug 15, 2022 at 21:26
If there are multiple lines in the file, how do you want the second column of all the lines to be stored in a single list element? Should it be a nested list? — Barmar
– Barmar, Commented Aug 15, 2022 at 21:31
yes it should be a nested list like this: [[0,1,2,3],[0,1,2,3],[0,1,2,3]]. But my code is not generating such a thing for me. Do you know how can i fix it — amirre
– amirre, Commented Aug 15, 2022 at 21:33

7shoe · Accepted Answer · 2022-08-15 21:33:50Z

1

How about using Pandas' read_csv() since you mention that the data has a table-like structure. In particular, you could use

import pandas as pd

colv = []
i = 1
colvar = "step7_1.colvar"
while os.path.isfile(colvar):
    df = pd.read_csv(colvar, sep=',')
    colv.append(list(df[df.columns[1]]))
    
    i+=1
    colvar = "step7_%d.colvar" %i

It returned

>colv
[[5, 6, 7, 8], [8, 9, 10, 11], [12, 13, 14, 15]]

for my vanilla sample files step7_%d.colvar.

You might need to adjust the separator character with sep.

answered Aug 15, 2022 at 21:33

7shoe

1,5261 gold badge8 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Barmar · Accepted Answer · 2022-08-15 21:33:31Z

0

Use a list comprehension to get the 2nd element of all the lines into a list.

colv = []
i = 1
colvar = "step7_1.colvar"
while os.path.isfile(colvar):
    with open(colvar, "r") as f_in:
        readline(f_in) # skip 1st line
        data = [line.split()[1] for line in f_in]
        colv.append(data)
    i+=1
    colvar = "step7_%d.colvar" %i

answered Aug 15, 2022 at 21:33

Barmar

789k57 gold badges555 silver badges669 bronze badges

Collectives™ on Stack Overflow

using while loop to read files

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related