Read each line from file as numpy arrays in python

Question

I have a file 'test.txt' with 4 lines as below:

[35,51,3701],[10,6,7700.75],[42,31,4700.5],[31,41,1700.25],[6,25,3901.25],[10,12,3301.5]
[4,13,3709.25],[13,24,3761.5]
[11,18,3101.25],[26,7,3671],[10,69,7800.75]
[5,5,3701.25]

I'll take the first line manually as an example. Here is what I'd like to do:

arr = np.array([[35,51,3701],[10,6,7700.75],[42,31,4700.5],[31,41,1700.25],[6,25,3901.25],[10,12,3301.5]])

p, b, a = [], [], []
for i in range(0, len(arr)):
        b = b + [arr[i][0]]
        a = a + [arr[i][1]]
        p = p + [arr[i][2]]

print (p)
print (b)
print (a)

The result is (this is exactly what I want):

[3701.0, 7700.75, 4700.5, 1700.25, 3901.25, 3301.5]
[35.0, 10.0, 42.0, 31.0, 6.0, 10.0]
[51.0, 6.0, 31.0, 41.0, 25.0, 12.0]

However, when I try to read the file line by line, it doesn't work. Here is the code I use:

f = open('test.txt')

for line in f:
  print(line)
  arr = np.array([line.rstrip()])
  p, b, a = [], [], []
  for i in range(0, len(arr)):
          b = b + [arr[i][0]]
          a = a + [arr[i][1]]
          p = p + [arr[i][2]]

  print (p)
  print (b)
  print (a)
  print ("\n\n")

and the result (not expected):

[35,51,3701],[10,6,7700.75],[42,31,4700.5],[31,41,1700.25],[6,25,3901.25],[10,12,3301.5]

['5']
['[']
['3']



[4,13,3709.25],[13,24,3761.5]

[',']
['[']
['4']



[11,18,3101.25],[26,7,3671],[10,69,7800.75]

['1']
['[']
['1']



[5,5,3701.25]
[',']
['[']
['5']

Can you please tell me how to have the result as in the manual example ? Thanks

Andrej Kesely · Accepted Answer · 2022-04-20 21:04:14Z

The problem is in line arr = np.array([line.rstrip()]). The line.rstrip() returns string, not numerical values. You can use ast.literal_eval to parse it, for example:

from ast import literal_eval

f = open("a.txt", "r")

for line in f:
    print(line)
    arr = np.array(literal_eval(line.rstrip() + ","))
    p, b, a = [], [], []
    for i in range(len(arr)):
        b = b + [arr[i][0]]
        a = a + [arr[i][1]]
        p = p + [arr[i][2]]

    print(p)
    print(b)
    print(a)
    print("\n\n")

Prints:

[35,51,3701],[10,6,7700.75],[42,31,4700.5],[31,41,1700.25],[6,25,3901.25],[10,12,3301.5]

[3701.0, 7700.75, 4700.5, 1700.25, 3901.25, 3301.5]
[35.0, 10.0, 42.0, 31.0, 6.0, 10.0]
[51.0, 6.0, 31.0, 41.0, 25.0, 12.0]



[4,13,3709.25],[13,24,3761.5]

[3709.25, 3761.5]
[4.0, 13.0]
[13.0, 24.0]



[11,18,3101.25],[26,7,3671],[10,69,7800.75]

[3101.25, 3671.0, 7800.75]
[11.0, 26.0, 10.0]
[18.0, 7.0, 69.0]



[5,5,3701.25]
[3701.25]
[5.0]
[5.0]

mozway · Accepted Answer · 2022-04-20 21:06:36Z

2

IIUC, you could try to use ast.literal_eval:

from ast import literal_eval
for line in f: # use your file here
    a = np.array(literal_eval(f'[{line}]'))
    p,b,a = a.T.tolist()
    print(p)
    print(b)
    print(a)
    print('---')

Alternatively, if you don't want to use numpy, use:

p,b,a = zip(*literal_eval(f'[{line}]'))

Output:

[35.0, 10.0, 42.0, 31.0, 6.0, 10.0]
[51.0, 6.0, 31.0, 41.0, 25.0, 12.0]
[3701.0, 7700.75, 4700.5, 1700.25, 3901.25, 3301.5]
---
[4.0, 13.0]
[13.0, 24.0]
[3709.25, 3761.5]
---
[11.0, 26.0, 10.0]
[18.0, 7.0, 69.0]
[3101.25, 3671.0, 7800.75]
---
[5.0]
[5.0]
[3701.25]
---

answered Apr 20, 2022 at 21:06

mozway

267k13 gold badges55 silver badges106 bronze badges

Collectives™ on Stack Overflow

Read each line from file as numpy arrays in python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related