Creating n*n matrix from text file contains list of lists in python

Question

I have a text file contains a list (#lines = 1137) of vectors (all are equal size= 1137), am trying to convert this list to 1137*1137 matrix. I created an empty matrix using numpy. But the problem after when I read the file using the following code, the vectors are treated as a sequence of characters, not as a vector or array

myMtrx = np.empty((1137,1137))
f = open("text.txt", "r")
for vector in f:
    print len(vector) 
    arrayF.append(vector.rstrip())

I recognized that by printing our the length of each vector, which is computed based on number of digits not elements in that vector. The vector in the text file looks like

[99.25, 14.74, 26.12, 20.91, 37.14, 79.03, 17.68,  28.4, ...., 0]

so when I print print arrayF[0][0] I receive [, where I need the output to be the 1st element of the 1st vector, which is 99.25.

I tried several ways using numpy, and writing the text file to CSV but nothing works, can you please assist me to solve this issue. You can access the text file through the following link give you an idea about its structure. text.txt

When you say for vector in f, what gets passed to your code in the vector variable is not a vector, but rather one entire line of the file (as a string). You want to do something like for line in f and write code inside the loop to split up each line into the tokens of interest (using regular-expressions and/or str.split(), for example) . Then make sure you convert those tokens to numeric values, before using them to fill up your array. — dhrumeel
– dhrumeel, Commented Feb 4, 2017 at 22:04
Thanks for the swift response. The content of vector is as described in my post [99.25,........0], but as you said, it's parsed as string not as array. But is there an easier method than reg-expression??? — Mohannad
– Mohannad, Commented Feb 4, 2017 at 22:12
If lines adhere to sample, you may strip the sqare brackets from the line and split on comma space and int() every list element coming out of rhe split. Makes sense? — Dilettant
– Dilettant, Commented Feb 4, 2017 at 22:16

stellasia · Accepted Answer · 2017-02-04 22:17:34Z

1

You are reading string from your file, that you need to convert to list. A solution like this one may do the trick:

for line in f:
   vector = line.strip("[]").split(",")
   ...

strip : remove all characters in "[]" from begining and end of string
split: transform string to list, cutting at each "," position

answered Feb 4, 2017 at 22:17

stellasia

5,6624 gold badges27 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

bluprince13 · Accepted Answer · 2017-02-05 00:59:02Z

0

Starting with a string, you need to do the following steps to get a list of numbers from it:

Use strip to get rid of []
Use split(",") to split the string at every comma and create a list of strings
Use map(float, vector) to convert it to a list of floats
Use list() to get an actual list rather than a mapped object

You should then be able to put this into your numpy matrix. It's not necessary to use map(float, vector) if using numpy though, because numpy will infer that they are floats automatically.

Here is a sample code:

myMtrx = np.empty((1137,1137))
f = open("text.txt", "r")

for idx, vector in enumerate(f):
    # vector = '[99.25, 14.74, 26.12, 20.91, 37.14, 79.03, 17.68,  28.4, 0]'
    vector = list(map(float, vector.strip("[]\n").split(",")))
    myMtrx[idx ,:] = vector

edited Feb 5, 2017 at 0:59

answered Feb 4, 2017 at 22:36

bluprince13

5,08116 gold badges56 silver badges95 bronze badges

3 Comments

Mohannad Over a year ago

Implemented suggested code, but received following error ValueError: invalid literal for float(): 17.6]

bluprince13 Over a year ago

Does the first line in your file end with 17.6? It could be the newline character after it that could be causing the issue. Try adding \n inside strip()

bluprince13 Over a year ago

Great! Please accept the answer if you're happy with it.

Ajax1234 · Accepted Answer · 2017-02-05 17:20:46Z

0

I would do this:

  f = open('first.txt').readlines()
  f = [i.strip('\n') for i in f]

  new_list = []
  final_list = []
  for i in f:
      new_list.append(i.split(' '))

  for i in new_list:
      final_list.append(map(int, i))

  print final_list

we read the contents into f, split them at the spaces and append them to new_list, and then map over each string row in the matrix formed and append that to final_list, which will give you the matrix you would like.

answered Feb 5, 2017 at 17:20

Ajax1234

71.7k9 gold badges67 silver badges110 bronze badges

Collectives™ on Stack Overflow

Creating n*n matrix from text file contains list of lists in python

3 Answers 3

Comments

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related