reading data file into multidimensional array in python

Question

I am new to python. I want to read data from file and store it in a multidimensional array. For example, I have this,

6 5.9
6.3 5.9
6.6 6.3

7.8 7.5
7.8 7.3
7.5 7.6

8.3 8
8.5 8
8.2 8.3

9.2 8.5
9 8.5
9.2 8.9

I want this stored in an array like this:

[ [['6', '5.9'], ['6.3', '5.9'], ['6.6', '6.3']], 
  [['7.8', '7.5'], ['7.8', '7.3'], ['7.5', '7.6']], 
      [['8.3', '8'], ['8.5', '8'], ['8.2', '8.3']], 
      [['9.2', '8.5'], ['9', '8.5'], ['9.2', '8.9']] ]

I have tried this so far:

with open("number.txt") as textFile:
    lines = [line.split() for line in textFile]
print(lines)

And it gave me like this:

[['6', '5.9'], ['6.3', '5.9'], ['6.6', '6.3'], [], ['7.8', '7.5'], ['7.8', '7.3'], ['7.5', '7.6'], [], ['8.3', '8'], ['8.5', '8'], ['8.2', '8.3'], [], ['9.2', '8.5'], ['9', '8.5'], ['9.2', '8.9']]

You may want to specify whether it will always be two elements per line, three lines per group, four groups per file, etc, or whether you want it more dynamic than that. — paxdiablo
– paxdiablo, Commented Nov 6, 2015 at 1:04
d_unknown, so presumably that means variable quantities of values rather than fixed, yes? In other words, able to handle arbitrary quantities in all three dimensions. — paxdiablo
– paxdiablo, Commented Nov 6, 2015 at 1:25

paxdiablo · Accepted Answer · 2015-11-06 01:20:56Z

2

The following code will give you the results you want:

import re

dim3 = []
dim2 = []
f = open ('inputfile.txt', 'r')
for s in f.readlines():
    s = s.strip()
    if s == '':
        dim3.append(dim2)
        dim2 = []
    else:
        dim1 = re.split('\s+', s)
        dim2.append(dim1)
if len(dim2) > 0:
    dim3.append(dim2)
f.close()

print(dim3)

It basically maintains dim2/3 dimension variables to hold the values, and the dim1 dimension variable to handle each line.

For each non-blank line, we calculate the array dim1 and add it to the current dim2. If we find a blank line, we add the current dim2 to dim3 and reset dim2.

At the end, we handle any leftover dim2 and the resultant dim3 is the multi-dimensional array that you desired (formatted for readability):

[[['6'  , '5.9'], ['6.3', '5.9'], ['6.6', '6.3']],
 [['7.8', '7.5'], ['7.8', '7.3'], ['7.5', '7.6']],
 [['8.3', '8'  ], ['8.5', '8'  ], ['8.2', '8.3']],
 [['9.2', '8.5'], ['9'  , '8.5'], ['9.2', '8.9']]]

The code is such that it will handle arbitrary dimension sizes, in the sense that it allows any quantity of numbers-per-line, lines-per-group and groups-per-file.

answered Nov 6, 2015 at 1:20

paxdiablo

888k243 gold badges1.6k silver badges2k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Vegas Over a year ago

I have no idea why this got downvoted, it's very straight to the point.

alanmanderson · Accepted Answer · 2015-11-06 07:38:11Z

1

First you read the data from the file 1 line at a time. Then for every empty line you start a new array. Otherwise you split the current array on spaces and append it to the return value.

You can use this code:

ret = []                                 #return value
with open('data.txt', 'r') as f:         #open file
    curArray = []                        #start current array
    for line in f:                       #loop through the lines
        line = line.rstrip()             #get rid of \n
        if len(line) == 0:               #if its an empty line
            ret.append(curArray)         #  append curArray to ret
            curArray = []                #  reset curArray
        else:
            numbers = line.split(' ')    #split array on spaces
            curArray.append(numbers)     #append new array to the curArray
print ret

This code assumes each line should be an array and each time there is an empty line (with only a newline character) a new array is started.

To get the sum of a column across all arrays write a function that takes the array and the index of the column you want to sum:

def sumColumn(arr3d, index):
    sum = 0
    for arr in arr3d:
        for arr2 in arr:
            sum+=float(arr2[index])
    return sum

#now print the sum of the first column using the initial data file.
print sumColumn(ret, 0)    # the columns are 0 indexed, so 0 is the first column

edited Nov 6, 2015 at 7:38

answered Nov 6, 2015 at 1:16

alanmanderson

8,31012 gold badges55 silver badges89 bronze badges

3 Comments

d_unknown Over a year ago

how do i get the sum of each column? example is the vertical addition from data file..

alanmanderson Over a year ago

addition is a little ambiguous in this context... do you want the whole sum for the entire first column or do you want subtotals at each linebreak?

alanmanderson Over a year ago

See my edit. For additional help you should create another question so this one doesn't get too much cruft.

Lencalot · Accepted Answer · 2015-11-06 01:20:20Z

0

This is assuming your arrays are going to be the sizes you specified. This is merely to demonstrate the logic you are going to need to use to solve this problem.

matrix = []
tempLine = []

i = 0
for line in file
    if i < 3:
        #assuming you can parse those values on the line
        tempLine.append([firstValue, secondValue])
        i += 1
    if i >= 3:
        i = 0
        matrix.append(tempLine)
        tempLine = []

print matrix

answered Nov 6, 2015 at 1:20

Lencalot

3954 silver badges20 bronze badges

Collectives™ on Stack Overflow

reading data file into multidimensional array in python

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related