Numpy multidimensional array only registering as 1D array

Question

I have a set of data from a spread sheet that I would like to pull into numpy. The values are magnetic field point from a magnet in the xy plane. There are then several layers corresponding to different z heights. My goal is to construct a 3 dimensional array from this so I can preform operations on it more easily.

However, I can't seem to construct a 3D array. The final result only registers as a 1d array, although each element is a 2d matrix (see bellow). Any attempts I've made at reproducing this with easier numbers doesn't seem to have the same effect (I'm able to generate a 3d array).

I would really appreciate any suggestions.

My code is as follows:

import xlrd
import numpy as np

book = xlrd.open_workbook('./magnetic_mapping.xlsx')
sheet = book.sheet_by_index(0)
n = 0
layer = []
layers = []
layer_name = None

#import pudb; pudb.set_trace()

while True:
    row = sheet.row_values(n) 
    # Look for the start of a new layer, reading a line with nothing 
    # will throw an IndexError, reading a line with a number will throw
    # an AttributionError.
    try:
        if row[0].split()[0] == 'Layer':
            # If we are already reading in data from a layer, save it before
            # starting a new layer
            if layer_name:
                layers.append(np.array(layer))
            layer_name = row[0]
            print layer_name
            layer = []
    except (IndexError, AttributeError):
        # If we are in a layer, and the line is not empty, read in the data
        if layer_name and row[0]:
            data = np.array(row[2:], dtype='S9')
            data[data == ''] = np.nan         # convert empty strings to nan
            layer.append(data.astype(float))

    # Break loop if at EOF, otherwise increment the spreadsheet row.
    if row[0] == 'END':
        break
    else:
        n+=1

# append the last layer recorded
layers.append(np.array(layer))
layers = np.array(layers)

In terminal:

In [1]: layers.shape
Out[1]: (7,)

In [2]: layers[0].shape
Out[2]: (27, 13)

ali_m · Accepted Answer · 2015-02-03 21:13:35Z

1

The problem is that your 'layers' have inconsistent shapes:

print([ll.shape for ll in layers])
# [(27, 13), (25, 13), (25, 13), (25, 13), (25, 13), (25, 13), (25, 13)]

Your first 'layer' has 27 rows, whereas the rest have 25. How you want to deal with this inconsistency is up to you - you might, for example, want to pad all of the smaller layers with NaNs to make them the same size as the largest layer. For the moment, a quick and dirty hack might be to truncate the rows of the first layer:

layers[0] = layers[0][1:-1]
arr = np.dstack(layers)
print(arr.shape)
# (25, 13, 7)

Note that I'm using np.dstack to concatenate the layers along the third dimension. If you use np.array(layers), you will get a (7,) array of type np.object, which is probably not what you want.

answered Feb 3, 2015 at 21:13

ali_m

74.6k28 gold badges230 silver badges314 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Numpy multidimensional array only registering as 1D array

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related