Numpy array with elements of different last axis dimensions

Question

Assume the following code:

import numpy as np

x = np.random.random([2, 4, 50])
y = np.random.random([2, 4, 60])

z = [x, y]
z = np.array(z, dtype=object)

This gives a ValueError: could not broadcast input array from shape (2,4,50) into shape (2,4)

I can understand why this error would occur since the trailing (last) dimension of both arrays is different and a numpy array cannot store arrays with varying dimensions.

However, I happen to have a MAT-file which when loaded in Python through the io.loadmat() function in scipy, contains a np.ndarray with the following properties:

from scipy import io

mat = io.loadmat(file_name='gt.mat')

print(mat.shape)
> (1, 250)

print(mat[0].shape, mat[0].dtype)
> (250,) dtype('O')

print(mat[0][0].shape, mat[0][0].dtype)
> (2, 4, 54), dtype('<f8')

print(mat[0][1].shape, mat[0][1].dtype)
> (2, 4, 60), dtype('<f8')

This is pretty confusing for me. How is the array mat[0] in this file holding numpy arrays with different trailing dimensions as objects while being a np.ndarray itself and I am not able do so myself?

Using np.array to make object dtype arrays is not reliable. Some cases it makes a 1d array, others, multidimensional. Or as in your case it raises an error. loadmat must be doing somethink like that suggesting in the answer - creating an array with the desired shape, and filling that (from a list). This works in all cases. — hpaulj
– hpaulj, Commented Apr 16, 2022 at 23:16

Jaka · Accepted Answer · 2022-04-16 21:47:27Z

4

When calling np.array on a nested array, it will try to stack the arrays anyway. Note that you are dealing with objects in both cases. It is still possible. One way would be to first create an empty array of objects and then fill in the values.

z = np.empty(2, dtype=object)
z[0] = x
z[1] = y

Like in this answer.

answered Apr 16, 2022 at 21:47

Jaka

1501 silver badge8 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

hpaulj Over a year ago

z[:] = [x, y] should work

Ahmed Mustafa Over a year ago

That was stupidly simple

Collectives™ on Stack Overflow

Numpy array with elements of different last axis dimensions

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related