reshape numpy matrix where each element is a matrix of different sizes

Question

I have a matrix with 4 elements, where each element is a matrix of different sizes. I am trying to reshape the matrix but it keeps showing me this error: ValueError: could not broadcast input array from shape (3,2) into shape (3,)

Can anyone please explain what this is and how to solve it?

Here is the code so far:

G_data = [-A, 
          -np.identity(n2),
         A,
         -np.identity(n2)]

G_temp = np.array(G_data, dtype=object)
G = object_array.reshape((2, 2))
print('Shape of G:', G.shape, '\n')

And full error message. reshape doesn't normally raise a value error like this. — hpaulj
– hpaulj, Commented May 27, 2024 at 0:14
@NickODell A can be of any size. I tried with sizes 2 by 2 and 2 by 3. — peachy
– peachy, Commented May 30, 2024 at 19:26
@hpaulj G_temp is made of A matrix, -I matrix, -A matrix and -I matrx. So, its 4 by 1. — peachy
– peachy, Commented May 30, 2024 at 19:27

hpaulj · Accepted Answer · 2024-05-30 20:24:03Z

I can produce your error message with:

In [508]: A=np.ones((3,2)); n2=3

In [509]: np.array([-A, np.identity(n2),-A,np.identity(n2)], object)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[509], line 1
----> 1 np.array([-A, np.identity(n2),-A,np.identity(n2)], object)

ValueError: could not broadcast input array from shape (3,2) into shape (3,)

Note this occurs in the object array creation, NOT in the reshape.

It's a good idea to report (and read yourself) the WHOLE error message. Ignoring the error stack can mislead us - and you.

edit

Making an object dtype array from a mix of (3,2) and (3,3) array gives this error:

In [3]: A=np.ones((3,2),int); N = np.identity(3)

In [4]: np.array([A,N,-N,A],object)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[4], line 1
----> 1 np.array([A,N,-N,A],object)

ValueError: could not broadcast input array from shape (3,2) into shape (3,)

Mixing (3,2) and (2,2) is ok:

In [5]: A=np.ones((3,2),int); N = np.identity(2)

In [6]: np.array([A,N,-N,A],object)
Out[6]: 
array([array([[1, 1],
              [1, 1],
              [1, 1]]), array([[1., 0.],
                               [0., 1.]]), array([[-1., -0.],
                                                  [-0., -1.]]),
       array([[1, 1],
              [1, 1],
              [1, 1]])], dtype=object)

Similarly you found that (2,3) and (2,2) had problems. Even with object dtype, np.array has problems with certain mixes of shapes. Here it occurs when the first dimension matches. Details are buried in the np.array code, but the problem has been known for a long time. It seem to come up more often when making 'ragged' arrays was 'automatic'. Now you at least have to specify object dtype, so unexpected 'ragged' arrays are not so common.

Anyways, the way around this is to create a 'blank' array of the right shape and dtype, and then fill it.

In [7]: res = np.empty((4,),object);res
Out[7]: array([None, None, None, None], dtype=object)

In [8]: res[:] = [A,N,-N,A]

In [9]: res
Out[9]: 
array([array([[1, 1],
              [1, 1],
              [1, 1]]), array([[1., 0.],
                               [0., 1.]]), array([[-1., -0.],
                                                  [-0., -1.]]),
       array([[1, 1],
              [1, 1],
              [1, 1]])], dtype=object)

You can even start with a 2d (or nd) array, but the source list still has to match:

In [10]: res = np.empty((2,2),object);res
Out[10]: 
array([[None, None],
       [None, None]], dtype=object)

In [11]: res[:] = [A,N,-N,A]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[11], line 1
----> 1 res[:] = [A,N,-N,A]

ValueError: could not broadcast input array from shape (4,) into shape (2,2)

In [12]: res[:] = [[A,N],[-N,A]]

In [13]: res
Out[13]: 
array([[array([[1, 1],
               [1, 1],
               [1, 1]]), array([[1., 0.],
                                [0., 1.]])],
       [array([[-1., -0.],
               [-0., -1.]]), array([[1, 1],
                                    [1, 1],
                                    [1, 1]])]], dtype=object)

This works even when the subarrays match in shape:

In [14]: res[:] = [[3*N,N],[-N,2*N]]; res
Out[14]: 
array([[array([[3., 0.],
               [0., 3.]]), array([[1., 0.],
                                  [0., 1.]])],
       [array([[-1., -0.],
               [-0., -1.]]), array([[2., 0.],
                                    [0., 2.]])]], dtype=object)

Where as using just np.array would result in an 4d array of numbers:

In [18]: res = np.array([[3*N,N],[-N,2*N]], object); res.shape
Out[18]: (2, 2, 2, 2)

For some mixes of shapes, specifying dtype=object is not enough. The surest way is to first create an object dtype array of the right size, and then fill it from the list.

Collectives™ on Stack Overflow

reshape numpy matrix where each element is a matrix of different sizes

1 Answer 1

edit

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

edit

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related