3

Recently, I've faced the problem with np.array(list) conversion. Assume we have a list of 3 numpy 2D arrays with shapes (x, y), (x, y), (x, z) so that shape[0] is the same for all arrays in the list. In that case, conversion to array fails with

ValueError: could not broadcast input array from shape (x, z) into shape (x)

Numpy tries to create and array of shape (3, x, y) instead of leaving it list-like structure (array of different arrays). If at least one shape[0] differs from the other, we get what we want, array of arrays with shape (3,)

I overcame that problem by adding an element of different type to list, and using np.array(list)[:-1]. So, is it a bug, or I missed something (like np.array() params, etc.)?

Some examples:

>>> import numpy as np
>>> x = np.ones((3,2))
>>> y = np.ones((3,2))
>>> z = np.ones((3,3))
>>> a = np.ones((2,3))
>>> xyz = np.array([x,y,z])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: could not broadcast input array from shape (3,2) into shape (3)
>>> xza = np.array([x,z,a])
[array([[ 1.,  1.],
        [ 1.,  1.],
        [ 1.,  1.]]) 
 array([[ 1.,  1.,  1.],
        [ 1.,  1.,  1.],
        [ 1.,  1.,  1.]])
 array([[ 1.,  1.,  1.],
        [ 1.,  1.,  1.]])]
>>> xyz2 = np.array([x,y,z,'tmp'])[:-1]
[array([[ 1.,  1.],
        [ 1.,  1.],
        [ 1.,  1.]])
 array([[ 1.,  1.],
        [ 1.,  1.],
        [ 1.,  1.]])
 array([[ 1.,  1.,  1.],
        [ 1.,  1.,  1.],
        [ 1.,  1.,  1.]])]

1 Answer 1

1

On version 1.9 I get different results (error)

3 same shape arrays produce the expected 3d array:

In [726]: np.array([np.ones((3,2)),np.ones((3,2)),np.ones((3,2))]).shape
Out[726]: (3, 3, 2)

but if one differs in columns, I get 3x3 object array filled with None:

In [727]: np.array([np.ones((3,2)),np.ones((3,2)),np.ones((3,3))])
Out[727]: 
array([[None, None, None],
       [None, None, None],
       [None, None, None]], dtype=object)

What's your version? More evidence of an area under development. This test rings a bell - I think I've seen a SO question like this before. But I'm not sure how search for it.

Matching github issue:

https://github.com/numpy/numpy/issues/6591

and from your comment:

https://github.com/numpy/numpy/issues/7103

which references a recent SO question (which I must have looked at):

ValueError: cannot copy sequence with size 5 to array axis with dimension 2


As you say, if the first dimension differs it creates the object array.

Most commonly I've seen it with 1d arrays or lists, e.g.

In [711]: np.array([[1,2,3],[2,3,4],[4,5,6]])
Out[711]: 
array([[1, 2, 3],
       [2, 3, 4],
       [4, 5, 6]])

In [712]: np.array([[1,2,3],[2,3,4],[4,5]])
Out[712]: array([[1, 2, 3], [2, 3, 4], [4, 5]], dtype=object)

I've also recommended creating a np.array((3,),dtype=object) array, and filling that.

Looks like it is testing the first dimension for uniformity, and starts to fill it in. Your error suggests it is trying to do some broadcasting, though I can't think of an example where that works.

We could examine the numpy Github issues. There probably are a variety of ones related to object arrays. This is something of a late comer kludge, expanding numpy arrays beyond the original uniform multidimensional concept.

Sign up to request clarification or add additional context in comments.

1 Comment

Thanks for the answer! My numpy version is 1.10.1. Seems, that they noticed the error, so now it throws ValueError instead array of 'None's. I explored GitHub issues, and found that issue. Not exactly the same problem, but it stems from the same place i suppose. May be implementing full shape (not only 1st dimension) check will help.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.