Python object array of empty arrays

Question

I am attempting to create a numpy array of empty arrays without using loops. Using loops, I can use a simplified operation like

a = np.empty((3, 3), object)
for i in range(a.size):
    a.ravel()[i] = np.array([])

Or a marginally more sophisticated approach based on np.nditer:

a = np.empty((3, 3), object)
it = np.nditer(a, flags=['multi_index', 'refs_ok'])
for i in it:
    a[it.multi_index] = np.array([])

I can't seem to find an indexing expression that will allow me to make this assignment in a vectorized manner.

I've tried the following:

>>> a[:] = np.zeros((0, 3, 3))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: could not broadcast input array from shape (0,3,3) into shape (3,3)

>>> a[..., None] = np.zeros((3, 3, 0))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: could not broadcast input array from shape (3,3,0) into shape (3,3,1)

>>> a = np.full((3, 3), np.array([]), object)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.9/site-packages/numpy/core/numeric.py", line 343, in full
    multiarray.copyto(a, fill_value, casting='unsafe')
  File "<__array_function__ internals>", line 5, in copyto
ValueError: could not broadcast input array from shape (0,) into shape (3,3)

Even np.nditer does not allow me to write back:

>>> a = np.empty((3, 3), object)
>>> for x in np.nditer(a, flags=['refs_ok'], op_flags=['readwrite']):
...    x[...] = np.array([])
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ValueError: could not broadcast input array from shape (0,) into shape ()

Is there any way to make an object array of empty arrays in a vectorized manner?

The exact output I'm looking for is

array([[array([], dtype=float64), array([], dtype=float64), array([], dtype=float64)],
       [array([], dtype=float64), array([], dtype=float64), array([], dtype=float64)],
       [array([], dtype=float64), array([], dtype=float64), array([], dtype=float64)]], dtype=object)

For the purposes of this question, I don't care if the references are all the same array or different arrays.

How is your required output different from an array of shape (3, 3, 0)? — ShlomiF
– ShlomiF, Commented Jun 30, 2021 at 6:43
@ShlomiF: Completely different behavior on cell assignment, different shape, completely different memory layout, etc. For example, the desired array actually has contents, and you can do stuff like a[0, 0] = 'asdf' to store stuff in the array. Trying to do that with an array of shape (3, 3, 0) wouldn't store anything - it'd broadcast the assignment over a subarray of shape (0,) and store the string in the resulting 0 cells. — user2357112
– user2357112, Commented Jun 30, 2021 at 6:54

user2357112 · Accepted Answer · 2021-06-30 07:11:43Z

2

You can get NumPy's broadcasting handling to treat an array as a scalar object instead of a source to broadcast from, by putting it inside another array:

template = numpy.empty((), dtype=object)
element = numpy.array([], dtype=float)
template[()] = element

Then you can perform a broadcasted assignment to store the element array in every cell of a result array:

result = numpy.empty((3, 3), dtype=object)
result[:] = template

The broadcasting logic will broadcast over template instead of element. This results in a 3-by-3 result array of object dtype, where every cell holds a reference to the single element array.

result = numpy.full((3, 3), template) would also work, but it seems even more confusing than the slice assignment, and the slice assignment is already pretty confusing - it's not immediately obvious whether the cells of result end up holding references to template or element.

edited Jun 30, 2021 at 7:11

answered Jun 30, 2021 at 7:05

user2357112

286k32 gold badges490 silver badges569 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

hpaulj Over a year ago

All elements of result are the same object (same id). Not it matters much, since there is little that one can do in-place to such an array. Well, inplace reshape works: result[0,0].shape=(0,2).

user2357112 Over a year ago

@hpaulj: Indeed. The question says that this is okay: "For the purposes of this question, I don't care if the references are all the same array or different arrays." The whole operation is weird enough that it's hard to say whether this aspect is weird in context.

hpaulj Over a year ago

So to avoid the OP's broadcasting problems, the object being assigned must itself be an object dtype array.

Collectives™ on Stack Overflow

Python object array of empty arrays

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related