Numpy arrays containing iterable objects

Question

Normally it's possible to put arbitrary objects into numpy arrays:

class Foo(object):
    pass
np.array([ Foo() ])
>>> array([<__main__.Foo object at 0x10d7c3610>], dtype=object)

However, it appears that objects implementing __len__ and __getitem__ are "unpacked" automatically:

class Foo(object):
    def __len__(self): return 3
    def __getitem__(self, i): return i*11
np.array([ Foo() ])
>>> array([[0, 11, 22]])

Is there any way to stop numpy from unpacking objects in this way? What I want is to put some objects into a numpy array and have them be stored as the objects themselves, without being unpacked. So the desired behavior is:

class Foo(object):
    def __len__(self): return 3
    def __getitem__(self, i): return i*11
np.array([ Foo() ])
>>> array([<__main__.Foo object at 0x10d7c3610>], dtype=object)

Now I understand that the duck typing idea implies that numpy should unpack anything that looks like a list. But perhaps it's possible to mark the class Foo in some way to tell numpy not to unpack it? For example, an ABC like:

numpy.Nonenumerable.register(Foo)

user2357112 · Accepted Answer · 2014-03-10 00:44:41Z

4

x = numpy.empty(appropriate_shape, dtype=object)
x[:] = your_list_of_foos

For example, for a 1-dimensional array of 1 Foo,

x = numpy.empty([1], dtype=object)
x[:] = [Foo()]

This has the benefit of working for types you don't control or that other parts of the system may wish to be unpacked. For example, if you want your list of 2 lists to be treated as a 1D array of lists,

x = numpy.empty([2], dtype=object)
x[:] = [[], []]

answered Mar 10, 2014 at 0:44

user2357112

286k32 gold badges490 silver badges570 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

DSM Over a year ago

+1, although I'd use empty rather than zeros, more for conceptual reasons than practical ones.

user2357112 Over a year ago

@DSM: Misremembered how empty behaves for a dtype of object. I thought it would be full of arbitrary memory addresses, rather than None. Switched the code to use empty.

Alex Flint Over a year ago

This works, thanks! However, creating numpy arrays in this way is pretty inconvenient, particularly because it doesn't allow things like np.dot([Foo(),Foo()], [Foo(),Foo()]), since in that case numpy does the conversion from list to ndarray internally. Of course it's possible to rewrite this using np.empty as you described, but it blows up the expression quite a bit. You did answer my question as stated, though, so I'm ticking this - thanks!

Graeme Stuart · Accepted Answer · 2014-03-10 00:39:12Z

0

perhaps enclose your array-like object in a container?

class Bar():
    def __init__(self):
        self.foo = Foo()

np.array([Bar()])

answered Mar 10, 2014 at 0:39

Graeme Stuart

6,0632 gold badges28 silver badges46 bronze badges

Comments

aruisdante · Accepted Answer · 2014-03-10 00:44:10Z

0

The intent of numpy is to work on numbers. So it's expected that unpacking those lists would be the correct behavior in a single-dimensional array (I.E. you're producing a concatenation of vectors).

What you probably want is an n-dimensional array, or perhaps a matrix.

Alternatively I believe you may be able to get the desired result using the optional parameters of the array constructor

np.array(MyListOfLists, dtype = list)

edited Mar 10, 2014 at 0:44

answered Mar 10, 2014 at 0:37

aruisdante

9,1652 gold badges36 silver badges38 bronze badges

2 Comments

user2357112 Over a year ago

This is an n-dimensional array.

Alex Flint Over a year ago

I do understand why this is numpy's default behavior and I agree it makes sense as default behavior. I was hoping there was a way to explicitly ask numpy not to unpack my objects, though. FWIW using the dtype parameter to the constructor does not help: np.array([Foo()], dtype=object) -> array([[0, 1, 2]], dtype=object)

Collectives™ on Stack Overflow

Numpy arrays containing iterable objects

3 Answers 3

3 Comments

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related