Convert list of objects to numpy array without coercing objects into array

Question

I want to turn a Python list of objects into a numpy array of objects for easier manipulation of the list, e.g. index slicing, etc. However, numpy coerces list-like objects into arrays during conversion

import numpy as np
class ListLike(object):
    def __init__(self, list):
        self.list = list
        self.otherdata = 'foo'
    def __getitem__(self, i):
        return self.list[i]
    def __len__(self):
        return len(self.list)
o1 = ListLike([1,2])
o2 = ListLike([3,4])
listOfObj = [o1, o2]
numpyArray = np.array(listOfObj)

>>> numpyArray
array([[1, 2],
       [3, 4]])
>>> type(numpyArray[0])
<class 'numpy.ndarray'>

This means I lose the objects and other data and methods associated with them

>>> numpyArray[0].otherdata
AttributeError: 'numpy.ndarray' object has no attribute 'otherdata'

How do I get this result instead?

>>> numpyArray
array([(<__main__.ListLike object at 0x000000002786DAF0>, <__main__.ListLike object at 0x000000002E56B700>)], dtype=object)
>>> type(numpyArray[0])
<class '__main__.ListLike'>

It would be even better if I could also control the depth of the coersion with something like this

o3 = ListLike([o1,o2])
numpyArray = np.array([o3, o3], depth=1)

>>> numpyArray
array([[(<__main__.ListLike object at 0x000000002786DAF0>, <__main__.ListLike object at 0x000000002E56B700>)], [(<__main__.ListLike object at 0x000000002786DAF0>, <__main__.ListLike object at 0x000000002E56B700>)]], dtype=object)

I think you're missing the whole point of numpy and why it's implemented the way it is. If you want to store objects, a list is a great way to do that. — Mark Ransom
– Mark Ransom, Commented Apr 28, 2022 at 0:24

hpaulj · Accepted Answer · 2022-04-28 00:55:14Z

2

In [35]: arr = np.empty(2, object)
In [36]: arr[:] = listOfObj
In [37]: arr
Out[37]: 
array([<__main__.ListLike object at 0x7f3b9c6f1df0>,
       <__main__.ListLike object at 0x7f3b9c6f19d0>], dtype=object)

Element methods/attributes have to be accessed with a list comprehension, just as with the list:

In [39]: arr[0].otherdata
Out[39]: 'foo'
In [40]: [a.otherdata for a in listOfObj]
Out[40]: ['foo', 'foo']
In [41]: [a.otherdata for a in arr]
Out[41]: ['foo', 'foo']

frompyfunc could also be used for this, though timing is about the same:

In [44]: np.frompyfunc(lambda a: a.otherdata,1,1)(arr)
Out[44]: array(['foo', 'foo'], dtype=object)

answered Apr 28, 2022 at 0:55

hpaulj

233k14 gold badges260 silver badges392 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Daniel Exxxe · Accepted Answer · 2022-04-29 21:18:15Z

0

I think that you wish achieve this:

class ListLike(np.array):
    def __init__(self, _list):
        super().__init__(_list)
        self.otherdata = 'foo'

But you can't subclass from a builtin_function.. though you can try subclass from np.ndarray

Instead i've managed to write

class ListLike(object):
    def __init__(self, _list):
        self._list = _list
        self.otherdata = 'foo'
    def __getitem__(self, key): return self._list[key]
    def __str__(self): return self._list.__str__()
    #Note 1
    #def __len__(self): return self._list.__len__()

tested with:

o1 = ListLike([1,2])
o2 = ListLike([3,4])
numpyArray = np.array([o1, o2], dtype=object)      
#Note 2
#numpyArray = np.array([o1, o2, 888], dtype=object) 
print(numpyArray[0], numpyArray[0].otherdata, end='.\t')
print("Expected: [1,2] foo")

Note 1

This works as long as you don't implement __len__ method for LikeList class.

Note 2

If you insist, then append some kind of sorcery to this line.

An empty item as: '',None, []
An instance of another dim as: LikeList([7])
Another object as: 5, ... (Ellipsis) , 'hello', 0.1

(i couldn't figure it out, why np.array doesn't detect dtype=object)

edited Apr 29, 2022 at 21:18

answered Apr 28, 2022 at 3:46

Daniel Exxxe

791 silver badge8 bronze badges

1 Comment

Daniel Exxxe Over a year ago

I think that you just discovered a numpy.array bug. "The inability to respect dtype=object argument"

Collectives™ on Stack Overflow

Convert list of objects to numpy array without coercing objects into array

2 Answers 2

Comments

I think that you wish achieve this:

Instead i've managed to write

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

I think that you wish achieve this:

Instead i've managed to write

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related