Sum up all values from all numpy subarray of different shape inside an array

Question

I have a numpy.array of numpy arrays of different shapes. When I call np.sum(my_array) I get this error:

Traceback (most recent call last):
return umr_sum(a, axis, dtype, out, keepdims)
ValueError: operands could not be broadcast together with shapes (13,5) (5,3)

All I want is sum of all values across all arrays like sum(my_array) = some float number

Is there some parameter that I missed or another method? I can only think of something like this

np.sum([np.sum(a) for a in my_array])

Is this an optimal way?

Update:

print(type(my_array))
print((my_array).shape)
print([(type(sub_array), sub_array.shape) for sub_array in my_array])

output:

<class 'numpy.ndarray'>
(2,)
[(<class 'numpy.ndarray'>, (13, 5)), (<class 'numpy.ndarray'>, (5, 3))]

What is my_array? Is it a python list containing numpy arrays? — John Lyon
– John Lyon, Commented Nov 6, 2014 at 21:30
What kind of ndarray is this that contains other ndarrays of varying shapes? Please print the results of type(my_array) for us. — derricw
– derricw, Commented Nov 6, 2014 at 21:35
TIL that you can have ndarrays with dtype 'object' whose elements can be arbitrary shapes and types. — derricw
– derricw, Commented Nov 6, 2014 at 21:57

espang · Accepted Answer · 2014-11-06 21:55:04Z

4

Using a generator should be better in most cases:

np.sum(np.sum(a) for a in my_array)

Without the '[ ... ]' you don't create a list.

%timeit np.sum( np.sum(a) for a in my_array )

100000 loops, best of 3: 5.73 µs per loop

%timeit np.sum( [np.sum(a) for a in my_array] )

100000 loops, best of 3: 9.97 µs per loop

answered Nov 6, 2014 at 21:55

espang

6836 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

userqwerty1 Over a year ago

Wow. Didn't know that I can omit '[..]' thanks. How do you make this %timeit? I've tried it but couldn't make it work like yours. Please, can you tell or post a link?

espang Over a year ago

I wrote this in a IPython Notebook. You should take a look at that, it is included when installing python with anaconda or enthougt. %timeit can be used there.

SlimJim · Accepted Answer · 2014-11-06 22:22:14Z

0

a = np.array(map(np.arange, range(16, 32)))

Eyy![28]: %timeit np.sum(map(np.sum, a))
10000 loops, best of 3: 90.5 µs per loop

Eyy![29]: %timeit np.sum(np.sum(b) for b in a)
10000 loops, best of 3: 86 µs per loop

Eyy![30]: %timeit np.sum([np.sum(b) for b in a])
10000 loops, best of 3: 90.2 µs per loop

also note that many times it can be easier to just have zeropadded numpy arrays if you know the maximum size in advance.

edited Nov 6, 2014 at 22:22

answered Nov 6, 2014 at 22:13

SlimJim

2,2722 gold badges22 silver badges26 bronze badges

Collectives™ on Stack Overflow

Sum up all values from all numpy subarray of different shape inside an array

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related