How to get a sum of a structured numpy array?

Question

I am trying to get the sum of a structured array. Do a need a "for" loop around the field names, or can I do it in one line. I do this about a million times per run, so I need to keep it quick.

>>> test = np.array([(1,2,3),(4,5,6)], dtype={'names' : ["One", "Two", "Three"], 'formats' : ["f8"]*3})
>>> test
array([(1., 2., 3.), (4., 5., 6.)],
      dtype=[('One', '<f8'), ('Two', '<f8'), ('Three', '<f8')])
>>> np.sum(test)
...
numpy.core._exceptions._UFuncNoLoopError: ufunc 'add' did not contain a loop with signature matching types (dtype([('One', '<f8'), ('Two', '<f8'), ('Three', '<f8')]), dtype([('One', '<f8'), ('Two', '<f8'), ('Three', '<f8')])) -> None

similar for np.sum(test, axis=0)

I was expecting a structured array: [(5, 7, 9), dtype=...]. At least, no errors.

You could use view(): stackoverflow.com/questions/54517950/… However, note that this is not type safe. — Nick ODell
– Nick ODell, Commented Aug 4, 2023 at 22:14
As long as the number fields is relatively small (conpared to the number of records), there's little harm in iterating on them. You could also also explore using numpy.lib.recfunctions.structured_to_unstructured to get a 2d array. Many of the recfunctions iterate on fields. — hpaulj
– hpaulj, Commented Aug 4, 2023 at 22:39

juanpa.arrivillaga · Accepted Answer · 2023-08-04 22:37:12Z

0

So, in this case, since your structure has the same primitive types, you can use a view easily enough:

>>> import numpy as np
>>> test = np.array([(1,2,3),(4,5,6)], dtype={'names' : ["One", "Two", "Three"], 'formats' : ["f8"]*3})
>>> test.view('f8').reshape(-1,3).sum(axis=0)
array([5., 7., 9.])

Now, if you couldn't easily create a view from your original structured dtype, you could use a loop over the fields, which shouldn't be terribly slow:

>>> result = np.zeros_like(test, shape=(1,))
>>> for field in test.dtype.fields:
...     result[field] = test[field].sum()
...
>>> result
array([(5., 7., 9.)],
      dtype=[('One', '<f8'), ('Two', '<f8'), ('Three', '<f8')])

answered Aug 4, 2023 at 22:37

juanpa.arrivillaga

97.6k14 gold badges141 silver badges190 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

engineercliff Over a year ago

That's it, thanks. I use scipy.minimize, so outside of the function, I can use the for loop. Inside, I'll use view.

Collectives™ on Stack Overflow

How to get a sum of a structured numpy array?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related