element wise sum of structured arrays in numpy

Question

I wonder if it is possible at all to perform element-wise sum (or other operations) of two structured numpy arrays of an identical shape.

arr1 = np.array([[1,2,3],[2,3,4]], dtype=[("x", "f8"),("y", "f8")])
arr2 = np.array([[5,4,3],[9,6,4]], dtype=[("x", "f8"),("y", "f8")])
arr3 = np.sum(arr1, arr2)

says "ufunc 'add' did not contain a loop with signature matching types dtype([('x', '

If it is not possible, it will be great to understand why that is impossible or impractical to implement in numpy.

Have to do the sum, or other math field by field. In general fields may include strings and/or other dtypes that don't implement math. — hpaulj
– hpaulj, Commented Feb 4, 2019 at 14:10
if the dtype is uniform... arr.view((arr.dtype[0], len(arr.dtype.names))).sum(axis=0) … but separate it out into components so you can see what is going on — NaN
– NaN, Commented Feb 4, 2019 at 14:24
@hpaulj That's very true. I never store strings in ndarrays, so I overlooked that simple fact...! If you want, I will take it as the answer. It may look too short, but your comment clearly solved the problem :) — Hoseung Choi
– Hoseung Choi, Commented Feb 4, 2019 at 14:40
This works for you? arr.view(np.float32).reshape(arr.shape + (-1,)) I think it is similar to @NaN solution — LocoGris
– LocoGris, Commented Feb 4, 2019 at 14:51
@NaN - I think my example was confusing. I've updated the example. Your previous suggestion is only applicable to a single array and shrinks the array along an axis. So, I'm afraid they don't work. — Hoseung Choi
– Hoseung Choi, Commented Feb 4, 2019 at 15:03

hpaulj · Accepted Answer · 2019-02-04 19:21:21Z

With your array:

In [236]: arr1 = np.array([[1,2,3],[2,3,4]], dtype=[("x", "f8"),("y", "f8")])
In [237]: arr1
Out[237]: 
array([[(1., 1.), (2., 2.), (3., 3.)],
       [(2., 2.), (3., 3.), (4., 4.)]], dtype=[('x', '<f8'), ('y', '<f8')])
In [238]: arr1['x']
Out[238]: 
array([[1., 2., 3.],
       [2., 3., 4.]])

Normally the data for a structured array is provided in the form a list(s) of tuples., same as displayed in Out[237]. Without the tuples np.array assigns the same value to both fields.

You have to do math on each field separately:

In [239]: arr1['y'] *= 10
In [240]: arr1
Out[240]: 
array([[(1., 10.), (2., 20.), (3., 30.)],
       [(2., 20.), (3., 30.), (4., 40.)]],
      dtype=[('x', '<f8'), ('y', '<f8')])

Math operations are defined for simple dtypes like int and float, and uses compiled code where possible.

This error means that the add ufunc has not been defined for this compound dtype. And I think that's true for all compound dtypes.

In [242]: arr1 + arr1
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-242-345397c600ce> in <module>()
----> 1 arr1 + arr1

TypeError: ufunc 'add' did not contain a loop with signature matching types dtype([('x', '<f8'), ('y', '<f8')]) dtype([('x', '<f8'), ('y', '<f8')]) dtype([('x', '<f8'), ('y', '<f8')])

Since the fields in this case have the same base dtype, we can define another compound dtype that can 'view' it:

In [243]: dt2 = np.dtype([('xy', 'f8', 2)])
In [244]: arr2 = arr1.view(dt2)
In [245]: arr2
Out[245]: 
array([[([ 1., 10.],), ([ 2., 20.],), ([ 3., 30.],)],
       [([ 2., 20.],), ([ 3., 30.],), ([ 4., 40.],)]],
      dtype=[('xy', '<f8', (2,))])
In [246]: arr2['xy']
Out[246]: 
array([[[ 1., 10.],
        [ 2., 20.],
        [ 3., 30.]],

       [[ 2., 20.],
        [ 3., 30.],
        [ 4., 40.]]])

Math on that field will be seen in the original array:

In [247]: arr2['xy'] += .1
In [248]: arr2
Out[248]: 
array([[([ 1.1, 10.1],), ([ 2.1, 20.1],), ([ 3.1, 30.1],)],
       [([ 2.1, 20.1],), ([ 3.1, 30.1],), ([ 4.1, 40.1],)]],
      dtype=[('xy', '<f8', (2,))])
In [249]: arr1
Out[249]: 
array([[(1.1, 10.1), (2.1, 20.1), (3.1, 30.1)],
       [(2.1, 20.1), (3.1, 30.1), (4.1, 40.1)]],
      dtype=[('x', '<f8'), ('y', '<f8')])

We can also view it as a simple dtype, but will have to adjust the shape:

In [250]: arr3 = arr1.view('f8')
In [251]: arr3
Out[251]: 
array([[ 1.1, 10.1,  2.1, 20.1,  3.1, 30.1],
       [ 2.1, 20.1,  3.1, 30.1,  4.1, 40.1]])
In [252]: arr3.reshape(2,3,2)
Out[252]: 
array([[[ 1.1, 10.1],
        [ 2.1, 20.1],
        [ 3.1, 30.1]],

       [[ 2.1, 20.1],
        [ 3.1, 30.1],
        [ 4.1, 40.1]]])

I like the trick using a view to get a "pure" number array! I think this is overkill to the present problem, but will be useful at some point.

Collectives™ on Stack Overflow

element wise sum of structured arrays in numpy

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related