Broadcasting over two arrays with different shapes (Numpy-Python)

Question

Suppose I have the following arrays:

first_array = array([[1, 8, 3, 9, 2],
                     [2, 6, 4, 1, 9],
                     [4, 2, 12, 8, 16],
                     [5, 3, 7, 18, 21],
                     [6, 20, 4, 8, 24]])

So an array with shape (5, 5)

Now I have a second array, which is a slice of the first:

second_array = array([[1, 8, 3, 9, 2],
                     [2, 6, 4, 1, 9]])

An array with shape (2, 5).

Now I want to subtract every vector of the first array by the vectors of the second array subsequently(excluding- in the first array- the vector of the second array I'm using to subtract), element-wise. I want to this for every vector of the second array.

So I want to have as an output:

subtracted_array = array([[[1, -2, 1, -8, 7],
                           [3, -6, 9, -1, 14],
                           [4, -5, 4, 9, 19],
                           [5, 12, 1, -1, 22]],
                          [[-1, 2, -1, 8, -7],
                           [2, -4, 8, 7, 7],
                           [3, -3, 3, 17, 12],
                           [4, 14, 0, 7, 15]]])

So this is an array with shape (2, 4, 5)

How do we do this with broadcasting?

Please add a Minimal reproducible example so that we can help you more. — yann ziselman
– yann ziselman, Commented Jun 21, 2021 at 12:37
Your example doesn't make sense. What are you subtracting off when you reach the end of the second array? — Mad Physicist
– Mad Physicist, Commented Jun 21, 2021 at 12:57
Please show an example with arrays shaped (6, 3) and (3, 3) or something like that, and generate the expected output by hand — Mad Physicist
– Mad Physicist, Commented Jun 21, 2021 at 12:57
I will edit it. But give me some time to do it, instead of closing to vote immediately. — Steven01123581321
– Steven01123581321, Commented Jun 21, 2021 at 12:59

yann ziselman · Accepted Answer · 2021-06-21 14:40:38Z

3

Does this code do what you meant you needed done?
You are welcomed to test it on your test case and update me if you need more help.

import numpy as np
arr = np.arange(50).reshape(10, 5)
arr_slice = arr[:2, :]

# "outer" tensor subtraction
arr_sub = arr_slice[:, None, :] - arr[None, :, :]
# create an index map of the vector combinations
idx0, idx1 = np.mgrid[:2, :10]
idx0, idx1 = idx0.flatten(), idx1.flatten()
# find the irrelevent combinations and remove them
remove_same = (1 - (idx0 == idx1)).astype(bool)
idx0, idx1 = idx0[remove_same], idx1[remove_same]

arr_sub = arr_sub[idx0, idx1, :].reshape(2, 9, 5)

EDIT: here's a more efficient method:

import numpy as np
arr = np.arange(50).reshape(10, 5)
arr_slice = arr[:2, :]

# create an index map of the vector combinations
idx0, idx1 = np.mgrid[:2, :10]
idx0, idx1 = idx0.flatten(), idx1.flatten()
# find the irrelevent combinations and remove them
remove_same = (1 - (idx0 == idx1)).astype(bool)
idx0, idx1 = idx0[remove_same], idx1[remove_same]

arr_sub = arr_slice[idx0, :] - arr[idx1, :]

arr_sub = arr_sub.reshape(2, 9, 5)

output:

arr = 
[[ 0  1  2  3  4]
 [ 5  6  7  8  9]
 [10 11 12 13 14]
 [15 16 17 18 19]
 [20 21 22 23 24]
 [25 26 27 28 29]
 [30 31 32 33 34]
 [35 36 37 38 39]
 [40 41 42 43 44]
 [45 46 47 48 49]]

arr_slice = 
[[0 1 2 3 4]
 [5 6 7 8 9]]

arr_sub = 
[[[ -5  -5  -5  -5  -5]
  [-10 -10 -10 -10 -10]
  [-15 -15 -15 -15 -15]
  [-20 -20 -20 -20 -20]
  [-25 -25 -25 -25 -25]
  [-30 -30 -30 -30 -30]
  [-35 -35 -35 -35 -35]
  [-40 -40 -40 -40 -40]
  [-45 -45 -45 -45 -45]]

 [[  5   5   5   5   5]
  [ -5  -5  -5  -5  -5]
  [-10 -10 -10 -10 -10]
  [-15 -15 -15 -15 -15]
  [-20 -20 -20 -20 -20]
  [-25 -25 -25 -25 -25]
  [-30 -30 -30 -30 -30]
  [-35 -35 -35 -35 -35]
  [-40 -40 -40 -40 -40]]]

EDIT 2: I subtracted the arrays in the wrong order (the whole array from the slice instead of the other way around). So here's a fix and this time I'm using your example:

import numpy as np

first_array = np.array(
    [[1, 8, 3, 9, 2],
     [2, 6, 4, 1, 9],
     [4, 2, 12, 8, 16],
     [5, 3, 7, 18, 21],
     [6, 20, 4, 8, 24]]
)

arr_slice = slice(None, 2)
second_array = first_array[arr_slice]

expected_subtracted_array = np.array(
    [[[[1, -2, 1, -8, 7],
       [3, -6, 9, -1, 14],
       [4, -5, 4, 9, 19],
       [5, 12, 1, -1, 22]],
      [[-1, 2, -1, 8, -7],
       [2, -4, 8, 7, 7],
       [3, -3, 3, 17, 12],
       [4, 14, 0, 7, 15]]]]
)

# create an index map of the vector combinations
idx0, idx1 = np.mgrid[:second_array.shape[0], :first_array.shape[0]]
idx0, idx1 = idx0.flatten(), idx1.flatten()
# find the irrelevent combinations and remove them
remove_same = (1 - (idx0 == idx1)).astype(bool)
idx0, idx1 = idx0[remove_same], idx1[remove_same]
arr_sub = first_array[idx1, :] - second_array[idx0, :]
arr_sub = arr_sub.reshape(second_array.shape[0], first_array.shape[0]-1, first_array.shape[1])

(arr_sub == expected_subtracted_array).all()

Output:

True

edited Jun 21, 2021 at 14:40

answered Jun 21, 2021 at 12:34

yann ziselman

2,0027 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Mad Physicist Over a year ago

Computing a bunch of zeros is wasteful. Will post answer momentarily.

yann ziselman Over a year ago

@MadPhysicist, you're right. I added a slightly more efficient method

Steven01123581321 Over a year ago

yes, I wrongly phrased the question in the beginning. Adjusted your code to what you have edited right now and it works indeed. Brilliant. Upvoted and accepted your answer.

Collectives™ on Stack Overflow

Broadcasting over two arrays with different shapes (Numpy-Python)

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related