Weird behaviour with numpy array operations

Question

Explain this:

>>> a = np.arange(10)
>>> a[2:]
array([2, 3, 4, 5, 6, 7, 8, 9])
>>> a[:-2]
array([0, 1, 2, 3, 4, 5, 6, 7])
>>> a[2:] - a[:-2]
array([2, 2, 2, 2, 2, 2, 2, 2])
>>> a[2:] -= a[:-2]
>>> a
array([0, 1, 2, 2, 2, 3, 4, 4, 4, 5])

The expected result is of course array([0, 1, 2, 2, 2, 2, 2, 2, 2, 2]).

I'm going to guess this is something to do with numpy parallelising things and not being smart enough to work out that it needs to make a temporary copy of the data first (or do the operation in the correct order).

In other words I suspect it is doing something naive like this:

for i in range(2, len-2):
    a[i] -= a[i-2]

For reference it works in Matlab and Octave:

a = 0:9
a(3:end) = a(3:end) - a(1:end-2)

a =

  0  1  2  3  4  5  6  7  8  9

a =

  0  1  2  2  2  2  2  2  2  2

And actually it works fine if you do:

a[2:] = a[2:] - a[:-2]

So presumably this means that a -= b is not the same as a = a - b for numpy!

Actually now that I come to think of it, I think Mathworks gave this as one of the reasons for not implementing the +=, -=, /= and *= operators!

"So presumably this means that a -= b is not the same as a = a - b for numpy!" These are not the same in pure python either stackoverflow.com/questions/2347265/… — Akavall
– Akavall, Commented Dec 4, 2013 at 15:15

JoshAdel · Accepted Answer · 2013-12-04 14:35:30Z

4

When you slice a numpy array as you are doing in the example, you get a view of the data rather than a copy.

See:

http://scipy-lectures.github.io/advanced/advanced_numpy/#example-inplace-operations-caveat-emptor

answered Dec 4, 2013 at 14:35

JoshAdel

69.1k27 gold badges146 silver badges146 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

bogatron · Accepted Answer · 2013-12-04 15:37:06Z

1

The unexpected behavior is due to array aliasing because (as @JoshAdel stated in his answer), slicing returns a view, rather than a copy of the array. Your example of the "naive" loop already explains how the result is computed. But I'll add two points to your explanation:

First, the unexpected behavior is not due to numpy parallelizing operations. If the operation were parallelized, then you shouldn't expect to [consistently] see the result of the naive loop (since that result depends on ordered execution of the loop). If you repeat your experiment several times - even for large arrays - you should see the same result.

Second, while your presumption is true in general, I would state it this way:

a -= b is the same as a = a - b for two numpy arrays when a and b are not aliased.

edited Dec 4, 2013 at 15:37

answered Dec 4, 2013 at 15:22

bogatron

19.3k7 gold badges56 silver badges49 bronze badges

Collectives™ on Stack Overflow

Weird behaviour with numpy array operations

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related