Is python numpy array operation += thread safe?

Question

How can I be sure this code is thread safe ?

import numpy as np
from threading import Thread

n_threads = 5

ones = np.ones((5, 5))
A = np.ones((5, 5))

def my_function():
    global A
    for i in range(250):
        A += ones # is += thread safe ?

threads = [Thread(target=my_function) for i in range(n_threads)]

for t in threads:
    t.start()

for t in threads:
    t.join()

print(A)

Should A be a critical shared memory ? Surprisingly I always get the same result, and all entries of the array all have the same value. I would expect for the threads to update the values of the matrix, and for some to be lost...

Thanks.

In my opinion, it is not safe even within one thread. When the array has "many" rows (whatever "many" means), they are often updated in blocks, so if if you do a+=a[0], the a[0] may change as you go. — DYZ
– DYZ, Commented Sep 3, 2017 at 23:15
@DYZ: They actually introduced defined semantics for operations like that in 1.13, the newest release. — user2357112
– user2357112, Commented Sep 3, 2017 at 23:21
While the GIL does provide some level of protection for operations like this on CPython, there are still cases where it fails, and other Python implementations (some of which do have some level of support for NumPy) may do no locking or finer-grained locking. — user2357112
– user2357112, Commented Sep 3, 2017 at 23:24
+= is not useful if you expect to add values repeatedly to an indexed value. It uses a buffer. See add.at for an unbuffered version. — hpaulj
– hpaulj, Commented Sep 4, 2017 at 0:03

John Zwinck · Accepted Answer · 2017-09-03 23:37:09Z

3

Your code is not safe. Some NumPy ufuncs (like add() which you implicitly use) can release the GIL. Probably the reason you never see any problems in your toy example is that it runs for such a short time and the data size is very small. You may also be avoiding problems by the simple nature of your calculation, but I imagine your real code is more complex.

In short, you can't do what you're doing in multiple threads without locking. And locking will probably defeat the purpose of multiple threads for code like this.

edited Sep 3, 2017 at 23:37

answered Sep 3, 2017 at 23:31

John Zwinck

252k44 gold badges346 silver badges459 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mário Costa Over a year ago

Sure, you're right. I've increased the matrix size to 1000, and 1000 iterations and the issue came up. Thanks!

Collectives™ on Stack Overflow

Is python numpy array operation += thread safe?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related