Python multprocessing: Increment values of shared variables across processes

Question

Here's a minimal example of what I want to achieve.

The global INIT_* values should be incremented by each process and made available for another function (print_locked in this example):

import multiprocessing as mp

INIT_NODE = mp.Value('i', 1000)
INIT_WAY = mp.Value('i', 1000)
INIT_REL = mp.Value('i', 1000)


def print_locked(a, b, c):
    print(a.value, b.value, c.value)


def process_results(a):
    with mp.Lock():
        INIT_NODE.value += 20000000
        INIT_WAY.value += 10000000
        INIT_REL.value += 1000000

    print_locked(INIT_NODE, INIT_WAY, INIT_REL)


def main():
    """Entry point for the program."""

    a = range(5)
    pool = mp.Pool(4)
    pool.map(process_results, a)


if __name__ == '__main__':
    main()

However, it seems that each process increments its own version of the initial values, which is exactly what I want to avoid.

Example output:

20001000 20001000 2001000
20001000 20001000 2001000
40001000 30001000 3001000
60001000 40001000 4001000
80001000 50001000 5001000

Mitiku · Accepted Answer · 2019-09-08 08:21:16Z

1

mp.Lock() create lock object but it does not lock the block of code. To lock critical region of code you should first call acquire method of mp.Lock object and call release method of the object after the critical region.



INIT_NODE = mp.Value('i', 1000)
INIT_WAY = mp.Value('i', 1000)
INIT_REL = mp.Value('i', 1000)
lock = mp.Lock()


def process_results(a):
    lock.acquire()

    INIT_NODE.value += 20000000
    INIT_WAY.value += 10000000
    INIT_REL.value += 1000000
    print_locked(INIT_NODE, INIT_WAY, INIT_REL)
    lock.release()

Now the output of this program is the same for all run

20001000 10001000 1001000
40001000 20001000 2001000
60001000 30001000 3001000
80001000 40001000 4001000
100001000 50001000 5001000

answered Sep 8, 2019 at 8:21

Mitiku

5,4223 gold badges21 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

nilsnolde Over a year ago

Damn sorry, had to undo the 'accept answer'. I ran it a few times and there's still collisions: 80001000 40001000 4001000 was produced twice from separate processes. Doesn't seem to be process-safe yet.. And I have no idea why. This shouldn't really happen right?

nilsnolde Over a year ago

Oh no forget it. I mixed 2 lines of code, releasing the lock BEFORE calling print_locked(). Seems to be all good now. Sorry for the confusion!

Collectives™ on Stack Overflow

Python multprocessing: Increment values of shared variables across processes

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related