Python share values

Question

In my project I have multiple flags like this:

file_a = False
file_b = False
file_c = False

I'm trying to run two processes: one (call it A from now) handles incoming messages on message queue, second (call it B from now) handles some data processing. B operates on boolean flags, and A sets its values:

def a():
    while True:
        ...
        ...
        file_a = True
        ...

def b():
    while True:
        ...
        if file_a:
            process(file_a)
            ...

a_proc = Process(target=a)
b_proc = Process(target=b)
a_proc.start()
b.proc.start()

But, the values don't seem to change. I read that I should use threading, and it seems to work, but my guideline is to use multiprocessing instead of threads.

akalikin · Accepted Answer · 2015-09-28 11:46:44Z

If you need to exchange data between processes with multiprocessing module, you can directly share memory:

`multiprocessing.Value`

Value is a wrapper around a ctypes object, which has an underlying value attribute representing the actual object in memory. All Value does is ensure that only a single process or thread may read or write this value attribute simultaneously.

from multiprocessing import Value
file_a = Value('i', 0)
file_b = Value('i', 0)
file_c = Value('i', 1)

This will create shared integer values for your file flags. Since it's python, the conversion between integer values and boolean values is simple:

>>> Value('i', True)
<Synchronized wrapper for c_int(1)>
>>> Value('i', False)
<Synchronized wrapper for c_int(0)>
>>> bool(Value('i', False).value)
False
>>> bool(Value('i', 50).value)
True

Preference of a taste, but maybe a better option, you can use c_bool from ctypes:

from multiprocessing import Value
from ctypes import c_bool
file_a = Value(c_bool, False)
file_n = Value(c_bool, False)
file_c = Value(c_bool, True)

>>> Value(c_bool, False)
<Synchronized wrapper for c_bool(False)>
>>> Value(c_bool, 5)
<Synchronized wrapper for c_bool(True)>

`multiprocessing.Manager` dictionary:

To collect multiple boolean flags, you could use dictionary, but it needs to be shared between processes, so Manager() comes in handy.

from multiprocessing import Manager
manager = Manager()
flags = manager.dict({'file_a' : False, 'file_b' : False, 'file_c' : True})

>>> flags
<DictProxy object, typeid 'dict' at 0x7f70822f06d0>
>>> flags['file_a']
False
>>> dict(flags)
{'file_a': False, 'file_c': True, 'file_b': False}

And finally collecting it all together:

I will go with Manager approach, simply because it will make the code cleaner:

from multiprocessing import Process, Manager
manager = Manager()

    def a():
        while True:
            ...
            ...
            flags['file_a'] = True
            ...

    def b():
        while True:
            ...
            if flags['file_a']:
                process(file_a)
                ...

    if __name__ == '__main__':
        flags = manager.dict({'file_a' : False, 'file_b' : False, 'file_c' : True})
        a_proc = Process(target=a)
        b_proc = Process(target=b)
        a_proc.start()
        b.proc.start()

akalikin · Accepted Answer · 2015-09-28 12:49:11Z

0

Your file_a, file_b, file_c are being loaded into each process separately. You need to use Value from multiprocessing

edited Sep 28, 2015 at 12:49

akalikin

1,13713 silver badges35 bronze badges

answered Sep 28, 2015 at 12:46

Josh Wilters

14 bronze badges

4 Comments

akalikin Over a year ago

Your answer could really use some explanations and examples

Josh Wilters Over a year ago

@akalikin im new here. will add comment

Josh Wilters Over a year ago

@akalikin sorry i cant comment yet

Schorschi Over a year ago

Question... can't you pass the (manager) managed variable (dict) to each process? Process(target=a,args(flags,)) and Process(target=b,args(flags,)), thus def a(flags) and def b(flags). The flags variable not passed by reference if this is done? Now that the managed variable is 'shared' seems the object reference should work?

Collectives™ on Stack Overflow

Python share values

2 Answers 2

`multiprocessing.Value`

`multiprocessing.Manager` dictionary:

And finally collecting it all together:

1 Comment

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

multiprocessing.Value

multiprocessing.Manager dictionary:

And finally collecting it all together:

1 Comment

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related

`multiprocessing.Value`

`multiprocessing.Manager` dictionary: