Variable does not change in Python multiprocessing

Question

I'm trying to use a dictionary in my multiprocessing code. The problem is when I changed the value of one key in the process, the changes just affect only that process. Apparently, variable arguments of the process are a copy of that variable. I tried to use dict method but it does not work too. What is my mistake?

import time 
from multiprocessing import Process, Manager     

def f1(list1, set1):
    list1.append(3)
    set1['func']='f1' 
    print('f1 dic',set1) 
    print('f1 list',list1) 
    while True:
        time.sleep(3)
        print('f1 dic',set1) 
        print('f1 list',list1) 

def f2(list1): 
    list1.append(4)
    settings_dic['func']='f2'
    print('f2 dic',settings_dic) 
    print('f2 list',list1) 
    while True:
        time.sleep(3)
        print('f2 dic',settings_dic) 
        print('f2 list',list1) 

if __name__ == '__main__':     
    # list1 = Manager().dict()    
    settings_dic = Manager().dict()  
    Global = Manager().Namespace()
    list1 = [1,2]
    settings_dic = { 
        'mode' : 'master', 
        'logger' : 'True',
        'func' : ''
    }       
    p_video_get = Process(target=f1, args=(list1,settings_dic,))
    p_video_get.daemon = True
    p_video_get.start() 
    p_packetTrasnmission = Process(target=f2, args=(list1,))
    p_packetTrasnmission.daemon = True
    p_packetTrasnmission.start()  
    
    list1.append(5)
    settings_dic['func'] = 'main'
    print('main dic',settings_dic)
    print('main list',list1)  

    while True:
        time.sleep(3)
        print('main dic',settings_dic)
        print('main list',list1)

Because your settings_dic is a regular dictionary not a — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Dec 22, 2020 at 8:55
You tried to use a Manager().dict() but you instead re-assign a regular dict to the same variable — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Dec 22, 2020 at 9:02

saquintes · Accepted Answer · 2020-12-22 09:27:11Z

5

UPDATE:

As pointed out by juanpa.arrivillaga, the problem with the code above is that the Manager.dict() is re-assigned as a normal dict meaning it's lost the reference to the Manager.dict() and isn't changing/passing that one around. The rest is my original response where I thought the OP was just trying to pass a dictionary around.

ORIGINAL:

When you start up a separate processes, that process has their own address space. Unless you set something up, they don't share any memory (and by extension, data). Each process gets a copy of the arguments you pass in, and their process's memory manage their own copy of it (this is done by pickling the object and sending it to the new process). Once it's copied, they do not sync up. To have one process change the data in another you have a couple of options.

You either need to setup some IPC via a shared comms or shared object. A Queue is an easy way to post updates to a data structure that other processes can then pop off and update their own copies.

Or if you are on 3.8+, you can setup a shared memory. I've not done this, and you'd still need to write the code to monitor and decode the changes. But it's another way to have one process communicate with another.

edited Dec 22, 2020 at 9:27

answered Dec 22, 2020 at 8:55

saquintes

1,0895 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

saquintes Over a year ago

@juanpa.arrivillaga: you are right. Missed that. Just saw the last assignment. Not sure why it was accepted it as it didn't address that piece. I'm happy to delete if the OP wants to un-accept. Or if you want to post the correct answer I'll contribute an up-vote.

juanpa.arrivillaga Over a year ago

just add to the answer

Collectives™ on Stack Overflow

Variable does not change in Python multiprocessing

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related