Global variable increment and python multiprocessing

Question

Hi I have following multiprocess code and I want to ensure that global variable ctr should get updated by all end nodes or leaves of decision tree. But it not happening.

    ctr=0

            def update(l,n):
               global ctr
               l.acquire()
               ctr+=n
               l.release()

    def func(x,i):
        p2=[]
        if i > 100:
           lock=Lock()
                        update(lock,len(rl))
        # create list a1
        # create list a2
        i=len(a1)
        for a in a1:
            for b in a2:
                if x > (a+b):
                    proc=Process(target=func,args=(a+b,i,))
                    p2.append(proc)

        for p in p2:
            p.start()
            p.join()

Blubber · Accepted Answer · 2013-06-13 06:37:29Z

1

When you start new processes with Python (or any other language for that matter) the same memory segments are used by all processes for read access. However, as soon as you start writing to some memory (e.g. update a variable) that variable get's copied into the new processes' own memory segment.

In other words, you can't update a global variable and expect multiple processes' to see the same value, you need some kind of shared memory for that.

Check out the documentation on Multiprocessing

Keep in mind though, that updating a global variable involves some amount of locking, basically forcing your multiple processes to behave in a serial manner. This can negatively impact performance depending on your use-case. You should try to avoid using global state whenever you can (this is more general advice, but in multiprocessing cases it's even worse to have global state.)

answered Jun 13, 2013 at 6:37

Blubber

2,2542 gold badges17 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Pratibha Over a year ago

I have edited the code with lock. but it is still not working.

Blubber Over a year ago

What I was saying is that you need to use some kind of shared memory mechanism, like a multiprocessing.Value, this is going to do the locking for you, so you don't have to. Just be aware that locking comes with a performance penalty.

Collectives™ on Stack Overflow

Global variable increment and python multiprocessing

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related