Python: Update Local Variable in a Parallel Process from Parent Program

Question

I am relatively new to programming, and what I am asking might be a task that is not possible. What I want to do is start a parallel process, which will continuously run until requested to stop by the user. Once the process has started, I would like to update one of the local variables inside the parallel process from the parent program without stopping the process. For a very simple example, the parallel process I would like to execute would be the following:

import time

def loop(i):
    while 1:
        print i
        i+=1
        time.sleep(1)

Which continuously iterates and updates i. For clarity the parent program will contain:

from multiprocessing import Process
from loop import loop

i = 1
p = Process(target = loop, args = (i))
p.start()

From the parent program, once "loop" has be initiated, I would like to be able to change the input "i" to another number and have loop continue to iterate given the next starting value. If anybody has any ideas on how to do this, it would be greatly appreciated.

mguijarr · Accepted Answer · 2013-10-07 19:50:52Z

3

You can use a queue to pass data between the processes:

from multiprocessing import Process, Queue
from loop import loop

i = 1
q = Queue()
q.put(i)
p = Process(target=loop, args=(q, ))
p.start()

Whenever you want to transmit a new value of i to the other process, just put it in the queue.

Change your loop.py module accordingly:

def loop(q):
  while True:
    i = q.get()
    print i

In your main code, you can put new values for the process:

while True:
  i+=1
  q.put(i)
  time.sleep(1)

answered Oct 7, 2013 at 19:50

mguijarr

8,0306 gold badges50 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

kindall Over a year ago

If you want your i to continue increasing by 1 until a new value is passed, in the consumer you can write i = (i + 1) if q.empty() else q.get(). I'd also use a maxsize=1 on the Queue constructor so putting blocks if the queue already has an item in it.

mguijarr Over a year ago

@kindall: I like this a lot, it was missing in my answer

Tim Peters · Accepted Answer · 2013-10-07 20:48:04Z

Nothing wrong with a queue here, but it's probably more idiomatic to use "shared memory" instead. Less overhead. Here's an example self-contained program:

import time

def loop(i):
    while 1:
        print i.value
        i.value += 1
        time.sleep(1)

if __name__ == "__main__":
    from multiprocessing import Process, Value

    i = Value("i", 1)  # "i" for integer, initial value 1
    p = Process(target=loop, args=(i,))
    p.start()
    for base in range(100, 600, 100):
        time.sleep(2)
        i.value = base

That will probably ;-) display:

But caution: in return for being speedier, this is also more brittle. The kinds of data you can share this way are basically inherited from the C language, and aren't generally as rich as Python data types. In the example above, the type code "i" means we're sharing a signed C int, which is usually a 32-bit signed integer. If, for example, i.value reaches 2147483647, and we add 1 to it, it will suddenly become -2147483648 (yup, negative!). Python ints are unbounded, but C ints aren't.

roippi · Accepted Answer · 2013-10-07 20:02:56Z

0

Use a queue. (python 2 is Queue)

There are a few ways to consume from the queue. The naive implementation (i.e. the one that has a race condition) is simply to do if q.empty(). This is bad practice but wouldn't hurt you since it's not mission critical if you miss a tick.

The better method is to use exceptions for flow control:

q = queue.Queue()
try:
    q.get(False) #non-blocking get()
    #update value of i, etc
except queue.Empty:
    #do stuff here as before

edited Oct 7, 2013 at 20:02

answered Oct 7, 2013 at 19:57

roippi

26k4 gold badges52 silver badges75 bronze badges

Collectives™ on Stack Overflow

Python: Update Local Variable in a Parallel Process from Parent Program

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related