Python: passing Pool objects to a function in different process

Question

What am I doing wrong in the following code?

import multiprocessing as mp
from multiprocessing import Process
import numpy as np

def fun(X):
    return X;

def funct(p,i):
    print 'Hey'
    res = [p.map(fun,range(2))];
    return res;


if __name__ == '__main__':
    pool = mp.Pool(2);
    output = mp.Queue();
    proc = [mp.Process(target = funct,args=(pool,i)) for i in range(2)]
    for p in proc:
        p.start()
    for p in proc:
        p.join()
    results = [output.get() for p in proc]
    print results

The program doesn't run after launching the process.

dano · Accepted Answer · 2015-06-18 17:43:49Z

5

multiprocessing.Pool doesn't support being passed to child processes, as noted in the multiprocessing documentation:

Note that the methods of a pool should only ever be used by the process which created it.

So there actually is no way to do what you're trying. I'm assuming your code above is just an example to demonstrate the problem, so I won't comment on that directly, but you'll need to refactor your actual code to only call pool methods from the parent process (or possibly create the pool in the child itself, but that's probably not a good solution).

answered Jun 18, 2015 at 17:43

dano

95.5k21 gold badges234 silver badges231 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Radio Controlled Over a year ago

However, "AssertionError: daemonic processes are not allowed to have children". You cannot create a Pool in pooled function, probably to prevent exponential CPU load growth. So I wanted to make a global pool object and draw from that pool in at all times. Doesn't seem to work either. Great...

Collectives™ on Stack Overflow

Python: passing Pool objects to a function in different process

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related