Python - multiprocessing pool with functions with 2 inputs arrays

Question

I m trying to use multiprocess in order to decrease the time of calculation of functions whose depend of 2D arrays whose shape is 2000x2000. I have 2 inputs arrays for the function but with p.map it doesnt work...(with one it s ok). How can i do to enable that?

from multiprocessing import Pool
from numpy import *
import time
tic=time.clock()

Y=(arange(2000.))
X=(arange(2000.))
(xx,yy)=meshgrid(X,Y)


r = sqrt((xx)**2 + (yy)**2)

theta = (arctan2((yy),(xx)))

def f(theta,r):
  return 1.*r**(-3/2.)*cos(-3/2.*theta)

p = Pool(4)
print p.map(f, theta,r)
toc=time.clock()

print 'Temps=', toc-tic

and i get an error : "The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()"

what version of python are you using? the answer can depend on it. — Julien Spronck
– Julien Spronck, Commented Apr 28, 2015 at 15:23
Also, map is going to iterate over every item in the iterable you pass to it, and call f on each individual element. Is that actually what you want here? It looks like you want to pass both arrays in their entirety to the child process, and have f work on the whole array. If that's the case, multiprocessing.Pool.map won't help improve performance at all. — dano
– dano, Commented Apr 28, 2015 at 16:02
I actually don't think that multiprocessing will help at all for what you need to do with the arrays. numpy already does a good job with that. — Julien Spronck
– Julien Spronck, Commented Apr 28, 2015 at 16:03

paulo.filip3 · Accepted Answer · 2015-04-28 15:55:47Z

2

A way to do solve that is to zip the input arrays

def f(values):
    return 1.*values[1]**(-3/2.)*cos(-3/2.*values[0])

p = Pool(4)
print p.map(f, zip(theta, r))

answered Apr 28, 2015 at 15:55

paulo.filip3

3,3071 gold badge25 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

user3601754 Over a year ago

Thanks it works, just one little problem i get a list of array... : "array([ -2.23942628e-05, -2.23605957e-05, -2.23268448e-05, ..., 7.27439741e-06, 7.27185262e-06, 7.26930745e-06]), array..." How can i get an array with same shape as r or theta?

paulo.filip3 Over a year ago

Oh, I see. You want the function to take the whole array instead of element by element. Then multiprocessing probably won't speed up much at all. In that case do print p.map(f, [[theta, r]])

paulo.filip3 Over a year ago

In that case you could also have done. from functools import partial; partial_f = partial(f, theta); res = p.map(partial_f, [r])

user3601754 Over a year ago

thanks for your help! :) i will try that and i tell you if i won time! ;)

Collectives™ on Stack Overflow

Python - multiprocessing pool with functions with 2 inputs arrays

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related