Python: FFT parallel computing

Question

I have a cluster of few PC's with Ubuntu and MPICH server - I use them to parallel computing with C++ and MPI. Now I want to do similar with Python. My question is - is there any easy method to make Fast Fourier Transform on many CPU cores (on many computers)? Example of usage would be nice. Here is method I use on single thread:

import numpy as np
N=1024
tab=np.random.rand(N,N,N) #declare some matrix in 3d
a=np.random.rand(N,N,N) #declare other matrix
tab=np.fft.ifftn(a*np.fft.fftn(tab))

It's nice to have multithreaded solution when we have 2^30 numbers...

Andy · Accepted Answer · 2016-01-05 20:58:12Z

1

MPI4py in connection with pyFFTW. FFTW is highly optimized and works well on multiple threads but the interface is a little different than your average FFT as it requires creating 'plans' outright. However, this was just done to optimize out processing of any static twiddle factors and such. Additionally, there are lots of options - forward/reverse, in/out-of-place, int, double, float, etc... See benchmarks for more info.

answered Jan 5, 2016 at 20:58

Andy

1,73510 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Cheshire Over a year ago

Sooo... how would you code it? I was looking on Stack for an example to work with, I wrote only because I couldn't find it.

Ed Smith · Accepted Answer · 2016-01-06 10:54:16Z

0

Check out this thread which discusses the development of MPI python with pyFFTW. From this it seems like pfft may be a better solution and the minimal example python code is given at https://github.com/rainwoodman/pfft-python/blob/master/example.py.

answered Jan 6, 2016 at 10:54

Ed Smith

13.3k2 gold badges48 silver badges58 bronze badges

Collectives™ on Stack Overflow

Python: FFT parallel computing

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related