Parallelize operations for each cell in a numpy array

Question

I am trying to figure out which is the best way to parallelize the execution of a single operation for each cell in a 2D numpy array.

In particular, I need to do a bitwise operation for each cell in the array.

This is what I do using a single for cycle:

for x in range(M):
    for y in range(N):
        v[x][y] = (v[x][y] >> 7) & 255

I found a way to do the same above using the vectorize method:

def f(x):
    return (x >> 7) & 255
f = numpy.vectorize(f)

v = f(v)

However, using vectorize doesn't seem to improve performance.

I read about numexpr in this answer on StackOverflow, where also Theano and Cython are cited. Theano in particular seems a good solution, but I cannot find examples that fit my case.

So my question is: which is the best way to improve the above code, using parallelization and possibly GPU computation? May someone post some sample code to do this?

You could look at multiprocessing.Pool, by defining a function for your bitwise operation and sending the list of all your cells. It will then use all your processors to evaluate the result, but you wiil need to reconstruct the array, wich can make you lose time. How big is your array? how long does your calculation take? — CoMartel
– CoMartel, Commented May 18, 2015 at 11:58
With such a simple element-wise transform, the overall performance may be memory IO bound. I.e., moving data to/from a GPU may make the operation slower overall. — rickhg12hs
– rickhg12hs, Commented May 18, 2015 at 12:17
Correct the indentation and variable names in the 1st code piece. — hpaulj
– hpaulj, Commented May 18, 2015 at 16:05
The normal way to index an 2d array is v[x,y]. Using [x][y] may be equivalent here, but if v[x] produces a copy rather than a view, it does not work. — hpaulj
– hpaulj, Commented May 18, 2015 at 16:58

plonser · Accepted Answer · 2015-05-18 12:19:14Z

4

I am not familiar with bitwise operations but this here gives me the same result as your code and is vectorized.

import numpy as np

# make sure it is a numpy.array
v = np.array(v)

# vectorized computation
N = (v >> 7) & 255

edited May 18, 2015 at 12:19

answered May 18, 2015 at 12:12

plonser

3,3632 gold badges21 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Parallelize operations for each cell in a numpy array

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related