Inefficient numpy code

Question

In Python I'm using the numpy package to do some math with matrices. In the code below I'm trying to calculate a new matrix from my orignal. xFactors and yFactors are both 3x3 matrices.

size = self.matrix.shape

for x in range(1, size[0] - 1):
    for y in range(1, size[1] - 1):
        subMatrix = self.matrix[x-1:x+2, y-1:y+2]

        newX = (xFactors * subMatrix).sum()
        newY = (yFactors * subMatrix).sum()

        self.newMatrix[x-1][y-1] = newX + newY

My problem is that this code is very inefficient. I tested te code with a 500x500 matrix and it takes up to two seconds. Do you have any ideas how I can optimize this code?

Looks like you could use 2D convolution there with xFactors and yFactors as kernels separately on matrix as input data. So, look into Scipy's convolve2d. — Divakar
– Divakar, Commented Nov 13, 2016 at 11:41
@Divakar only If xFactors and self.matrix are both numpy.array and not numpy.matrix (in other words if OP is using element-wise multiplication and not matrix multiplication in calculating newX and newY — chthonicdaemon
– chthonicdaemon, Commented Nov 13, 2016 at 11:46
Yes both are of the type numpy.array. @Divakar can you explain this a bit more? Do you mean the scipy method? — Cilenco
– Cilenco, Commented Nov 13, 2016 at 11:51

chthonicdaemon · Accepted Answer · 2016-11-13 12:09:24Z

2

If xFactors and self.matrix are both numpy.array and not numpy.matrix (in other words if you are using element-wise multiplication and not matrix multiplication in calculating newX and newY), then this should do the same thing a lot faster:

from scipy.signal import convolve2d

self.newMatrix = convolve2d(self.matrix, xFactors + yFactors, mode='valid')

In the original code, it was not clearly stated that xFactors and yFactors were square. If they weren't one would need to make them square by repeating them as needed if the above addition doesn't broadcast correctly.

edited Nov 13, 2016 at 12:09

answered Nov 13, 2016 at 11:52

chthonicdaemon

19.9k2 gold badges55 silver badges70 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

Eric Over a year ago

For this to work, I think xFactors and yFactors need to be square, whereas in the original code, they would broadcast as needed. So really, this should start with a xFactors, yFactors = np.broadcast_arrays(xFactors, yFactors) to be equivalent

chthonicdaemon Over a year ago

@Eric you are right, but the OP didn't mention that. For completeness, I'll add a note about that to my answer.

Eric Over a year ago

Actually, better yet - add before convolving, since convolution distributes over addition - should be a fair bit faster, and does the broadcasting for free

chthonicdaemon Over a year ago

If xFactors and yFactors are both (3,1) or both (1,3) the convolution will not work the same way as in the original code and the addition will not broadcast to (3,3).

chthonicdaemon Over a year ago

No,

self.newMatrix = abs(convolve2d(self.matrix, xFactors, mode='valid')) + abs(convolve2d(self.matrix, yFactors, mode='valid'))

.

|

Collectives™ on Stack Overflow

Inefficient numpy code

1 Answer 1

10 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Related