Is there any way to replace a for loop with something more efficient in python

Question

My code below checks surrounding pixels to my object's pixel in python.

self.surr = [None, None, None, None, None, None, None, None]

for i in range(9):
#for x in range(-1, 2):
    #for y in range(-1, 2):
        if i != 5:
            x = i % 3 - 2
            y = int((i % 3) / 3) - 1

        if x == 0 and y == 0:
            pass
        else:
            PAI = allPixels[(self.x + x) % width][(self.y + y) % height] if allPixels[(self.x + x) % width][(self.y + y) % height] != None else None

        self.surr[(y * 3) + x] = (PAI)

return self.surr

This returns a list of length 8 that holds either a Pixel object or None. allPixels is a 2D array that also holds wither a Pixel object or None. I've tried then nested loops that are commented out but they run just a bit slower that the method I'm currently using. This however, is still too slow as if there are 3000 pixels on the screen, which is the lower bounds of the total pixels there will be on the screen in the end, do the maths and you have a LOT going on every frame.

How can I make this run faster using maybe NumPy or some other method?

If you want to see the whole code, it can be found here: https://pastebin.com/EuutUVjS

Thanks for any help you can give me!

As a side note: 3000 pixels may sound like a lot to you, but it's really nothing to a computer. Even if it takes multiple microseconds for each loop iteration instead of nanoseconds, it's still fast enough to do at 60fps without using most of your frame time. — abarnert
– abarnert, Commented May 20, 2018 at 20:18

abarnert · Accepted Answer · 2018-05-20 20:17:10Z

3

What you're doing inherently requires looping—but if you can move that looping into numpy, it'll often get 5-20x faster.

In your case, what you're trying to do is to compare each pixel to its neighbors. How can you do that as an array-wide operation? Simple: compare the array to the same array shifted by 1.

Here's a simpler example:

>>> a = np.array([1,2,4,8,16])
>>> for i in range(1, len(a)):
...     print(a[i] - a[i-1], end=' ')
1 2 4 8
>>> print(a[1:] - a[:-1])
[1 2 4 8]

So, for a 2D array, it's just:

north = a[:-1]
ne = a[:-1,1:]
east = a[:,1:]
se = a[1:,1:]
south = a[1:]
sw = a[1:,:-1]
west = a[:,:-1]
nw = a[:-1,:-1]

Note that this isn't wasting a whole lot of time or memory building 8 extra arrays; it's just creating 8 views over the same memory.

See this answer on compsci for an example of using these shifted arrays for a Conway Game of Life simulation.

If you want to handle the borders differently, you may need to "zero-extend" the array, but that's the only complexity you're likely to run into.

However, there's a limit to how much benefit you can get out of numpy if you're storing Python objects in it. Normally, you want to store arrays of numbers.

I don't know what's in your Pixel objects, but let's pretend they're just color values, as three floats. In that case, you can use a 2D array with a structured dtype of three floats, or just a 3D array (row by column by r-g-b), either way using NaN values in place of None.

If you do that, array-wide operations can operate at near-machine-native speeds, including using SIMD operations for data-parallelism. If you don't, only the looping happens at native speeds; the arithmetic within the loop is still just as slow as in non-Numpy Python.

edited May 20, 2018 at 20:17

answered May 20, 2018 at 19:03

abarnert

368k54 gold badges626 silver badges691 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Thomas Ayling Over a year ago

Okay, thanks but how would i do something like this for a 2D array?

abarnert Over a year ago

@ThomasAyling Updated the answer.

Thomas Ayling Over a year ago

In the example that you gave, is the array that stores all the pixels noe dimentional?

abarnert Over a year ago

@ThomasAyling The first example shows how to get the pixels to the left of each pixel in a 1D array. The second example shows how to get the pixels in each of the 8 directions from each pixel in a 2D array, and the Game of Life link is similarly using a 2D array.

Thomas Ayling Over a year ago

i tried implementing this emthod but i get this error: TypeError: list indices must be integers or slices, not tuple

|

Collectives™ on Stack Overflow

Is there any way to replace a for loop with something more efficient in python

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related