8

I'd trying to write a script that will detect an RGB value on the screen then click the x,y values. I know how to perform the click but I need to process the image a lot faster than my code below currently does. Is this possible with Python?

So far I'm reading a row at a time, when x = 1920 I go onto the second row but it takes about 10 seconds to do one row. By that time the person on screen would have moved into a completely different spot and I have only done one row!

Can I speed this code up OR is there a better way to achieve what I want? If it is not possible in Python I am open to C++ options :)

import Image

x = 0
y = 0

im = Image.open("C:\Users\sean\Desktop\screen.jpg")
pix = im.load()
print im.size #get width and height of the image for iterating over
while x < 1920:
    print pix[x,y] #get RGBA value of the pixel of an image
    print "x is:" +str(x)
    x = x + 1
    print "y is: " +str(y)
    if x == 1920:
        x = 0
        y = y + 1
3
  • 1
    You could try opencv python wrapper? Commented Apr 11, 2015 at 17:44
  • Link: OpenCV Commented Apr 11, 2015 at 17:57
  • What do you mean by "click the x,y"? Commented Apr 11, 2015 at 18:15

6 Answers 6

8

Generally, you want to avoid per-pixel loops in Python. They will always be slow. To get somewhat fast image processing, you need to get used to working with matrices instead of individual pixels. You have basically two options, you can either use NumPy or OpenCV, or a combination of the two. NumPy is a generic mathemtical matrix/array library, but you can do many image-related things with it. If you need something more specific, OpenCV supports many common operations on images.

Sign up to request clarification or add additional context in comments.

Comments

4

Thanks for the responses, below is the code I used, I didn't change my original. Turns out it is fast enough but printing is a very costly operation :) It finds the x and y coords of the RGB value in less than a second

#check for certain RGB in image

##need to screen grab

import Image, sys

x = 0
y = 0

im = Image.open('C:\\Users\\sean\\Desktop\\test.jpg')
pix = im.load()
print im.size #get width and height of the image for iterating over
while x < 1914:
    value = pix[x,y] #get RGBA value of the pixel of an image
    if value == (33, 179, 80):
        #call left_click(x,y)
        print x,y
    x = x + 1
    if x == 1914:
        x = 0
        y = y + 1
print "Finished"
sys.exit()

2 Comments

printing is only a very costly operation in the default IDLE environment, it's a lot faster in other tools like pycharm and fastest without IDE
@walter Printing is always a lot slower than just comparing a pixel value.
3

Image.getpixel is considered very slow. Instead, consider using Image.getdata. That will give you a sequence with data for all the pixels through which you can iterate.

Something like this:

import Image
import math

x = 0
y = 0

im = Image.open("IMG_2977.JPG")
(width, height) = im.size
print width
print height

pix = im.getdata()

i = 0

for pixel in pix:
    print pixel
    x = i % ( width )
    y = math.trunc( i / width)
    print "x is: {}".format(x)
    print "y is: {}".format(y)
    i += 1

Without printing (just storing pixel in a variable) that runs in 2 seconds of user time (0.02 seconds of processor time) on my MacBook Pro.

1 Comment

And if you did that without the division operation it would be so, so much faster. Division is expensive, try to avoid it inside a loop.
3

You may want to do one of two things here.

1. Get a single pixel from the image

In this case, you don't need to iterate through the entire file. Just use im.getpixel. @Daniel makes a valid point that this is slow in a loop, but if you just want a single pixel, this is very efficient.

from PIL import Image
im = Image.open('screenshot.png')

im.getpixel((x, y))    # Returns the colour at (x, y)

2. Process multiple pixels from the image

This is best done using NumPy as @Lukáš suggests. If you want to do something like get the average colour of the 10 x 10 grid around the pixel, for example.

You can get the data as a NumPy array using scipy.misc.fromimage

from PIL import Image
from scipy.misc import fromimage

im = Image.open('screenshot.png')
data = fromimage(img)

Let's compare the time it takes to get this data against a for loop.

In [32]: pix = im.load()

In [33]: %timeit fromimage(im)
10 loops, best of 3: 8.24 ms per loops

In [34]: %timeit [pix[x, y] for x in xrange(im.size[0]) for y in xrange(im.size[1])]
1 loops, best of 3: 637 ms per loop

To summarise:

  • scipy.misc.fromimage is the fastest, at ~8ms for a 1920x1080 image
  • Looping through pix[x, y] takes ~640ms, about 80 times slower

Comments

2

There is something called pyautogui and it will find the entire image on the screen within 1-5 seconds usually, which is not too fast but seems better than you current option

Comments

2

You can get the first half of image and second half of image in 2 threads and process these halfs, but for me it speed up only for 15%. Normal speed for me is 2,7 secs at the image of height 375 and width 483. Threads speed it up to 2,3 secs. This is why I'm searching this question.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.