Getting local mouse position within window with python

Question

I want to automate a few of the menial tasks that I do every single day both at my rented and home office.

I was looking into how I'd go about getting the local cursor/mouse position within a window rather than the global position on the screen, but as far as I've read online it seems like that's no easy task.

I know that I could use image recognition with PyAutoGui, PyWinAuto, Lackey or the java/python hybrid SikuliX. This of course works, but it seems like small variances in screen sizes and window sizes effectively renders most of the previously taken images unusable and you end up having to re-take most if not all.

The relationship between elements within the window stays the same. (They only scale up/down based on window size)

Is something like this possible? How much work am I looking at here? I appreciate any nudges in the right direction and I'm open to using other modules if available.

Please and thank you.

running.t · Accepted Answer · 2017-07-27 12:44:29Z

1

On linux you can use xdotool utility. You can isntall it on most of well-known distributions. E.g on ubuntu using:

sudo apt-get update
sudo apt-get install xdotool

According to manual you can use xdotool mousemove command with coordinates relative to specified window.

There is also a python binding for this tool - python-libxdo. To install it in python use:

pip install python-libxdo

On windows there is AutoIt scripting utility. And pyautoit python binding. But I have never tested it so I can't tell if it can help you with your automation tasks.

edited Jul 27, 2017 at 12:44

answered Jul 27, 2017 at 11:16

running.t

5,7395 gold badges37 silver badges54 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Dixos Over a year ago

Oh, that's brilliant @running.t! Do you know if there are any Windows equivalents?

running.t Over a year ago

@Dixos: Se my eddited answer.

Vasily Ryabov Over a year ago

pywinauto is also for Windows, and it's not image based, but uses text properties that is more reliable and maintainable. Also AutoIt doesn't support MS UIAutomation technology that is a must for many modern apps.

Vasily Ryabov Over a year ago

To estimate how much work is needed, just read the Getting Started Guide for pywinauto.

Collectives™ on Stack Overflow

Getting local mouse position within window with python

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related