How to write a bot to a toy in a virtual machine?

Question

Python

There is a toy for Windows, it spins in a virtual machine, a virtual machine (virtualbox) on Debian.
In the toy, I want to transfer part of the actions to the bot (while you are sleeping, it is swinging =)), there is no api for the toy, I will not be able to reverse engineer the protocols.
I think the easiest way is to take a picture, find the right places and click there. I know python.
What do you advise? Thanks in advance.

Answer 1

You can do it in Python as well. Take a picture, process it, move the mouse and click - he can do it for himself.

Answer 2

note,
code.tutsplus.com/tutorials/how-to-build-a-python-... can help

Answer 3

The easiest way is www.sikuli.org
It is written in Jython
It helped me a lot with automation at one time.

Answer 4

There was a post Browser robot on Sikuli IDE for the MusicWars game - maybe it will help ...