I want to be able to make a program that emulates what a human at a computer can do. IE: I want to be able to read the visual information of a window (a 2d array of colour would be perfect) and be able to send virtual key syms and mouse movements to the window. Is this at all possible, and if so how is it done?

4 Years
Discussion Span
Last Post by Labdabeta

I looked through that, and it seemed to have what I was looking for in terms of reading the info of the window (I can get its coordinates on the screen then use getPixel on the area?) but it had nothing on how to send fake input to a window. Perhaps the broken link contained the info?


That is great for keyboard input... but what about mouse... I have clarified in my mind what I want a little more now. I want the following self explanatory functions:

WINDOW_HANDLE_TYPE getWindowBasedOnTitle(const char *title);
void sendKeySym(char key, WINDOW_HANDLE_TYPE wnd);
void setMouseXY(int x, int y, WINDOW_HANDLE_TYPE wnd);
void simClick(bool left, WINDOW_HANDLE_TYPE wnd);

You may also want to have a look at AutoIt
It might save you the trouble of re-inventing the wheel if you dont need the gritty details.


Ok, I read the links and I understand how to identify a window and how to copy the output of its device context. What I still do not understand is how to send virtual keysyms/mouse clicks/mouse movements?

This question has already been answered. Start a new discussion instead.
Have something to contribute to this discussion? Please be thoughtful, detailed and courteous, and be sure to adhere to our posting rules.