README.md

AutoCoconut, a creative tool for OpenQA

AutoCoconut is a tool that runs in the background and records keyboard and mouse events and then turns them into a list of events (or a workflow) and creates screenshots for the important events.

Such information can be helpful when creating test cases for GUI testing frameworks, such as OpenQA, or for describing bug reproducers, etc.

Development

Currently, the development has reached Phase 1.

Which means that the script is able:

  • record various events, mouse buttons and actions (click, double click, drag, vertical scroll), keyboard events (press and release)
  • identify various types of keys (modifiers, special keys, character keys, etc.)
  • find pre-defined patterns in single events and interpret them
  • take screenshots to illustrate the workflow (or create needles for OpenQA)
  • produce various output - raw file, json file, or a workflow description in markdown, html, or perl

Documentation

So far, AutoCoconut works as a CLI application. It can be started using the autococonut.py script. The script monitors the mouse and keyboard and records their events, such as clicks, keypresses, etc. and makes a list of these single events. Later, some of the single events are merged into super events by the Interpreter part to make it more understandable. For instance, when some presses a sequence of keys, such as "h", "e", "l", "l", and "o", the Interpreter recognizes it correctly as typing "hello" instead of pressing single keys.

It also takes the pictures of screens to capture either the click areas (for mouse events) or the result of the action (keyboard events) or both. The very first screen is taken right upon the start of the script and the last one is taken upon the termination of the script. When an extra screenshot is needed, a dedicated key can be used to take it whenever it is required. Using the print key will not be recorded in the list of events.

The list of events can be obtained as a raw list of all single events, as an interpreted json file, or as workflow descriptions of various formats.

Running the script without any arguments will take screenshots with one-second delay and produce the markdown formatted workflow description directly on the command line. The script can be terminated using the F10 key by default. An extra screenshot can be taken using the F9 key.

CLI arguments and their explanation

The script also accepts various arguments to control the flow and the output:

-s, --stopkey: When recording the keyboard events, we need some key that stops the recording and terminates the script gracefully. Normally, this could be achieved by hitting Ctrl-C on the terminal, but since Ctrl-C is an important shortcut in GUI, too, we want to capture that combination correctly without stopping the recording activity. Therefore, it is blocked and it cannot terminate the execution of the script. You can specify another key that will terminate it. By default, the stop key is the F10 key.

-t, --printkey: Currently, the script will take a screenshot after each super event. Sometimes, especially when working with applications that have slower response, this might not be enough. Therefore, an extra screenshot can be taken anytime using the dedicated print key. This redefines the default F9 print key.

-p, --pause: Defines a time (in seconds) that the script waits before taking the post action screenshot. The time can be given in decimal numbers, too. The shorter the wait period is, the more quickly the events can follow each other, but with applications with slow response, the post screenshots might not show the correct screen. With longer waiting time, you have to wait longer before you make another step. The default is 1 second.

-o, --output: You can choose one of several outputs. The raw output returns a json file with all single events without interpretation. In this json file, all key presses and releases are recorded separately, including the combinations. The json output provides an interpreted list of super events organized in a json file. The md, html, and openqa outputs produce a list of steps in that chosen format. The openqa format lists the OpenQA test commands that can be used for OpenQA scripts.

-f, --file: If a filename is given, the output will be saved to a file instead being displayed on a command line.

-r, --resolution: Not implemented yet. It will come during the development Phase 2. If the resolution is given, the screen resolution will be changed to your selected resolution first, and then the script will start the recording.