Please, allow me to use taskotron for my packages without the need of knowing how taskotron works, or what it is. My basic idea (I'm open to other suggestions) follows:

Run the VM with minimal set of packages.

Install the package that's being tested (have subfolders for subpackages if needed with similar logic).

Execute every test_* file in the rpm-tests/package repository in some predictable order (sorted with sort). Each script is a test and the filename is the test name.

That file could be Bash, Python, etc (but basically anything that can be executed (I see a problem with dependencies, I will address it later)).

Run each file and add artifacts with stdout and stderr. Exit code determines the outcome:

• 0 is PASSED
• 1 is FAILED
• any other exit code is ERROR

If you need other outcomes, such as NEEDSINSPECTION, assign some reasonable exit code, like 3.

Convert the exist code and stdout, stderr to the result format you expect.

This is exactly all I would need for all of my packages, of course others might need more complicated stuff, so if the runtask.yml is there, run it as it is being run currently.

Note that this requires very limited knowledge from the tests author:

• how exit codes are mapped to outcomes (very intuitive)
• that tests scripts should be called test_, so you could have other files (very intuitive)
• that stdout and stderr will be captured and useful info can be put there (very intuitive)

The only problem is that if you need some dependencies, for that part I have some ideas:

• analyze the tests_* files and install the required interpreter (from shebang) automatically
• allow a special file (for example init* or setup*) that would run first and you could put dnf install whatever or other stuff in there (download upstream testuite maybe...)

Thanks.