Hardcoding 'x86_64' into the first table effectively means that any time this test passes when run on any arch, it'll report a result to the 'x86_64' column. I don't like that.

I think I'd actually prefer if the title of the column on the wiki page was just 'Result', indicating it can be tested on any arch, and we use 'Result' in this code too.

Also, if we go with my suggestion in https://pagure.io/fedora-qa/os-autoinst-distri-fedora/pull-request/82 to have each test module run even if a previous one fails, then we can use the mechanism for reporting passes by passed test module not by overall test result. So if one of the tests fails but the other three pass, we will report those three passes.

Hardcoding 'x86_64' into the first table effectively means that any time this test passes when run on any arch, it'll report a result to the 'x86_64' column. I don't like that.
I think I'd actually prefer if the title of the column on the wiki page was just 'Result', indicating it can be tested on any arch, and we use 'Result' in this code too.

I changed that to Results, both in the code and in the wiki.

As far as the other note is concerned, I am not sure that I understand that correctly. So, in the code, I have written a test suite that consists of single testcases that will run in that order. Will they only be reported if the whole test suite passes? Or not? And if so, how do I force it to update after each testcase passes, no matter what the overall result for the whole suite will be?

Please make it 'Result', not 'Results'. There's absolutely no need for it to be inconsistent with literally every other case of this in the entire set of result pages...

For the other thing...what this means:

"modularity-tests": [
    "QA:Testcase_Modularity_module_list",
    "QA:Testcase_Modularity_enable-disable_module",
    "QA:Testcase_Modularity_install_module",
    "QA:Testcase_Modularity_update_norepos",
],

is "if the test suite called modularity-tests passes, report passes for all these wiki tests". It's "all or nothing": if the overall test suite fails, we report nothing. (We never report failures, note, we only report passes).

From there...I am actually gonna have to sit down and think a bit, because I've just realized I wasn't quite remembering how all this works correctly. The way I asked you to set up the test now, it will never show its overall result as failed, which is kinda funky. I think we'll have to change it again, but it'll actually require tweaking the result reporting code a bit, and it's not super simple. Maybe we can chat about it tomorrow or Monday.

Sorry for the confusion :)

I will try to take a look, if there is an option in the OpenQA docs that
would make it for us.

This isn't openQA stuff, it's our stuff. It's mostly my fault. ;) All this wiki reporting stuff is crazy hacks on our part, it's nothing at all to do with upstream.

I think I'm gonna say for now, let's just go with a simple but not-quite-optimal implementation, and we can try to get fancy in future. So for this PR, please leave the wiki report bits as they are for now, and just change 'Results' to 'Result'. Thanks!

So I decided the tests fit better into Base than Installation (they do not, after all, have anything to do with installing the distro...) and moved them there. I tweaked the PR to reflect this change, and also corrected modularity-tests to modularity_tests, then merged it. Thanks!