ping @sumantrom , what is the current state here? Thanks!

Hello Adam,
after some of the reorganization of the QA team, I think I should be the one working on this issue. I have already started to talk with various people, including @sgallagh, to get more info on Modularity in Fedora 29.
What I think is needed now:

• revise the release criteria
• revise the test cases and add some new, if necessary

If you are coming to Flock, we can talk about it, otherwise, I will probably fully devote to this, when we come back from Flock.

In an email communication with Stephen, I got the most important features, we will have to test. For the start, I suggest to pay attention to them, especially to points 1 to 3.

1. Can you get a list of available modules with dnf module list
2. Can you install a package from a default module stream without using the dnf module subcommand (e.g. if nodejs:10 is the default stream for nodejs, dnf install nodejs should work and bring in the version from the 10 stream.
3. Verify that updates applied via both DNF and PackageKit are selected and applied correctly for the enabled module streams.
4. Can you build a module stream and submit it to Bodhi, tracking it through -updates-testing and then to stable?

Perhaps, each of the test cases could cover one item on this list.

I don't think we need to do anything further to the release criteria for now. I would like to focus on getting test cases written and published. I'd like this to happen sooner than "after flock", to be honest, we should be doing it now. Thanks.

Ok, I will start working on it now, or rather as soons as possible because I am now on PTO. I will be back on Friday and I will use the whole day to work on this to fix it before Flock then.

Hello,
I have updated the three test cases and tried to follow them on Fedora Rawhide to test if they worked. While working on it, I came upon several questions that I will try to find answers to and update, if necessary.
I believe that we can test the modularity function the way it is written. Please, let me know, if you have any ideas how to make it better.

https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_module_list
https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_enable_disable_module
https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_install_remove_update_module

@sgallagh can you take a look at those and see if they look good to you? Thanks!

Hello,
I have updated the three test cases and tried to follow them on Fedora Rawhide to test if they worked. While working on it, I came upon several questions that I will try to find answers to and update, if necessary.
I believe that we can test the modularity function the way it is written. Please, let me know, if you have any ideas how to make it better.
https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_module_list

The "Optional" section should be dropped as we have merged the modular repos into the standard fedora-repos package. Thus, the dnf module list test must pass without installing additional repo packages or the criteria has not been met.

Also, I'd say that if dnf module list returns zero results, that should be an explicit failure.

https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_enable_disable_module

dnf module enable foo:stream/profile is actually not a valid command. Please don't list it. You can install a profile, but the enablement only applies to streams. Remove all references to profiles from the enable/disable tests.

This test (or the next one) also needs to include verification that the default streams and profiles are correctly used if the user omits them from the command line.

e.g. if module "foo" has default stream "1.0" and the "1.0" stream in turn has the default profile "server", then dnf module install foo must install the packages from the "1.0" stream and "server" profile.

Regarding removal of modules, I'm not sure we want to expressly include that in the criteria tests, simply because the mechanics of it may be difficult to express. When you remove a profile, it isn't the inverse of "install". It's "remove any of these packages whose removal won't result in any package outside of the profile also getting removed".

For updates, please also verify that dnf update (note the lack of the "module" keyword) stays on the current stream.

https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_install_remove_update_module

Thanks for the review, @sgallagh ! Can you get the revisions done soon, @lruzicka ? Thanks.

Also, I'd say that if dnf module list returns zero results, that should be an explicit failure.

Does this also apply for commands like dnf module list --enabled? If there are no enabled modules, it should return and empty list, shouldn't it?

dnf module enable foo:stream/profile is actually not a valid command. Please don't list it. You can install a profile, but the enablement only applies to streams. Remove all references to profiles from the enable/disable tests.

I removed the references of this command in enable/disable tests, but I would like to know, whether such command is a valid when installing the modules?

Regarding removal of modules, I'm not sure we want to expressly include that in the criteria tests, simply because the mechanics of it may be difficult to express. When you remove a profile, it isn't the inverse of "install". It's "remove any of these packages whose removal won't result in any package outside of the profile also getting removed".

I do not understand the mechanism, to be sincere, for safety, I deleted this from the test.

If you think, @sgallagh or @adamwill, that I have understood the instructions in the review correctly, I can publish the tests and create a field in the matrix. Please, let me know. Thanks.

https://bugzilla.redhat.com/show_bug.cgi?id=1616167 is a proposed blocker for a currently-broken scenario the tests should probably cover - should be enough info in the bug to add a test.

I am suggesting a test case to check whether the functionality is (or is not) affected by the mentioned bug.
https://fedoraproject.org/wiki/Lruzicka/Testcase_Modularity_without_repos

The test cases have been published in the public QA space and added to the validation matrix. I think, we can close this issue.