As discussed in today's QA meeting, we need to ensure we have permanent test cases covering release-blocking modularity functionality in the validation pages. We do have three existing test cases from the F28 test day:
so those can act as a starting point. We would at least need to polish those up and move them to 'production' page names, and add them to one of the validation matrix templates (likely base). We also need to consider if more tests are needed, and if other tools add module support (e.g. GNOME Software), whether we should add tests for those.
Tagging @sumantrom , @sgallagh and @langdon to start - sumantrom is tasked to work on this from our side, Stephen and Langdon can help with reviewing the tests and deciding what coverage is necessary. Thanks folks!
Metadata Update from @adamwill: - Issue set to the milestone: Fedora 29
ping @sumantrom , what is the current state here? Thanks!
Hello Adam, after some of the reorganization of the QA team, I think I should be the one working on this issue. I have already started to talk with various people, including @sgallagh, to get more info on Modularity in Fedora 29. What I think is needed now:
If you are coming to Flock, we can talk about it, otherwise, I will probably fully devote to this, when we come back from Flock.
Metadata Update from @lruzicka: - Issue assigned to lruzicka (was: sumantrom)
In an email communication with Stephen, I got the most important features, we will have to test. For the start, I suggest to pay attention to them, especially to points 1 to 3.
dnf module list
dnf module
dnf install nodejs
Perhaps, each of the test cases could cover one item on this list.
I don't think we need to do anything further to the release criteria for now. I would like to focus on getting test cases written and published. I'd like this to happen sooner than "after flock", to be honest, we should be doing it now. Thanks.
Ok, I will start working on it now, or rather as soons as possible because I am now on PTO. I will be back on Friday and I will use the whole day to work on this to fix it before Flock then.
Hello, I have updated the three test cases and tried to follow them on Fedora Rawhide to test if they worked. While working on it, I came upon several questions that I will try to find answers to and update, if necessary. I believe that we can test the modularity function the way it is written. Please, let me know, if you have any ideas how to make it better.
https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_module_list https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_enable_disable_module https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_install_remove_update_module
@sgallagh can you take a look at those and see if they look good to you? Thanks!
Hello, I have updated the three test cases and tried to follow them on Fedora Rawhide to test if they worked. While working on it, I came upon several questions that I will try to find answers to and update, if necessary. I believe that we can test the modularity function the way it is written. Please, let me know, if you have any ideas how to make it better. https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_module_list
The "Optional" section should be dropped as we have merged the modular repos into the standard fedora-repos package. Thus, the dnf module list test must pass without installing additional repo packages or the criteria has not been met.
fedora-repos
Also, I'd say that if dnf module list returns zero results, that should be an explicit failure.
https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_enable_disable_module
dnf module enable foo:stream/profile is actually not a valid command. Please don't list it. You can install a profile, but the enablement only applies to streams. Remove all references to profiles from the enable/disable tests.
dnf module enable foo:stream/profile
This test (or the next one) also needs to include verification that the default streams and profiles are correctly used if the user omits them from the command line.
e.g. if module "foo" has default stream "1.0" and the "1.0" stream in turn has the default profile "server", then dnf module install foo must install the packages from the "1.0" stream and "server" profile.
dnf module install foo
Regarding removal of modules, I'm not sure we want to expressly include that in the criteria tests, simply because the mechanics of it may be difficult to express. When you remove a profile, it isn't the inverse of "install". It's "remove any of these packages whose removal won't result in any package outside of the profile also getting removed".
For updates, please also verify that dnf update (note the lack of the "module" keyword) stays on the current stream.
dnf update
https://fedoraproject.org/wiki/User:Lruzicka/Draft/dnf_install_remove_update_module
Thanks for the review, @sgallagh ! Can you get the revisions done soon, @lruzicka ? Thanks.
Does this also apply for commands like dnf module list --enabled? If there are no enabled modules, it should return and empty list, shouldn't it?
dnf module list --enabled
I removed the references of this command in enable/disable tests, but I would like to know, whether such command is a valid when installing the modules?
I do not understand the mechanism, to be sincere, for safety, I deleted this from the test.
If you think, @sgallagh or @adamwill, that I have understood the instructions in the review correctly, I can publish the tests and create a field in the matrix. Please, let me know. Thanks.
https://bugzilla.redhat.com/show_bug.cgi?id=1616167 is a proposed blocker for a currently-broken scenario the tests should probably cover - should be enough info in the bug to add a test.
I am suggesting a test case to check whether the functionality is (or is not) affected by the mentioned bug. https://fedoraproject.org/wiki/Lruzicka/Testcase_Modularity_without_repos
The test cases have been published in the public QA space and added to the validation matrix. I think, we can close this issue.
Metadata Update from @lruzicka: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.