The use case I care about is the following -- upgrading module streams like this:
dnf -y install @ruby:2.5 dnf -y module reset ruby dnf -y module enable ruby:2.6 dnf -y update
What is the problem with this: Suppose a package rubygem-abrt is available in both streams, and is built from the same hash, so it has the same NVR (differs only in the modular part of the release part -- %dist RPM macro), e.g.:
rubygem-abrt
%dist
rubygem-abrt-0.3.0-6.module_f30+4125+c677839c (part of ruby:2.5) rubygem-abrt-0.3.0-6.module_f30+3838+0c262d71 (part of ruby:2.6)
See that NVR of the rubygem-abrt from ruby:2.6 is NVR lower than from ruby:2.5, so what happens on dnf -y update above is that rubygem-abrt is not updated at all and the user needs to run dnf distro-sync to downgrade this package.
dnf -y update
dnf distro-sync
While we can document this weird behaviour, I consider this counter-intuitive for users and also forcing packagers to add ruby stream into the Release in RPM or keeping the release number always strictly higher in the higher stream seems like nonesense.
Given stream names are usually versions of the main component, adding the stream into the %dist would probably work in most of the cases, so I believe this should be handled by modularity by default.
As a side-effect, this would also make it much easier to tell what stream the package is part of, which is currently not possible just from the NVR.
@vondruch @jaruga @pvalena FYI, as we have the same issue in RHEL-8.
May be there should be injected something like Requires: @ruby:2.5 instead of modifying NVR. In the end, the package is part of the module, so it is surprising there is no requirement like this.
Requires: @ruby:2.5
While it makes sense to me, it would also mean that the dnf module enable ruby:2.6 would not work, as it would effectively break the yum metadata consistency/RPM state... unless the dnf module enable ruby:2.6 would also run dnf distro-sync to bring the RPM status back to the valid state, which would make sense to me.
dnf module enable ruby:2.6
But looking at Koji [1], the initial suggestion would also be helpful to understand where the build comes from.
Exactly, not only in koji, also in repository, on installed system, ...
What worries me is that you can have various parts from various modules installed at once. Shouldn't the content from disabled module be always replaced if possible [on update]?
That would make sense to me, but I'm wondering what the modularity folks' opinion is. @psabata wants to comment here maybe? :) (if he does not want to, I'd like to kindly ask then)
@hhorak, this doesn't seem related to MBS. Perhaps the following repo would be more appropriate for this ticket: https://pagure.io/modularity
@mprahl Are you sure? I thought mbs is responsible for setting %dist, which is one way to handle this. issue.
@hhorak MBS does set %dist, but it's not up to the MBS maintainers to make a decision about how it's set. That has historically been the Modularity Working Group.
@mprahl Ok, understood. Can we still keep the discussion here? I cross-filled https://pagure.io/modularity/issue/143 but to not split the discussion, I'd like to keep the discussion in one place and there is some relevant info here already...
How are you going to upgrade from "barani" to "konsky" stream? Also how do you now if it should be upgraded? There is no such thing as "upgrade stream", there is a "switch stream".
How are you going to upgrade from "barani" to "konsky" stream?
You are certainly right that there is a chance for streams like this. However, this goes against the official suggestions:
https://docs.fedoraproject.org/en-US/modularity/making-modules/naming-guidelines/
You can admit that the names you suggested are supported, but I don't think they are sensible. This freedom also goes against well established practices how to handle NVRs in Fedora and keep the upgrade path sane.
I'd like to see resolved the common case before we focus on solving exceptions.
Also, if you want some rough statistics about modules in Fedora, here is list of modules and their naming collected by @jaruga a while ago:
https://pagure.io/jaruga-modules-branches
Not sure what you can get from the list, but it is just a mess IMO.
This freedom also goes against well established practices how to handle NVRs in Fedora and keep the upgrade path sane.
Actually, may be we should take this to FPC, because the truth is that for example RPM/DNF do not really impose constraints on NVR. The update works or not. The Fedora Packaging Guidelines imposes the restrictions which assures the upgrade path.
That is not the exception. How do you, for example, upgrade "rolling" to "latest"? Stream name was designed to be an arbitraty string, not something you should be able to version.
FPC does not own modularity restrictions. It is up to Modularity WG.
"rolling" and "latest" are definitely exceptions. Most upstreams version their official releases. There is no reason why peculiar handling of "rolling" and "latest" should prevent updates from "1.0" to "2.0".
Metadata Update from @jkaluza: - Issue close_status updated to: Invalid - Issue status updated to: Closed (was: Open)
Feel free to keep the discussion here if you wish, but I'm going to close this issue for now, because the discussion here is not MBS related, but it is about modularity in general and maybe also about dnf which we have no big knowledge about in this project.
dnf
@jkaluza facepalm
@ignatenkobrain There are obviously some streams where calling the stream switching "an upgrade" does not make sense . And there are obviously some where it does make perfect sense, and there are a lot of them, so let's focus on those, please.
My question is if below dnf -y update after dnf -y module enable ruby:2.6 is the right operation to switch the stream?
dnf -y module enable ruby:2.6
dnf -y module enable ruby:2.6 dnf -y update
dnf -y update is shown on the official document. But https://docs.fedoraproject.org/en-US/modularity/using-modules/#_updating_the_system
Certainly dnf -y update does not work on below case when switching from ruby:2.5 to 2.6.
dnf -y update also does not work for most RPMs when switching from ruby:2.6 to 2.5.
@hhorak, note that I'm not saying that you don't have a point. As I said, feel free to discuss this here, but I think people around MBS which would read this ticket has no knowledge to answer how dnf module is working and if it works correctly in its current implementation.
dnf module
It has been already pointed out byt @ignatenkobrain that the "body" to decide this change is the Fedora modularity WG. Since you have already created ticket at https://pagure.io/modularity/issue/143, there is no need to have this ticket open and track it in our tracking system, because until it is decided whether this is an issue or not and what the fix should be, MBS devs have nothing to do here.
but I'm going to close this issue for now, because the discussion here is not MBS related, but it is about modularity in general and maybe also about dnf which we have no big knowledge about in this project.
This ticket started from below question that below dist 's situation.
See https://koji.fedoraproject.org/koji/packageinfo?packageID=14740 for detail.
And it think that the logic creating the dist string is below part in this repository.
https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/utils/general.py#_367
def get_rpm_release(module_build): ... return "{prefix}{base_module_marking}{index}+{dist_hash}".format( prefix=prefix, base_module_marking=br_module_marking, index=index, dist_hash=dist_hash )
So, I think there is a reason to open the ticket.
but I'm going to close this issue for now, because the discussion here is not MBS related, but it is about modularity in general and maybe also about dnf which we have no big knowledge about in this project. This ticket started from below question that below dist 's situation. rubygem-abrt-0.3.0-6.module_f30+4125+c677839c (part of ruby:2.5) rubygem-abrt-0.3.0-6.module_f30+3838+0c262d71 (part of ruby:2.6) See https://koji.fedoraproject.org/koji/packageinfo?packageID=14740 for detail. And it think that the logic creating the dist string is below part in this repository. https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/utils/general.py#_367 def get_rpm_release(module_build): ... return "{prefix}{base_module_marking}{index}+{dist_hash}".format( prefix=prefix, base_module_marking=br_module_marking, index=index, dist_hash=dist_hash ) So, I think there is a reason to open the ticket.
This ticket started from below question that below dist 's situation. rubygem-abrt-0.3.0-6.module_f30+4125+c677839c (part of ruby:2.5) rubygem-abrt-0.3.0-6.module_f30+3838+0c262d71 (part of ruby:2.6)
See https://koji.fedoraproject.org/koji/packageinfo?packageID=14740 for detail. And it think that the logic creating the dist string is below part in this repository. https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/utils/general.py#_367 def get_rpm_release(module_build): ... return "{prefix}{base_module_marking}{index}+{dist_hash}".format( prefix=prefix, base_module_marking=br_module_marking, index=index, dist_hash=dist_hash )
As stated in my comment above, although MBS controls the %dist value, the MBS maintainers do not decide how that gets set. That has historically been done by the Modularity WG. MBS just implements the specification they come up with.
If the Modularity WG decides to change the %dist value, then a ticket can be opened in this repo requesting that change.
I see. I like you update below document to prevent same situation. https://docs.fedoraproject.org/en-US/modularity/reporting-issues/
MBS (Module Build Service) issue tracker — Issues regarding MBS specifically.
I wisth Pagure will have this kind of template feature like GitHub not to create proper ticket in the future :) https://github.com/containers/libpod/blob/master/.github/ISSUE_TEMPLATE.md https://help.github.com/en/articles/creating-issue-templates-for-your-repository
I see. I like you update below document to prevent same situation.
I opened the ticket to update the documentation. https://pagure.io/modularity/issue/145
I see. I like you update below document to prevent same situation. I opened the ticket to update the documentation. https://pagure.io/modularity/issue/145
Thanks, commented there with some extra wording to prevent confusion where the bugs should be filled.
About the top comment, after testing dnf module command, I concluded the best ways to switch the stream.
The dnf -y update is documented on https://docs.fedoraproject.org/en-US/modularity/using-modules/#_updating_the_system . but I think it can be modified for better user experience.
The best way I recognized is
dnf -y module reset ruby
dnf -y --allowerasing module install ruby:2.6
or
dnf -y --allowerasing module install ruby:2.5
are minimal steps to switch the stream installing the RPMs without a trouble.
On my Fedora 30, the default profile is not available, running dnf -y --allowerasing module install ruby:2.6/default with profile is necessary. And to understand removed RPMs in the process of the installation in advance, people can check without -y at first.
dnf -y --allowerasing module install ruby:2.6/default
-y
For example in case of modules/ruby, there is one profile to install ruby, ruby-devel, ruby-libs RPMs. We can see all the RPMs in the module ruby:2.6 by dnf module info ruby:2.6.
dnf module info ruby:2.6
So, when we want to switch current RPMs to all the RPM in the module ruby:2.6, we need to create below RPM list based on the result of dnf module info ruby:2.6, as we do not have the profile that means all the RPMs in case of modules/ruby:2.6.
$ cat rpm_list.txt ruby ruby-devel ruby-libs rubygems rubygems-devel rubygem-bigdecimal rubygem-bundler rubygem-did_you_mean rubygem-io-console rubygem-irb rubygem-json rubygem-minitest rubygem-net-telnet rubygem-openssl rubygem-power_assert rubygem-psych rubygem-rake rubygem-rdoc rubygem-test-unit rubygem-xmlrpc rubygem-abrt rubygem-mysql2 rubygem-pg rubygem-bson rubygem-mongo
Then run below commands to switch the stream, and install the RPMs.
dnf -y module enable ruby:2.6 dnf -y --allowerasing distro-sync $(cat rpm_list.txt)
Here is the steps to back to non-modular RPMs.
dnf -y module reset ruby dnf -y --allowerasing distro-sync $(cat rpm_list.txt)
Okay, what about case with dwm module? It has version-numbered streams and also latest.
Really, stream was designed to be an arbitrary string. Nobody guarantees any upgrade/downgrade paths. Basically you are not supposed to even change streams after you installed content.
The mistake here is considering the NVSC as equivalent to the EVRD. It’s basically an identifier for a virtual repository construct within the repository. Thus, it can be anything.
The mistake here is considering the NVSC as equivalent to the EVRD.
I mean the NSVC as equivalent to the NEVR, sorry.
I tested dwm case now. It looks okay.
Check initial condition on my Fedora 30.
$ dnf module list dwm Last metadata expiration check: 0:05:37 ago on Thu 18 Jul 2019 01:11:50 PM CEST. Fedora Modular 30 - x86_64 Name Stream Profiles Summary dwm latest user, default [d] Dynamic window manager for X dwm 6.0 user, default [d] Dynamic window manager for X dwm 6.1 [d] user, default [d] Dynamic window manager for X dwm 6.2 user, default Dynamic window manager for X Fedora Modular 30 - x86_64 - Updates Name Stream Profiles Summary dwm latest user, default [d] Dynamic window manager for X dwm 6.0 user, default [d] Dynamic window manager for X dwm 6.1 [d] user, default [d] Dynamic window manager for X dwm 6.2 user, default Dynamic window manager for X Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled
Install dwm on default stream 6.1.
$ sudo dnf -y --allowerasing module install dwm ... Installing group/module packages: dwm x86_64 6.1-8.module_f30+2897+cb44e1e7 fedora-modular 36 k Installing dependencies: dmenu x86_64 4.9-1.fc30 fedora 29 k st x86_64 0.8.1-4.fc30 fedora 50 k Installing module profiles: dwm/default Enabling module streams: dwm 6.1 ...
$ rpm -q dwm dwm-6.1-8.module_f30+2897+cb44e1e7.x86_64
Install dwm on stream latest.
$ sudo dnf -y module reset dwm $ sudo dnf -y --allowerasing module install dwm:latest ... Upgrading: dwm x86_64 6.2-2.20190202gitcb3f58a.module_f30+2936+f41b7c53 fedora-modular 35 k Installing module profiles: dwm/default Enabling module streams: dwm latest ... $ rpm -q dwm dwm-6.2-2.20190202gitcb3f58a.module_f30+2936+f41b7c53.x86_64
Install dwm on stream 6.2. As the stream 6.2 does not have default setting stream, I needed to run sudo dnf -y --allowerasing module install dwm:6.2/default. But it's no problem.
sudo dnf -y --allowerasing module install dwm:6.2/default
$ sudo dnf -y module reset dwm $ sudo dnf -y --allowerasing module install dwm:6.2 ... No default profiles for module dwm:6.2 Dependencies resolved. ============================================================================================== Package Architecture Version Repository Size ============================================================================================== Enabling module streams: dwm 6.2 Transaction Summary ============================================================================================== Complete! $ sudo dnf -y --allowerasing module install dwm:6.2/default ... Downgrading: dwm x86_64 6.2-2.module_f30+2900+5c3cad7b fedora-modular 35 k Installing module profiles: dwm/default ... $ rpm -q dwm dwm-6.2-2.module_f30+2900+5c3cad7b.x86_64
I also tested only rubygem-abrt that "has the same NVR (differs only in the modular part of the release part".
Below steps with "sudo dnf -y --allowerasing distro-sync" looks okay without error.
$ rpm -q rubygem-abrt rubygem-abrt-0.3.0-6.fc30.noarch $ sudo dnf -y module reset ruby $ sudo dnf -y module enable ruby:2.5 $ sudo dnf -y --allowerasing distro-sync rubygem-abrt $ rpm -q rubygem-abrt rubygem-abrt-0.3.0-6.module_f30+4125+c677839c.noarch $ sudo dnf -y module reset ruby $ sudo dnf -y module enable ruby:2.6 $ sudo dnf -y --allowerasing distro-sync rubygem-abrt $ rpm -q rubygem-abrt rubygem-abrt-0.3.0-6.module_f30+3838+0c262d71.noarch
Based on the discussion, I sent pull-request to update the document Using Modules - Switching module streams section. https://pagure.io/fedora-docs/modularity/pull-request/64
Login to comment on this ticket.