#1342 Upgrade path not kept when switching module streams -- stream should be part of the %dist
Closed: Invalid 2 years ago by jkaluza. Opened 2 years ago by hhorak.

The use case I care about is the following -- upgrading module streams like this:

dnf -y install @ruby:2.5
dnf -y module reset ruby
dnf -y module enable ruby:2.6
dnf -y update

What is the problem with this: Suppose a package rubygem-abrt is available in both streams, and is built from the same hash, so it has the same NVR (differs only in the modular part of the release part -- %dist RPM macro), e.g.:

rubygem-abrt-0.3.0-6.module_f30+4125+c677839c (part of ruby:2.5)
rubygem-abrt-0.3.0-6.module_f30+3838+0c262d71 (part of ruby:2.6)

See that NVR of the rubygem-abrt from ruby:2.6 is NVR lower than from ruby:2.5, so what happens on dnf -y update above is that rubygem-abrt is not updated at all and the user needs to run dnf distro-sync to downgrade this package.

While we can document this weird behaviour, I consider this counter-intuitive for users and also forcing packagers to add ruby stream into the Release in RPM or keeping the release number always strictly higher in the higher stream seems like nonesense.

Given stream names are usually versions of the main component, adding the stream into the %dist would probably work in most of the cases, so I believe this should be handled by modularity by default.

As a side-effect, this would also make it much easier to tell what stream the package is part of, which is currently not possible just from the NVR.

@vondruch @jaruga @pvalena FYI, as we have the same issue in RHEL-8.


May be there should be injected something like Requires: @ruby:2.5 instead of modifying NVR. In the end, the package is part of the module, so it is surprising there is no requirement like this.

May be there should be injected something like Requires: @ruby:2.5 instead of modifying NVR. In the end, the package is part of the module, so it is surprising there is no requirement like this.

While it makes sense to me, it would also mean that the dnf module enable ruby:2.6 would not work, as it would effectively break the yum metadata consistency/RPM state... unless the dnf module enable ruby:2.6 would also run dnf distro-sync to bring the RPM status back to the valid state, which would make sense to me.

But looking at Koji [1], the initial suggestion would also be helpful to understand where the build comes from.

But looking at Koji [1], the initial suggestion would also be helpful to understand where the build comes from.

Exactly, not only in koji, also in repository, on installed system, ...

What worries me is that you can have various parts from various modules installed at once. Shouldn't the content from disabled module be always replaced if possible [on update]?

What worries me is that you can have various parts from various modules installed at once. Shouldn't the content from disabled module be always replaced if possible [on update]?

That would make sense to me, but I'm wondering what the modularity folks' opinion is. @psabata wants to comment here maybe? :) (if he does not want to, I'd like to kindly ask then)

@hhorak, this doesn't seem related to MBS. Perhaps the following repo would be more appropriate for this ticket:
https://pagure.io/modularity

@mprahl Are you sure? I thought mbs is responsible for setting %dist, which is one way to handle this. issue.

@hhorak MBS does set %dist, but it's not up to the MBS maintainers to make a decision about how it's set. That has historically been the Modularity Working Group.

@mprahl Ok, understood. Can we still keep the discussion here? I cross-filled https://pagure.io/modularity/issue/143 but to not split the discussion, I'd like to keep the discussion in one place and there is some relevant info here already...

How are you going to upgrade from "barani" to "konsky" stream? Also how do you now if it should be upgraded? There is no such thing as "upgrade stream", there is a "switch stream".

How are you going to upgrade from "barani" to "konsky" stream?

You are certainly right that there is a chance for streams like this. However, this goes against the official suggestions:

https://docs.fedoraproject.org/en-US/modularity/making-modules/naming-guidelines/

You can admit that the names you suggested are supported, but I don't think they are sensible. This freedom also goes against well established practices how to handle NVRs in Fedora and keep the upgrade path sane.

I'd like to see resolved the common case before we focus on solving exceptions.

Also, if you want some rough statistics about modules in Fedora, here is list of modules and their naming collected by @jaruga a while ago:

https://pagure.io/jaruga-modules-branches

Not sure what you can get from the list, but it is just a mess IMO.

This freedom also goes against well established practices how to handle NVRs in Fedora and keep the upgrade path sane.

Actually, may be we should take this to FPC, because the truth is that for example RPM/DNF do not really impose constraints on NVR. The update works or not. The Fedora Packaging Guidelines imposes the restrictions which assures the upgrade path.

I'd like to see resolved the common case before we focus on solving exceptions.

That is not the exception. How do you, for example, upgrade "rolling" to "latest"? Stream name was designed to be an arbitraty string, not something you should be able to version.

Actually, may be we should take this to FPC, because the truth is that for example RPM/DNF do not really impose constraints on NVR. The update works or not. The Fedora Packaging Guidelines imposes the restrictions which assures the upgrade path.

FPC does not own modularity restrictions. It is up to Modularity WG.

That is not the exception. How do you, for example, upgrade "rolling" to "latest"? Stream name was designed to be an arbitraty string, not something you should be able to version.

"rolling" and "latest" are definitely exceptions. Most upstreams version their official releases. There is no reason why peculiar handling of "rolling" and "latest" should prevent updates from "1.0" to "2.0".

Metadata Update from @jkaluza:
- Issue close_status updated to: Invalid
- Issue status updated to: Closed (was: Open)

2 years ago

Feel free to keep the discussion here if you wish, but I'm going to close this issue for now, because the discussion here is not MBS related, but it is about modularity in general and maybe also about dnf which we have no big knowledge about in this project.

@ignatenkobrain There are obviously some streams where calling the stream switching "an upgrade" does not make sense . And there are obviously some where it does make perfect sense, and there are a lot of them, so let's focus on those, please.

My question is if below dnf -y update after dnf -y module enable ruby:2.6 is the right operation to switch the stream?

dnf -y module enable ruby:2.6
dnf -y update

dnf -y update is shown on the official document. But
https://docs.fedoraproject.org/en-US/modularity/using-modules/#_updating_the_system

Certainly dnf -y update does not work on below case when switching from ruby:2.5 to 2.6.

rubygem-abrt-0.3.0-6.module_f30+4125+c677839c (part of ruby:2.5)
rubygem-abrt-0.3.0-6.module_f30+3838+0c262d71 (part of ruby:2.6)

dnf -y update also does not work for most RPMs when switching from ruby:2.6 to 2.5.

@hhorak, note that I'm not saying that you don't have a point. As I said, feel free to discuss this here, but I think people around MBS which would read this ticket has no knowledge to answer how dnf module is working and if it works correctly in its current implementation.

It has been already pointed out byt @ignatenkobrain that the "body" to decide this change is the Fedora modularity WG. Since you have already created ticket at https://pagure.io/modularity/issue/143, there is no need to have this ticket open and track it in our tracking system, because until it is decided whether this is an issue or not and what the fix should be, MBS devs have nothing to do here.

but I'm going to close this issue for now, because the discussion here is not MBS related, but it is about modularity in general and maybe also about dnf which we have no big knowledge about in this project.

This ticket started from below question that below dist 's situation.

rubygem-abrt-0.3.0-6.module_f30+4125+c677839c (part of ruby:2.5)
rubygem-abrt-0.3.0-6.module_f30+3838+0c262d71 (part of ruby:2.6)

See https://koji.fedoraproject.org/koji/packageinfo?packageID=14740 for detail.

And it think that the logic creating the dist string is below part in this repository.

https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/utils/general.py#_367

def get_rpm_release(module_build):
...
    return "{prefix}{base_module_marking}{index}+{dist_hash}".format(
        prefix=prefix, base_module_marking=br_module_marking, index=index, dist_hash=dist_hash    )

So, I think there is a reason to open the ticket.

but I'm going to close this issue for now, because the discussion here is not MBS related, but it is about modularity in general and maybe also about dnf which we have no big knowledge about in this project.

This ticket started from below question that below dist 's situation.
rubygem-abrt-0.3.0-6.module_f30+4125+c677839c (part of ruby:2.5)
rubygem-abrt-0.3.0-6.module_f30+3838+0c262d71 (part of ruby:2.6)

See https://koji.fedoraproject.org/koji/packageinfo?packageID=14740 for detail.
And it think that the logic creating the dist string is below part in this repository.
https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/utils/general.py#_367
def get_rpm_release(module_build):
...
return "{prefix}{base_module_marking}{index}+{dist_hash}".format(
prefix=prefix, base_module_marking=br_module_marking, index=index, dist_hash=dist_hash )

So, I think there is a reason to open the ticket.

As stated in my comment above, although MBS controls the %dist value, the MBS maintainers do not decide how that gets set. That has historically been done by the Modularity WG. MBS just implements the specification they come up with.

If the Modularity WG decides to change the %dist value, then a ticket can be opened in this repo requesting that change.

As stated in my comment above, although MBS controls the %dist value, the MBS maintainers do not decide how that gets set. That has historically been done by the Modularity WG. MBS just implements the specification they come up with.

I see. I like you update below document to prevent same situation.
https://docs.fedoraproject.org/en-US/modularity/reporting-issues/

MBS (Module Build Service) issue tracker — Issues regarding MBS specifically.

I wisth Pagure will have this kind of template feature like GitHub not to create proper ticket in the future :)
https://github.com/containers/libpod/blob/master/.github/ISSUE_TEMPLATE.md
https://help.github.com/en/articles/creating-issue-templates-for-your-repository

I see. I like you update below document to prevent same situation.

I opened the ticket to update the documentation.
https://pagure.io/modularity/issue/145

I see. I like you update below document to prevent same situation.

I opened the ticket to update the documentation.
https://pagure.io/modularity/issue/145

Thanks, commented there with some extra wording to prevent confusion where the bugs should be filled.

About the top comment, after testing dnf module command, I concluded the best ways to switch the stream.

The use case I care about is the following -- upgrading module streams like this:

dnf -y install @ruby:2.5
dnf -y module reset ruby
dnf -y module enable ruby:2.6
dnf -y update

The dnf -y update is documented on https://docs.fedoraproject.org/en-US/modularity/using-modules/#_updating_the_system .
but I think it can be modified for better user experience.

The best way I recognized is

In case of the switched RPM is included in a profile's RPMs,

dnf -y module reset ruby
dnf -y --allowerasing module install ruby:2.6

or

dnf -y --allowerasing module install ruby:2.5

are minimal steps to switch the stream installing the RPMs without a trouble.

On my Fedora 30, the default profile is not available, running dnf -y --allowerasing module install ruby:2.6/default with profile is necessary. And to understand removed RPMs in the process of the installation in advance, people can check without -y at first.

In case of the switched RPM is not included in a profile's RPMs,

For example in case of modules/ruby, there is one profile to install ruby, ruby-devel, ruby-libs RPMs.
We can see all the RPMs in the module ruby:2.6 by dnf module info ruby:2.6.

So, when we want to switch current RPMs to all the RPM in the module ruby:2.6, we need to create below RPM list based on the result of dnf module info ruby:2.6, as we do not have the profile that means all the RPMs in case of modules/ruby:2.6.

$ cat rpm_list.txt 
ruby
ruby-devel
ruby-libs
rubygems
rubygems-devel
rubygem-bigdecimal
rubygem-bundler
rubygem-did_you_mean
rubygem-io-console
rubygem-irb
rubygem-json
rubygem-minitest
rubygem-net-telnet
rubygem-openssl
rubygem-power_assert
rubygem-psych
rubygem-rake
rubygem-rdoc
rubygem-test-unit
rubygem-xmlrpc
rubygem-abrt
rubygem-mysql2
rubygem-pg
rubygem-bson
rubygem-mongo

Then run below commands to switch the stream, and install the RPMs.

dnf -y module enable ruby:2.6
dnf -y --allowerasing distro-sync $(cat rpm_list.txt)

Here is the steps to back to non-modular RPMs.

dnf -y module reset ruby
dnf -y --allowerasing distro-sync $(cat rpm_list.txt)

Okay, what about case with dwm module? It has version-numbered streams and also latest.

Really, stream was designed to be an arbitrary string. Nobody guarantees any upgrade/downgrade paths. Basically you are not supposed to even change streams after you installed content.

The mistake here is considering the NVSC as equivalent to the EVRD. It’s basically an identifier for a virtual repository construct within the repository. Thus, it can be anything.

The mistake here is considering the NVSC as equivalent to the EVRD.

I mean the NSVC as equivalent to the NEVR, sorry.

dwm

Okay, what about case with dwm module? It has version-numbered streams and also latest.

I tested dwm case now. It looks okay.

Check initial condition on my Fedora 30.

$ dnf module list dwm
Last metadata expiration check: 0:05:37 ago on Thu 18 Jul 2019 01:11:50 PM CEST.
Fedora Modular 30 - x86_64
Name          Stream           Profiles                   Summary                             
dwm           latest           user, default [d]          Dynamic window manager for X        
dwm           6.0              user, default [d]          Dynamic window manager for X        
dwm           6.1 [d]          user, default [d]          Dynamic window manager for X        
dwm           6.2              user, default              Dynamic window manager for X        

Fedora Modular 30 - x86_64 - Updates
Name          Stream           Profiles                   Summary                             
dwm           latest           user, default [d]          Dynamic window manager for X        
dwm           6.0              user, default [d]          Dynamic window manager for X        
dwm           6.1 [d]          user, default [d]          Dynamic window manager for X        
dwm           6.2              user, default              Dynamic window manager for X        

Hint: [d]efault, [e]nabled, [x]disabled, [i]nstalled

Install dwm on default stream 6.1.

$ sudo dnf -y --allowerasing module install dwm
...
Installing group/module packages:
 dwm          x86_64        6.1-8.module_f30+2897+cb44e1e7        fedora-modular         36 k
Installing dependencies:
 dmenu        x86_64        4.9-1.fc30                            fedora                 29 k
 st           x86_64        0.8.1-4.fc30                          fedora                 50 k
Installing module profiles:
 dwm/default                                                                                 
Enabling module streams:
 dwm                        6.1    
...
$ rpm -q dwm
dwm-6.1-8.module_f30+2897+cb44e1e7.x86_64

Install dwm on stream latest.

$ sudo dnf -y module reset dwm
$ sudo dnf -y --allowerasing module install dwm:latest
...
Upgrading:
 dwm   x86_64   6.2-2.20190202gitcb3f58a.module_f30+2936+f41b7c53      fedora-modular    35 k
Installing module profiles:
 dwm/default

Enabling module streams:
 dwm            latest      
...

$ rpm -q dwm
dwm-6.2-2.20190202gitcb3f58a.module_f30+2936+f41b7c53.x86_64

Install dwm on stream 6.2.
As the stream 6.2 does not have default setting stream, I needed to run sudo dnf -y --allowerasing module install dwm:6.2/default. But it's no problem.

$ sudo dnf -y module reset dwm
$ sudo dnf -y --allowerasing module install dwm:6.2
...
No default profiles for module dwm:6.2
Dependencies resolved.
==============================================================================================
 Package               Architecture         Version               Repository             Size
==============================================================================================
Enabling module streams:
 dwm                                        6.2                                              

Transaction Summary
==============================================================================================

Complete!

$ sudo dnf -y --allowerasing module install dwm:6.2/default
...
Downgrading:
 dwm        x86_64        6.2-2.module_f30+2900+5c3cad7b          fedora-modular         35 k
Installing module profiles:
 dwm/default
...

$ rpm -q dwm
dwm-6.2-2.module_f30+2900+5c3cad7b.x86_64

rubygem-abrt

I also tested only rubygem-abrt that "has the same NVR (differs only in the modular part of the release part".

Below steps with "sudo dnf -y --allowerasing distro-sync" looks okay without error.

$ rpm -q rubygem-abrt
rubygem-abrt-0.3.0-6.fc30.noarch

$ sudo dnf -y module reset ruby
$ sudo dnf -y module enable ruby:2.5
$ sudo dnf -y --allowerasing distro-sync rubygem-abrt

$ rpm -q rubygem-abrt
rubygem-abrt-0.3.0-6.module_f30+4125+c677839c.noarch


$ sudo dnf -y module reset ruby
$ sudo dnf -y module enable ruby:2.6
$ sudo dnf -y --allowerasing distro-sync rubygem-abrt

$ rpm -q rubygem-abrt
rubygem-abrt-0.3.0-6.module_f30+3838+0c262d71.noarch

Based on the discussion, I sent pull-request to update the document Using Modules - Switching module streams section.
https://pagure.io/fedora-docs/modularity/pull-request/64

Login to comment on this ticket.

Metadata