I have proposed the bug as a prioritized bug. Do you want FESCo to vote it a blocker?

My understanding is that, since FreeIPA is a critical component of Fedora Server and it can't even be installed (see the referenced BZ that you've commented on -- it is filed from the IPA perspective), it is automatically release blocking.

But if you want to vote to agree on that, it is up to you. :-)

I'm mostly using the FESCO ticket to raise the awareness of this type of issue to FESCO and modularity teams: default module streams currently filter subpackage that are shipped as ursine packages. What should happen? Does policy need to be updated? Do these modules need to be removed as default module streams? etc.

Do these modules need to be removed as default module streams?

I think that is the only possible way forward at the moment.

Should we discuss this during next Monday's meeting?

default module streams currently filter subpackage that are shipped as ursine packages

So right now, release requirements can't be satisfied as a result of this. It seems to me that there's no other option than to disable default streams until this issue is resolved in modularity.

I felt this ticket didn't have enough time before the next FESCo meeting, plus already a number of people said they may miss the meeting. I think we should continue the discussion here and bring it to a vote at the next meeting. Of course, we can add it to the meeting if necessary.

In addition to fesco#2257, this is also related to fesco#2003. Even the most recent comment on fesco#2003 is the issue here again.

Given that the situation is currently broken for both users and developers of FreeIPA, I see no other immediate option than to remove these modules from the default module streams.

From a FESCo standpoint, I think we should also strongly consider right now requiring that RPMs in modules must also be maintained as ursine packages in Fedora. That is, you can opt in to making a module, but everything still also has to be a regular RPM. Certainly for build dependencies.

There are other topics related to this issue, but I'm trying to think of the immediate issue at hand. FreeIPA needs to work in F32.

Note that a similar problem exists for maven and non-modular Eclipse: glassfish-el is also filtered by modular maven: https://bugzilla.redhat.com/show_bug.cgi?id=1800528

Per lastest comments on on bz#1801882, Mikolaj is proposing making modular maven parallel installable with ursine maven.

• This has roughly been discussed since October and is still stalled.
• I'm not sure the impacts of a module removing a package from the filters portion -- will DNF pick that up on next metadata update? Or will systems installed with a broken module stay broken?
• Another solution commented by Miro was to stop shadowing ursine packages with modular packages. More generally, I think FESCO should determine a policy for what gets modularized, in light of Ursa Major/Prime/... not existing yet. Build tools (gcc, ant, maven, nodejs, jdk, make, ninja, ...) probably should be in that category, or at least be provided in a way that doesn't conflict with ursine equivalents (which should also be maintained).

My 2c.

Another solution commented by Miro was to stop shadowing ursine packages with modular packages. More generally, I think FESCO should determine a policy for what gets modularized, in light of Ursa Major/Prime/... not existing yet. Build tools (gcc, ant, maven, nodejs, jdk, make, ninja, ...) probably should be in that category, or at least be provided in a way that doesn't conflict with ursine equivalents (which should also be maintained).

I think this point is most important for F32 and going forward. With the modular repo(s) disabled on my F31 system, I am able to install pki-ca and everything works as expected. So from a user's perspective it feels like we're already there, we've just got this in the way. Again, speaking as what I think users would interpret the current behavior as.

For FESCo action, I propose (might need to be separate tickets?):

1) Vote to disable default module streams in all modules. This is a demonstrable problem right now and there is no indication it will be resolved before F32.
2) Define a module policy for Fedora. What can and cannot be in a module and the process for proposing new modules. This is going to be tedious, but we are in a situation where modules enabled now are breaking things for users and developers.
3) Move the modular repo configuration data out of the 'fedora-repos' package and in to a 'fedora-modular-repos' package providing another mechanism for users to avoid opting in to modules on their systems.

1) this will get my immediate +1. I've been saying that for months, but never proposed it, as there was not enough support in (previous) FESCo.
2) I believe this should be (have been) done by the people doing the modularity objective. FESCo can help (provide input) and should definitively approve it, but I don't see us writing it
3) also +1 from me

1) I agree fully. but that's nothing new :grin:
2) I agree with Miro. A set of actual Guidelines and policies for Modules would be good, but we're not the ones who need to write those.
3) Also agree, and in my opinion either that package should not be installed by default (at least on Workstation), or the modular repos need to be disabled by default. It's just too easy to wreck user's systems right now, with the repos enabled on all systems by default.

ad 3) I think that having the repos installed by default, but uninstallable is OK as long as we don't have any default modular streams. here's why:

a) we still get bug reports if modules are installed by accident even when there should be no implicit action of such
b) users only need one explicit action to install a module (dnf module install foo), not two (dnf install fedora-modular-repos && dnf module install foo)
c) paranoid users can still uninstall the package

Per discussion with @dcantrel today, another item to discuss/vote on would be:

4) Allow the reintroduction of default module streams after all filtering statements have been removed from the module definitions, in enforcement of modularity#146.

Note that we might still need a DNF change per comment 8 on bz#1800528 by Mikolaj.

It isn't immediately clear if topic 1 from above would be sufficient or whether it also requires a DNF change to fix released distros, but it will help with F32 and above.

My formal proposal for FESCo:

For Fedora 32 and onward, all default modular streams are banned. Existing defaults are removed. Once a comprehensive modular policy for Fedora is created and approved by FESCo, this ban can be lifted. The future policy should allow modularizing, but the primary objective is to make sure that the user experience and nonmodular packaging experience is not significantly worse than before modularity. (proposal ends here)

Notes: 3) is nice to have, but I don't think it requires FESCo approval (yet anyway). And I don't fully agree with 4).

Existing defaults are removed.

@sgallagh had a doc on what it would take to remove default streams. Do we have an updated version of it?

From the notes we made for our devconf talk:
All streams in Fedora 32 / rawhide:
1 stream and 1 package: hub, django, librealsense, lizardfs, octave, setools, standard-test-roles, sway, cobbler
1 package group: avocado, cri-o, dwm, ghc, mariadb, mysql, nest, nginx, nodejs, openmpi, postgres, subversion, swig
dep tree / java: eclipse, maven, ant, jmc
language stack: perl
other: perl-bootstrap

Defaults:

$ git clone https://pagure.io/releng/fedora-module-defaults/
$ git grep 'stream: ' *.yaml
ant.yaml:    stream: 1.10
avocado.yaml:    stream: 69lts
dwm.yaml:    stream: 6.2
maven.yaml:    stream: 3.5
scala.yaml:    stream: 2.10

It seems that scala module is gone, even though it is "default".

The only two non-trivial modules that both default and actually built are ant and maven.
Both are actively maintained by Stewardship SIG. The modular version is one patch version higher in the module in both cases.
EDIT: The default module has lower minor version than the non-modular rpm. The non-default module has the higher patch version. So dropping the default would mean a version bump, which is probably for the best.

For the other modules on both lists, it should be technically trivial to update the non-modular version (if wanted) and normally build in rawhide.

It seems we could just use the solution proposed by @sgallagh and simply drop the defaults.

The only two non-trivial modules that both default and actually built are ant and maven.
Both are actively maintained by Stewardship SIG. The modular version is one patch version higher in the module in both cases.
EDIT: The default module has lower minor version than the non-modular rpm. The non-default module has the higher patch version. So dropping the default would mean a version bump, which is probably for the best.

@zbyszek Just to clarify: The Stewardship SIG maintains the "ursine" packages that are also part of maven and ant modules to make sure packagers of dependent projects (including LibreOffice, eclipse, DogTag PKI / FreeIPA) don't get the Java rug pulled out from under them. mizdebsk and his team are responsible for the maven and ant modules, but they stopped maintaining or contributing to the respective "ursine" packages in any way more than a year ago.

For Fedora 32 and onward, all default modular streams are banned. Existing defaults are removed. Once a comprehensive modular policy for Fedora is created and approved by FESCo, this ban can be lifted. The future policy should allow modularizing, but the primary objective is to make sure that the user experience and nonmodular packaging experience is not significantly worse than before modularity. (proposal ends here)

@churchyard This sounds good. The only thing I'd want to add is that there should be some "gating" mechanism in place to actually enforce those policies before I'd ever want default streams to come back. Right now, any module maintainer can f*** up thousands of fedora deployments with a typo or a missing line in a YAML file

What is part of the policy is out of scope here. What you say IMHO fits into the general "The future policy should allow modularizing, but the primary objective is to make sure that the user experience and nonmodular packaging experience is not significantly worse than before modularity." sentence. I think that if we try to compose the entire policy here, we will never approve this in timely manner.

What is part of the policy is out of scope here. What you say IMHO fits into the general "The future policy should allow modularizing, but the primary objective is to make sure that the user experience and nonmodular packaging experience is not significantly worse than before modularity." sentence. I think that if we try to compose the entire policy here, we will never approve this in timely manner.

Sure. It was not my intention to talk about policy details, but to just add that - whatever policy is the result of this - it needs an actual enforcement process as well.

I agree with that fully and will make sure that we don't approve a policy that cannot be enforced by automation (well, at least I won't vote in favor for such policy).

+1 to @churchyard 's proposal.

Also I've hit today https://bugzilla.redhat.com/show_bug.cgi?id=1806303 which means, if we release F32 with any default modules, due to the way DNF works, we won't be able to easily remove them (without change of DNF).

<mhroncok> zbyszek told me on telegram to vote +1. but he can back that up in the ticket later if needed

Confirm. Apologies for missing the meeting.

I see that the PR for dropping default streams has been merged in fedora-module-defaults already. How long will it take to propagate from there?

I think it would be good if @cipherboy could confirm that his original issue is actually "solved" for fresh f32/rawhide by this change.

I see that the PR for dropping default streams has been merged in fedora-module-defaults already. How long will it take to propagate from there?

As soon as the next successful compose completes (and the mirror network catches up).

I think it would be good if @cipherboy could confirm that his original issue is actually "solved" for fresh f32/rawhide by this change.

openQA FreeIPA server deployment tests passed in the 20200225 compose, indicating this has now taken effect.

I'm wondering, is there still some default module stream active for rawhide? I'm still seeing modular package additions / removals / upgrades in rawhide compose reports. Is that expected?

I assume you mean this (from "Fedora rawhide compose report: 20200507.n.0 changes"):
===== ADDED PACKAGES =====
Package: cri-o-2:1.16.6-1.module_f33+8871+ba4b04dc
Summary: Kubernetes Container Runtime Interface for OCI-based containers
RPMs: cri-o
Size: 118.68 MiB

I'm wondering, is there still some default module stream active for rawhide?

According to fedora-module-defaults, no. I think this is just an idiosyncrasy in the reporting tool. It seems to report non-default modules as "added packages" (?). This should probably be filtered out.

I assume you mean this (from "Fedora rawhide compose report: 20200507.n.0 changes"):

Yeah, I was looking at that exact report.

===== ADDED PACKAGES =====
Package: cri-o-2:1.16.6-1.module_f33+8871+ba4b04dc
Summary: Kubernetes Container Runtime Interface for OCI-based containers
RPMs: cri-o
Size: 118.68 MiB

I'm wondering, is there still some default module stream active for rawhide?

According to fedora-module-defaults, no. I think this is just an idiosyncrasy in the reporting tool. It seems to report non-default modules as "added packages" (?). This should probably be filtered out.

@zbyszek Probably. Though I'm always wondering why some module updates show up as "added" packages, some as "removed" packages, others as "upgraded" and many others still as "downgraded" ...