Hey, after recent events in Fedora and after discussions on 2 latest FESCo meetings, I'm worried that modularity is too "wild" and that the situation is actively hurting Fedora as a project.
Let me give a bit context. This is how I remember things, correct me if I'm wrong:
Since we abandoned the "modularize everything" idea, every presentation about Modularity I've seen, every article I've read, every change proposal... everything made me think that modules are extra goodies that we can stick on top of Fedora - and that gives much benefits to everybody.
Instead, what we see today, is that large parts of Fedora are moving to modules only, we see that non-modular content depends on modular content (as a result of the Modules for Everyone change, this is technically possible yet I don't remember this being approved as a thing to do), we try to develop mechanisms to enable this in the buildroot.
Shouldn't we stop here and reconsider? What is the scope of modularity? Ultimately, what parts of Fedora we wish to maintain in modules? I'm afraid that currently, the answer is: Whatever the maintainer wants.
Here's why I think this hurts Fedora:
fedpkg local
I believe that we should define clear boundaries and expectations that say:
We should actively work with the maintainers of the first category to make it happen, but we should check if the conditions of the second category are met and we should forbid modularizing the third category.
At the end, there should be criteria for creating modules and somebody should check them. Essentially creating a new module or moving stuff into a module should be a Fedora change (of a sort) - it should be discussed, approved, announced.
A quote from the mailing list to ilustrate the current frustration:
It seems a bit crazy to me that we have packages built for Fedora that aren't available for users to install. Why wouldn't we make everything maximally available? I used to love Fedora, because I just play with all the bits. But now, a lot of those bits are going away... I have less to play with... and the focus seems more targeted towards Fedora's internal needs, and not Fedora's users needs. Contributing to Fedora is so much harder now. Do we have to make it harder by making certain packages unavailable to regular users (and casual packager-contributors like me)?
I'm afraid that currently, the answer is: Whatever the maintainer wants.
I agree on this.
rawhide only packages for Rust
Sorry , but how does it break anything? By doing this we were able to ship updates for F28 and F29 which we were not able to do with non-modular content.
It is not clearly visible what versions of what are actually delivered. In theory, my package can build against libfoo3 from nonmodular Fedora, but users get libfoo4 from the WhatnotModule and things will break; as a maintainer I have no idea when this can happen.
As long as you have proper dependencies in RPM, it should be good.
I feel that pain. On one side, having single repo with everything is cool. But there is no tooling to manage content over multiple releases of Fedora. I mean, it is called modularity nowadays.
I think we should be aiming to go fully-modular, but this requires a lot of work and it seems we do not have enough people to do that work (or even define it). I see future UX as being half-modular (bash for example can live in module with defined lifecycle while some random libfoo can be non-modular). When default stream changes, we rebuild non-modular content in automated way (not the way we do mass rebuilds now).
So I think whatever decision we make, we need to make sure that there are enough people who would be working on defining what is needed to be done to make modularity easier for both users and packagers and then work on a tooling.
rawhide only packages for Rust Sorry , but how does it break anything? By doing this we were able to ship updates for F28 and F29 which we were not able to do with non-modular content.
Hey Igor!
I wouldn't word it as breaking something, but I will say that I've found it difficult to understand how to manage my rpick module yaml file. I know you wrote a tool to generate that file so you don't have to manage it by hand, but it seems like this is something that the infrastructure should do for me. Since the infrastructure doesn't do that, I would say that modularity does seem to make managing packages more difficult than the traditional RPMs did, at least for Rust packages.
Wasn't modularity marketed as being good for languages like Rust, or am I misremembering that?
So you (we, Fedora) have 2 possibilities:
Conclusions from this:
So you are right that modules are not very packagers-friendly. But maintaining 500+ packages in multiple stable releases is even more painful. And yes, I do agree that we should have much more automation. And no, it seems nobody is looking to work on that (I am willing to help as much as I can, but I can't do all work).
Problem 1: I fear there is a lack of shared vision for Modularity.
To me it sometimes feels like we're maintaining and bugfixing in a situation that still requires design thinking.
One consequence of a missing shared vision is a lack of guidance on how to use certain Modularity features — namely the module API and to some extent the buildroot-only modules. It is very easy for people to build silos of private dependencies instead of collaborating with each other. Integration is a huge part of a Linux distributions' value. Are we even sure those concepts have been designed to solve problems and help the community as a whole?
Another one is that we don't seem to agree on to what extent can non-default module streams affect the rest of the system. As an extreme example, would a module stream only installable in a container be valuable? To me personally, yes. But do we all agree?
Problem 2: We move too slow.
I feel people tend to work on too many things at once, and some of the things get very little progress every week. I see this on myself, too.
As an example, some crutial parts of the architecture — like getting default modules to the buildroot — are being discussed for months but not implemented. And that has terrible consequences (the java packages for example). Generating a lot of unnecessary work for people slowing us down even more.
Moving forward?
I feel that first we need change the way we work in order to move forward effectively. I believe fixing these two problems would get us there mostly.
On Tue, 2019-04-02 at 18:35 +0000, Igor Gnatenko wrote:
Modularity way: have all crates in their latest versions in rawhide and somehow generate modulemds out of that. Traditional way: have all crates in their versions (most likely outdated, just look at F29) and not being able to build your app because they are outdated...
Is #2 different for Rust than it is for other languages in Fedora?
Conclusions from this: =20 We don't have way to easily update multiple packages and keep them up to date in stable releases (soon, we won't be able to do this in rawhide either, due to gating); We don't have way to easily generate modulemds and somewhat automatically create modules (and it seems we don't have any people who would be working on this tooling anytime soon); =20 =20 So you are right that modules are not very packagers-friendly. But maintaining 500+ packages in multiple stable releases is even more painful. And yes, I do agree that we should have much more automation. And no, it seems nobody is looking to work on that (I am willing to help as much as I can, but I can't do all work).
Conclusions from this: =20 We don't have way to easily update multiple packages and keep them up to date in stable releases (soon, we won't be able to do this in rawhide either, due to gating); We don't have way to easily generate modulemds and somewhat automatically create modules (and it seems we don't have any people who would be working on this tooling anytime soon); =20
=20 So you are right that modules are not very packagers-friendly. But maintaining 500+ packages in multiple stable releases is even more painful. And yes, I do agree that we should have much more automation. And no, it seems nobody is looking to work on that (I am willing to help as much as I can, but I can't do all work).
Yeah, I agree that it's not easy either way.
No. Java and Go are the same.
Metadata Update from @churchyard: - Issue tagged with: meeting
OK, I think this thread has gotten a bit lost in specifics, whereas @churchyard 's initial post was much more high-level. I think the initial point is valid: Modularity is lacking some key top-level policy (and documentation for that policy). Unfortunately, as is all too-common in Open Source, we landed the technology before that policy was established. As a result, we have situations we're now reacting to, like the Java package retirement.
I think I generally agree with @churchyard on the three categories of package, but they're not of equal size.
I think this is probably the easiest case to describe. The things that make sense to always treat as a module fit all of these conditions:
Realistically, this is going to be the majority of packages in Fedora, I think. (Working under the assumption that we manage a heavy focus on packager experience improvements). We need to figure out where the lines are for software to move from the traditional approach into a modular approach, and in such a way that it doesn't break those who aren't ready to do it yet.
It is worth noting that once we land the work we are doing for getting modular default stream content into the non-modular buildroots (for both mock and Koji builds), a lot of these problems become much easier. For most purposes, there will be no difference within a release as to whether their dependency was provided by a non-modular package or a modular package in a default stream.
So, the hard part now is setting these rules. Timing matters as well: if rules are set today, without the modular-buildroot changes, then I think the answer can be drawn pretty simply: Until we revise this policy, do not move a package from the non-modular set to a default module stream if you (or the appropriate SIG) are not the owners of every package known to depend on it and are moving them all to one or more module streams at the same time. It is permissible to create a new, non-default module stream for alternate versions of any package, but if your package has known non-modular dependencies, it must remain in the non-modular repo for now.
Once we fix the buildroot issue, it should be safe to more or less open the floodgates. As long as mock/koji/dnf can properly resolve the buildrequires from things that are either part of the non-modular set or contained in a default module stream, it should be fine for most packagers. Edge-cases will likely crop up (it's possible for default module streams to conflict with one another), but those are hard to predict and should be managed on an individual basis.
This will probably be the smallest group, ultimately. This should be an identified set of packages whose API is so fundamental that it is essentially the definition of what constitutes a "Fedora Release". I think this is going to be a very minimal set of things (probably not too dissimilar to the content of a container base image plus the kernel and boot stack).
On Thu, 2019-04-04 at 05:47 +0000, Igor Gnatenko wrote:
Is #2 different for Rust than it is for other languages in Fedora? =20 No. Java and Go are the same.
What about languages like Python or C/C++ or others? Is this problem unique to Rust, Java, and Go for some reason?
I think I'm not understanding why modularity is thought of as being particularly helpful to Rust/Java/Go (i.e., why those languages and not all languages).
@sgallagh Thanks!  
On Thu, 2019-04-04 at 05:47 +0000, Igor Gnatenko wrote: Is #2 different for Rust than it is for other languages in Fedora? =20 No. Java and Go are the same. What about languages like Python or C/C++ or others? Is this problem unique to Rust, Java, and Go for some reason? I think I'm not understanding why modularity is thought of as being particularly helpful to Rust/Java/Go (i.e., why those languages and not all languages).
What about languages like Python or C/C++ or others? Is this problem unique to Rust, Java, and Go for some reason? I think I'm not understanding why modularity is thought of as being particularly helpful to Rust/Java/Go (i.e., why those languages and not all languages).
Golang/Rust binaries are statically linked against fast-moving libraries, which break API often. Which means if you want to build the latest version of a package, you often need to update several libraries to make it work. This is especially true in Golang where semver is rarely used (though encouraged now) and libraries often do not make releases at all, and just assumes you're building from GIT master branch (which can break compatibility anytime). In Rust, you have a strong semver culture, but it seems a lot of devs are afraid of making a 1.0.0 release and being stable. In Golang, we lack manpower for keeping everything up-to-date, even in Rawhide. Since libraries publishing releases are rare, you need to periodically sync with GIT. No Anitya to warn you of a new release. No simple way to know if an update will break other dependent packages either. Keeping everything compatible on several branches is tricky. Sometimes your build fails because that particular library has not been updated for a while except on Rawhide.
@sgallagh @churchyard
I wouldn't actually try to answer the question: "modularize or not" through a policy. We want people to get creative, and want them to find new usages for modules and develop this idea further. And as soon as tool is there, it is open, and we shouldn't really restrict all the ways the tool can be used, however weird it might be.
But we should restrict the way how Fedora uses the tool. And I see Fedora part and Fedora control on Modularity focused on those modules which we call default.
Thus I want to suggest to limit the policy to default modules only and enforce certain limitations on them. And then the question "modularize or not this thing in Fedora" will be answered based on those limitations we enforce.
So my suggestion:
A: Default modules MUST be supported and work the same way as regular rpm packages
Corollary A1: Packages provided by default modules MUST be supported in full (not as "just a dependency for this particular rpm") Corollary A2: Default modules MUST be supported through entire lifetime of Fedora release their are default for. Corollary A3: Default modules MUST follow Fedora release schedule and policies (Change Proposals, Freezes and Alpha/Beta releases) Corollary A4: Default modules MUST NOT depend on non-default modules
B: Non-modular buildroot MUST NOT contain non-default modules
The reasoning is: if user see no visible difference between modular and non-modular rpm (which is the case for default modules), then there shouldn't be any hidden difference as well.
So my suggestion: A: Default modules MUST be supported and work the same way as regular rpm packages Corollary A1: Packages provided by default modules MUST be supported in full (not as "just a dependency for this particular rpm")
So, in practice, you want a policy on default module streams that all produced (i.e. not filtered out) artifacts in the stream must be part of the API definition for that stream?
I'm not sure I like that. The whole purpose of the API definition is to allow for such cases (e.g. I'm supporting this library for my own purposes and don't want to own it for all possible situations). That said, this is not a hill I'm willing to die on, so if FESCo in general feels that this is a restriction we want on the default streams, I'll fall in line.
Corollary A2: Default modules MUST be supported through entire lifetime of Fedora release their are default for.
Absolutely agreed. Please note, we have just adopted a lifecycle policy that all streams of a module in Fedora must have their EOL aligned with the EOL of a Fedora release. So if you can't support a stream (default or not) through the end of a release's lifetime, you should not include that stream at all.
Corollary A3: Default modules MUST follow Fedora release schedule and policies (Change Proposals, Freezes and Alpha/Beta releases)
Agreed. We already have policy that changes to a default stream have to go through the Change Process (unless they are just moving the same version from non-modular to a default module stream). Policy suggestion: this must be complete by Beta Freeze or deferred to the next release.
Corollary A4: Default modules MUST NOT depend on non-default modules
I disagree with this. I think the ability to have this happen is a major strength of Modularity as it was designed. One of the major issues we have in Fedora today is that if two important packages depend on different versions of the same dependency, only one of them gets to be in Fedora. (As a contrived example, let's consider a desire to ship two different PHP web applications. AppA requires PHP 7.2 and can't work because 7.3 broke compatibility. AppB can only work with 7.3+.
By requiring that default streams cannot depend on non-default streams, only one of AppA or AppB would be permissible to have a default stream.
Maybe this is less problematic if we set stronger guidelines on the stream summary and description text so it's easier to discover non-default streams, I suppose.
What problems, specifically, do you see this rule resolving? It might reduce potential stream conflicts, I suppose. But it's not as if conflicts don't exist in the non-modular RPMs too.
B: Non-modular buildroot MUST NOT contain non-default modules The reasoning is: if user see no visible difference between modular and non-modular rpm (which is the case for default modules), then there shouldn't be any hidden difference as well.
I am willing to agree to this at this point in time. In the fullness of time, I'd like to see the ability to do buildroot overrides to have non-default streams in the buildroot for modular RPMs if-and-only-if it doesn't change the runtime dependencies. (E.g. I might want to build an application with a pre-release golang or an older Sphinx because the default broke compat, etc.) But that's technology we don't have and likely won't for a year or more, so let's agree to this rule for now.
So, in practice, you want a policy on default module streams that all produced (i.e. not filtered out) artifacts in the stream must be part of the API definition for that stream? I'm not sure I like that. The whole purpose of the API definition is to allow for such cases (e.g. I'm supporting this library for my own purposes and don't want to own it for all possible situations). That said, this is not a hill I'm willing to die on, so if FESCo in general feels that this is a restriction we want on the default streams, I'll fall in line.
The issue here is that as a user running dnf install X I have no way to know that I am installing partially supported package. What if I am building my app depending on it?
dnf install X
It also takes the name in the default namespace, which means it gets in the way for people who want to maintain this package properly.
It essentially makes it look like package is available and maintained, while it is not.
Corollary A4: Default modules MUST NOT depend on non-default modules I disagree with this. I think the ability to have this happen is a major strength of Modularity as it was designed. One of the major issues we have in Fedora today is that if two important packages depend on different versions of the same dependency, only one of them gets to be in Fedora. (As a contrived example, let's consider a desire to ship two different PHP web applications. AppA requires PHP 7.2 and can't work because 7.3 broke compatibility. AppB can only work with 7.3+. By requiring that default streams cannot depend on non-default streams, only one of AppA or AppB would be permissible to have a default stream. Maybe this is less problematic if we set stronger guidelines on the stream summary and description text so it's easier to discover non-default streams, I suppose. What problems, specifically, do you see this rule resolving? It might reduce potential stream conflicts, I suppose. But it's not as if conflicts don't exist in the non-modular RPMs too.
I disagree with this. I think the ability to have this happen is a major strength of Modularity as it was designed. One of the major issues we have in Fedora today is that if two important packages depend on different versions of the same dependency, only one of them gets to be in Fedora. (As a contrived example, let's consider a desire to ship two different PHP web applications. AppA requires PHP 7.2 and can't work because 7.3 broke compatibility. AppB can only work with 7.3+. By requiring that default streams cannot depend on non-default streams, only one of AppA or AppB would be permissible to have a default stream. Maybe this is less problematic if we set stronger guidelines on the stream summary and description text so it's easier to discover non-default streams, I suppose. What problems, specifically, do you see this rule resolving? It might reduce potential stream conflicts, I suppose. But it's not as if conflicts don't exist in the non-modular RPMs too.
The problem I am trying to address is the closure for the policy. For example, I say default module X needs to follow Fedora Change process. But then if it depends on non-default module Y, and Y doesn't follow Fedora change process, then we invalidate this requirement for X. Or say, X has a promise that CVEs are going to be fixed and Y has not, than again, the CVE promise for X is invalidated.
But I see your point. I guess we should come with another name. Like core modules, which follow the guidelines but may not be exposed as default. And then those "good enough" modules could be used as provider for dependencies for default modules.
It's getting harder to contribute to Fedora with all the mass orphaning of dependencies, and I don't have time to figure it all out.
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/RESI7MA5XJKNKHOK3RBGTND2LJ7DQYMN/
@churchyard Yes, the mass-orphaning of Java is turning into a disaster. I've red-flagged this internally today; I'm trying desperately to get the necessary infrastructure enhancements landed as soon as possible.
We will discuss this during the FESCo meeting on Friday at 15:00UTC in #fedora-meeting-1 on irc.freenode.net.
Correction: the meeting will be in #fedora-meeting.
AGREED: FESCo forbids the addition of new default stream settings without FESCo approval until further notice while we work out the buildroot problems. (+7, 0, -0)
ACTION: mhroncok to file a bug on the modularity Taiga describing the use cases we need to keep in mind when writing those (modularity default streams) rules
I will leave this ticket open and assigned to @churchyard as a reminder to file the Taiga card.
Metadata Update from @bowlofeggs: - Issue untagged with: meeting - Issue assigned to churchyard
@bowlofeggs for existing packages or for new ones?
@ignatenkobrain Effective immediately, the fedora-module-defaults repo will not be merging any changes without FESCo exception. We will revisit this once we have the buildroot situation worked out.
@sgallagh, reading IRC logs it seemed that we don't want to "move ursine packages to module-only", but somehow that was lost in the vote. Was that intentional?
Because that means, I can't add any more Rust applications without FESCo exception.
@ignatenkobrain That was intentional. Yes, you need FESCo approval for Rust applications.
The reasoning was that we didn't want to keep arguing over where the line of "acceptable" defaults was in the meeting. We opted to disallow all new defaults, with the option to have FESCo vote to approve some individually.
Question: can I get a vote from FESCo on lessening the proscription on defaults changes to exclude changes that do not touch the default stream?
In other words, if someone wants to add a set of default profiles for a new, non-default stream, I think it's reasonable to allow that without a FESCo ticket.
We've stopped "new default stream settings". I think this does not affect non-default streams at all. What am I not getting?
I may have misread our decision. Carry on.
@churchyard we left this open as a reminder for you to file a follow-up ticket in taiga... Any progress?
Sorry, I completely procrastinated this one off. Will try to get back to this after Czech PyCon (the week after next).
Metadata Update from @churchyard: - Issue tagged with: stalled
At a high level, these are the Fedora Council's overall goal's for modularity (from our meeting today):
I want, therefore, to answer this part from Miro's initial post:
From a Council point of view, my response to this is: yes, and that's fine.
When a maintainer wants to provide multiple versions of software, maintaining that in modules should be a good answer. (This is point 1, and point 2 should help.)
Additionally, modules should make it easy to maintain a single stream of software across Fedora EPEL and Fedora OS — so therefore, that should also be encouraged.
We realize that we're not at the perfect world where this is all true, but we're in support of moving towards things that make it true.
To clarify, I'm all in favor of: maintainers should have the ability to provide multiple versions of software via modularity.
However, IMHO maintainers should not have the ability to move existing default version of software to modules only without coordinating with all dependent package maintainers - and in case of large stacks, this should be done via a Fedora change proposal, not randomly.
To clarify, I'm all in favor of: maintainers should have the ability to provide multiple versions of software via modularity. Great. However, IMHO maintainers should not have the ability to move existing default version of software to modules only without coordinating with all dependent package maintainers
To clarify, I'm all in favor of: maintainers should have the ability to provide multiple versions of software via modularity. Great.
However, IMHO maintainers should not have the ability to move existing default version of software to modules only without coordinating with all dependent package maintainers
I disagree here. The need to coordinate with dependent package maintainers will be overwhelmingly mitigated by landing Ursa Prime, because there will be no change to the dependent packages' workflow compared to non-modular content (barring bugs creeping in that we would fix in Ursa Prime, obviously).
and in case of large stacks, this should be done via a Fedora change proposal, not randomly.
This I agree with 100%.
I am very happy that Ursa Prime will mitigate most of the "BuildRequiring modular package from a nonmodular package" issues. There are more things to coordinate than that and even if most of the coordination will end up being bunch of people acking the change, it works with me.
This is now required to define in order to allow default modular streams. Assigning to @dmach.
Metadata Update from @churchyard: - Issue assigned to dmach (was: churchyard)
@dmach I would appreciate if we can do something with this hanging ticket.
I see that people are being confused and keep packages non-modular, but expect that they will install without problems if you install a module. For example, https://bugzilla.redhat.com/show_bug.cgi?id=1830658.
I guess if we won't do anything with this ticket for let's say 3 months, I would propose to vote if we want to stop producing modular content and let Modularity people do it in some other place.
From what I see from today's rawhide:
Also if I understand correctly, the flatpak builds leverage modularity, so probably @otaylor or related people can speak up and come up with idea how to move forward?
I propose that it should be up to individual users whether to make use of Modularity or not. In other words, that Modularity shall become fully optional, i.e.:
*modular*.repo
Then the Modularity people can play with the technology as much as they want, without disturbing anybody. (But they will probably find that users will prefer other ways to get alternative software versions, i.e., Copr for leaf packages, parallel-installable compatibility packages for non-leaf packages.)
(Once that is implemented, whether the optional modular content remains on the current infrastructure or is moved to some other infrastructure where it does not interfere with RPM development becomes mostly an implementation detail.)
ban module-only packages: modular versions should only be allowed as alternatives to existing packages,
+1
encourage using parallel-installable compatibility packages rather than modules wherever reasonable (e.g., libraries, language interpreters, …).
keep default streams banned (as they are now provisionally),
disable *modular*.repo by default,
not sure yet, but certainly +1 for making it easy to disable it
(I was wondering whether this should be downgraded from "ban" to "should not". One case is that people who are experimenting with modules. But they should be able to play with modules locally, and there should be no need to put those modules in the user-visible Fedora repos. So I think a clear ban i.e. "must not" is appropriate.)
As with everything in IT, this decision may be reversed at some point, but I don't see that happening soon, and by pretending that the decision is provisional, we aren't doing our users and packagers any favours.
I'm ~0 on this one too. Fedora Modular 32 - x86_64 is 4.9 MB, and insignificant compared to the normal F32 + updates which is together 81MB. When I do some dnf list ... operations, they are .3 seconds (~10%) faster without the modular repos. So unless there's some noticeable difference in other cases, I think we should keep them enabled as long as there are streams available.
Fedora Modular 32 - x86_64
dnf list ...
+1 to what @churchyard said.
Regarding last point: I think we should disable modular repos by default and just have an easy way to enable them so that people who are interested to play with / use them, can easily do so. Probably best way would be going back to "Boltron" (IIRC how that thing was called) and have modular repos in their own fedora-repos subpackage which are enabled by default, but the package is not installed by default.
Separation of modular repos to a separate package was discussed in https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/QMUHN3VF5F56JOEHO4MTVST262II7XXA/#QMUHN3VF5F56JOEHO4MTVST262II7XXA. There seems to be overall agreement that this would be a good thing. How could we drive this forward?
@zbyszek I think we can get an agreement on today's meeting and just do it. I can prepare patches to actually do so.
I just sent out a proposal to cancel today's meeting ;)
If you could prepare the patches, that'd be great. I think we can just as well approve them directly.
https://src.fedoraproject.org/rpms/fedora-repos/pull-request/62
I guess we should approve this somehow. PROPOSAL: approve the general approach in fedora-repos/pull-request/62.
(I wrote "general approach" because there might be some buglets in the PR. I don't think we should review the details of the PR here, but approve the approach of "divide the package, use self-obsoletes to keep both packages on upgrade".)
Yes, this approach looks good to me - on upgrade, the repositories don't change, only the new -modular subpackage is installed, and for "fresh" installations, the -modular subpackage is not pulled in by default.
So +1, if we need a vote :)
-1
@psabata -1 to what proposal exactly? all of them?
@psabata also if you can specify the reason, would be useful.
Yes, pretty much all of them; perhaps with the exception of splitting the repos into a separate package, although I find that change pointless.
This is a recurring topic, usually with the same parties involved. My reasons are the same as always. I don't want to push modularity into obscurity. I'm against disabling the repository by default and mandating that all packages must exist as non-modular as well, with modules only providing alternatives. That's a step back. Even banning defaults was just making the experience worse, even though I understand why it was done. I was in favour as long as it was a temporary measure in effect only until the upgrade experience was going to be fixed.
This is one of our objectives and all these tiny cuts are just killing it instead of improving it.
Stepping back is needed because the steps that were made have caused problems.
Allowing packages to become modular-only does not work out. We have seen non-leaf packages unilaterally converting to modular-only, without any communication with the maintainers of dependent packages. (See, e.g., https://bugzilla.redhat.com/show_bug.cgi?id=1830658 .) And it is also clear that a package becoming modular-only forces all its users, both maintainers of dependent packages and actual end users, to use Modularity, whether they actually want it or not.
Considering all the issues with Modularity, some of which simply cannot be fixed by design (e.g., you will never be able to install all combinations of modules without conflicts because they can depend on conflicting versions of the same module, which are not parallel-installable), forcing Modularity onto users is exactly what is not wanted. I realize that you want to improve Modularity, but these inherent limitations cannot be improved.
After a week:
Already tagged with meeting.
I propose that it should be up to individual users whether to make use of Modularity or not. In other words, that Modularity shall become fully optional, i.e.: ban module-only packages: modular versions should only be allowed as alternatives to existing packages,
How is this needed to make Modularity optional? If there can be both in parallel, having a module does not block someone anyone from also maintaining a non-modular package. The possibility to have packages modular and non-modular at the same time contradicts that having a modular package forces anything as you wrote later:
@till:
How is this needed to make Modularity optional?
Because, if the package I need is only available as a module, how can I choose to not use Modularity?
If there can be both in parallel, having a module does not block someone anyone from also maintaining a non-modular package.
It still forces me to also maintain the dependency in the non-modular repository, because the maintainer unilaterally made his package modular-only without even talking to me.
And if I am not a packager, I am entirely stuck. When I say "make Modularity optional", I am talking about making Modularity optional for users. Because that is what "optional" truly means for me.
The possibility to have packages modular and non-modular at the same time contradicts that having a modular package forces anything
You are seeing this only from the packager's perspective and not from the user's perspective.
(And even for a packager, it is not helpful if the dependencies of their packages can unilaterally be made modular-only.)
I have no idea why I haven't been getting email updates about this ticket, but I just found out about it being revived. I have... quite a bit to say here.
First, I agree with @psabata about the fact that all of these proposed measures seem like they are a deliberate attempt to force Modularity out of the distribution by making it progressively less useful to users. If that is your goal, say so. Otherwise you are being disingenuous. I don't agree with @kkofler on this subject, but at least he's being honest about his intentions.
Admittedly, progress on Modularity issues has indeed been slower than expected. It has not stalled, however. FESCo has approved a Change for Fedora 33 to address the most serious problem with Modules: the lack of an acceptable upgrade path.
The remaining problems described in this thread are pretty much of FESCo's making: issues with things like avocado-vt would not be present if the avocado module was permitted to have a default stream in Fedora. As we now have support for default streams in the Koji buildroot, once the upgrade path issues are resolved, I fail to see what the problem would be with having module-only packages.
avocado-vt
avocado
Regarding parallel-installability compat packages: from the very beginning, we've recommended that if packages can be parallel-installed and it is meaningful to do so, then they should absolutely be made into compat packages. Modularity is intended more for those things that cannot (for one reason or another) be parallel-installed. Things that have to claim a well-known DBUS interface or own a specific path on the filesystem.
To use an example from the past: I'd argue that libgit should be delivered as a parallel-installable RPM, but the bindings for it could be delivered as a module (with a default stream always pointing to the most recent library).
disable modular.repo by default, (+0, 3, -1)
Absolutely not. -1
keep default streams banned (as they are now provisionally), (+3, 0, -1)
-1 I would like to counter-propose that they be allowed once the Change supporting the upgrade path has landed. If you feel there are other blockers to permitting default streams, please state them explicitly in this ticket.
ban module-only packages: modular versions should only be allowed as alternatives to existing packages, (+3, 0, -1)
-1 I very much want to see module-only packages as they are easier to maintain. I look forward to the day that I can build nodejs just as a module and simply select the default stream for each release rather than have to build it as both non-modular (for any release where it's the default) and then modular for "all supported releases".
encourage using parallel-installable compatibility packages rather than modules wherever reasonable (e.g., libraries, language interpreters, …). (+2, 0, -1)
+1 As I said, Modules are not a replacement for compat packages. They are intended for the cases where compat packages are not possible (or are considerably more effort than is worthwhile). If it is meaningful for a package to be parallel-installed, then compat packages should be the preferred option.
package modular repos separately (+3, 1, -0)
0 I think this is entirely dependent on how we decide to proceed on default streams. If we turn them back on, we cannot disable modular repos.
If you feel there are other blockers to permitting default streams, please state them explicitly in this ticket.
A modular policy. As was stated when we approved the ban.
If you feel there are other blockers to permitting default streams, please state them explicitly in this ticket. A modular policy. As was stated when we approved the ban.
Right, that is definitely long overdue. I'll see what I can craft up.
@sgallagh:
Nothing in my proposal is about making Modularity less useful to users. It is all about making the non-modular distribution more useful to users.
Nothing in my proposal prevents users who deliberately opt in to Modularity from getting all its functionality. They also get the drawbacks, such as unresolvable version conflicts (A depends on C version x, B depends on C version y, so A and B cannot coexist), but those are inherent to Modularity and will exist either way. But my proposal means that the drawbacks would only hit those people who have actually opted in to Modularity, not all Fedora users.
(Please note that when I write "users", I mean end users, most of whom are not packagers. I personally consider any other use of the term "users" to be misleading.)
The remaining problems described in this thread are pretty much of FESCo's making: issues with things like avocado-vt would not be present if the avocado module was permitted to have a default stream in Fedora.
They would also not be present if the avocado package had not been allowed to unilaterally become module-only, with no communication with the avocado-vt maintainer.
My proposal for banning default streams has always included also banning module-only packages, which would have completely prevented this kind of issues from happening. The problem happened because only half of the proposal was implemented by FESCo, due to pressure from the pro-Modularity faction.
As we now have support for default streams in the Koji buildroot, once the upgrade path issues are resolved, I fail to see what the problem would be with having module-only packages.
The problem is version conflicts, which are the antithesis to a consistent distribution, and which cannot be solved within Modularity's design. I do not see why I should not be allowed to install 2 unrelated applications A and B on the same release of the same distribution just because they happen to transitively depend on conflicting versions of some library C.
As for the upgrade path issue, the proposed solution can necessarily only address part of the issues. If I want to upgrade from Fedora 42 to Fedora 43, should it upgrade all modules to the version from Fedora 43 or only those where the Fedora 42 version has become obsolete/EOL? The answer will depend on the user, and possibly on the individual module. This is complexity that a non-modular, internally consistent distribution just does not have.
Everything can be parallel-installed given sufficient effort. Conflicting names can be renamed.
Case in point: kdelibs 3 and 4. KDE upstream, at the time, did not support coinstalling the -devel parts in the same prefix, only the runtime parts. I found a solution to allow kdelibs3-devel and kdelibs4-devel to coexist: the library symlinks were moved to %{_libdir}/kde4/devel, some development-only binaries renamed, the CMake files patched to automatically pick up the changed names. We still ship the patched setup. (Thankfully, for KDE Frameworks 5, upstream changed their mind and renamed the libraries completely rather than just bumping the soversion, so we do not need this kind of tricks there.)
kdelibs
-devel
kdelibs3-devel
kdelibs4-devel
%{_libdir}/kde4/devel
I very much want to see module-only packages as they are easier to maintain. I look forward to the day that I can build nodejs just as a module and simply select the default stream for each release rather than have to build it as both non-modular (for any release where it's the default) and then modular for "all supported releases".
That allows you to make a few builds fewer, at the expense of significantly complicating the end user experience. Do we really want to optimize our processes for packagers at all costs, at the expense of users? I do not see avoiding a minor inconvenience for you as a packager as being worth the complexity of "Modules for everyone" for end users (packages hidden by modules, version conflicts, upgrade paths, etc.).
Please also keep in mind that we packagers are also end users and that we typically spend more time being users than actively packaging.
This!
First, I agree with @psabata about the fact that all of these proposed measures seem like they are a deliberate attempt to force Modularity out of the distribution by making it progressively less useful to users. If that is your goal, say so. Nothing in my proposal is about making Modularity less useful to users. It is all about making the non-modular distribution more useful to users. That allows you to make a few builds fewer, at the expense of significantly complicating the end user experience. Do we really want to optimize our processes for packagers at all costs, at the expense of users?
First, I agree with @psabata about the fact that all of these proposed measures seem like they are a deliberate attempt to force Modularity out of the distribution by making it progressively less useful to users. If that is your goal, say so.
That allows you to make a few builds fewer, at the expense of significantly complicating the end user experience. Do we really want to optimize our processes for packagers at all costs, at the expense of users?
I very much agree with @kkofler here. Modularity might have advantages for some power users, but any gains are overwhelmed by the costs borne by both users and other packagers. In particular, it disproportianately increases the maintenance effort for small-scale maintainers who happen to depend on any of the modularized rpms.
Issues with Modularity have been discussed at length, and saying that "remaining problems described in this thread are pretty much of FESCo's making" is completely ignoring reality. Yes, FESCo has asked questions, but the fact that the answers are lacking is not FESCo's fault.
Nevertheless, all the proposals in this ticket are very careful to not impact users of modules negatively. The proposed package split is done in a way that preserves all modular functionality, modules can still be produced and installed, etc, etc. The only thing that those proposal change for users is the ability to opt-out of modules.
encourage using parallel-installable compatibility packages rather than modules wherever reasonable (e.g., libraries, language interpreters, …). (+2, 0, -1) +1 As I said, Modules are not a replacement for compat packages.
+1 As I said, Modules are not a replacement for compat packages.
Thank you.
I'd argue that libgit should be delivered as a parallel-installable RPM, but the bindings for it could be delivered as a module (with a default stream always pointing to the most recent library).
That is a very good example, because it shows that modularization causes cascading problems. Language bindings depend on the underlying library, but then we have other packages on top. If the bindings become modularized, the next step is to have non-coinstallable packages that use those bindings. At that point things become painful for users.
I would like to counter-propose that they be allowed once the Change supporting the upgrade path has landed.
-1 on this. That Change moves things in the right direction, but it should not result in an automatic decision about default streams.
Many problems have been discussed: design problems with modularity, end-user usability issues, bad interactions with non-modular packages, bad reporting from tools, lack of integration into package search, lack of visibility into the build process and source of packages, unclear ownership and bug reporting paths, general complication of packaging, problematic interaction with other downstreams, missing modularity documentation, etc. Just summarizing this would be probably a few days of work. Please don't expect this to be done in this ticket.
@kkofler
keep default streams banned (as they are now provisionally), ban module-only packages: modular versions should only be allowed as alternatives to existing packages,
I think we are stuck in a self contradicting loop here.
And the reason for is the misconception about "module-only" packages.
Packages in a module are regular Fedora packages. This is a defining feature for the Modularity, it is what makes Modularity appealing in comparison with SCL or parallel installability. It is basically the whole point of it.
There is no "modular version" of a package, there is a version of a package packed in a module.
These package versions are standard Fedora versions of the package. For example, i have a module for Fedora 32 with minetest package of version 4.x which I had in Fedora 30. And I haven't made any changes to the package, I only created metadata file for modular repository to point to that old git commit which was used to build package for Fedora 30.
So when you say there shouldn't be modular package without non-modular version of it, that's exactly what default streams are doing. Default stream is exactly that "the non-modular version of a package".
The reason why we have the debate about non-modular vs modular packages is not that we disagree on the goal, but that we haven't ever implemented the Modularity project to achieve this goal.
There are of course reasons for it. Organizational, resource limitations, rollout of a feature would be done differently... But still, Modularity project simply doesn't have fully-functional prototype yet.
Now, it might be that Modularity just can_not be implemented as it is planned. But currently we are blocked from trying and learning that.
We can not deliver the change but we can not move on also, if block ourselves from even moving.
And note that we discuss in this ticket how the Modularity issues are unresolved again and again and how it prevents default streams in Fedora, while the only thing that asked is to try and address and maybe resolve these issues on the side without any impact on the Fedora release process (https://pagure.io/fesco/issue/2390).
In #2390 I honestly fail to understand why would FESCo even consider blocking development of any experimental feature in a sidetag by any Fedora community members.
Yeah.
I think this semantic nitpicking is not very useful. The underlying rpm is obviously very similar, but both the way that this rpm is built by packagers, and how the installation of the rpm is handled on the user systems is quite different.
Modularity project simply doesn't have fully-functional prototype yet.
Yes, I think we can agree on that. But we seem to disagree on how to proceed from here. I don't think that the live Fedora packaging ecosystem should be place where the development of the modular prototype happens. I think that development of alternative versions of software in modules (as opposed to having rpms that are only available through modules), is a reasonable compromise: it allows developers of Modularity to work on it and have real packages for testing, and it allows packagers who want to test Modularity to build such packages. At the same modular packages remain optional, so other packagers don't have to depend on module-only rpms, and users may or may not use modules as they wish.
@churchyard and @dcantrell and I listed various reasons in the ticket. Let's continue that part of the discussion there.
@bookwar:
Sorry, but I fail to see the contradiction…
… and I also do not believe that I am misconceiving anything.
There is still a technical difference between a package that is packed in a module (a modular package), and a package that is standalone, not packed in a module (a non-modular package). They are both built differently and delivered differently.
And the same package from the same specfile (possibly even from the same git hash?) can also be built and delivered simultaneously in the modular and the non-modular way.
No, that is not exactly what default streams are doing. Default streams are technically a completely different approach than the non-modular default versions I am asking for. They aim at delivering the same user experience, but they fall short of that in several ways, which have been pointed out repeatedly on the mailing lists, so I do not think it is necessary to repeat them again in this thread, is it?
A default stream means take all packages from this stream of this module and treat them as the default, overriding any non-modular builds of the same packages. They are still built in the modular way, they can still depend on specific versions of specific other modules (and in particular, cause version conflicts with other modules or replace even more non-modular packages), they are still delivered through modular repositories, etc.
A non-modular default version (what I also call "banning module-only packages") means to take the default version of each package and to build it directly in the respective fNN branch, outside of any module, as if Modularity did not exist. This does not prevent also offering the same version and/or other versions of the package in modules. It just means the user does neither have to explicitly opt in to modules nor to accept being implicitly opted in by default streams.
fNN
No, at least for me, that is not the reason. I disagree with the fundamental concepts of Modularity, even if fully implemented, and I do not agree that users should be required to use modules, neither explicitly nor implicitly.
Now, it might be that Modularity just can_not be implemented as it is planned. But currently we are blocked from trying and learning that. We can not deliver the change but we can not move on also, if block ourselves from even moving.
The default Fedora distribution with thousands to millions of users is not the right place to do experiments ("trying and learning"). I do not see an issue with doing experiments if and only if users are only affected when explicitly opting in to those experiments, which clearly means no default streams and no module-only packages. But if you cannot work under those terms, then you need to do your experiments in an entirely separate repository, outside of Fedora infrastructure.
I think we are stuck in a self contradicting loop here. Yeah. There is no "modular version" of a package, there is a version of a package packed in a module. I think this semantic nitpicking is not very useful. The underlying rpm is obviously very similar, but both the way that this rpm is built by packagers, and how the installation of the rpm is handled on the user systems is quite different. Modularity project simply doesn't have fully-functional prototype yet. Yes, I think we can agree on that. But we seem to disagree on how to proceed from here. I don't think that the live Fedora packaging ecosystem should be place where the development of the modular prototype happens.
Yes, I think we can agree on that. But we seem to disagree on how to proceed from here. I don't think that the live Fedora packaging ecosystem should be place where the development of the modular prototype happens.
What is a "live Fedora packaging ecosystem"?
I think that development of alternative versions of software in modules (as opposed to having rpms that are only available through modules), is a reasonable compromise: it allows developers of Modularity to work on it and have real packages for testing, and it allows packagers who want to test Modularity to build such packages.
It does not. It allows packagers to use parts of the Modularity, which have been developed. It doesn't allow to finish development of a complete Modularity feature.
As I pointed above - possibility to use default module stream on equal grounds with any other package is the cornerstone of the Modularity. To proceed with Modularity feature we need to work specifically on that part, and development of additional modules somewhere else is not relevant for it.
At the same modular packages remain optional, so other packagers don't have to depend on module-only rpms, and users may or may not use modules as they wish. In #2390 I honestly fail to understand why would FESCo even consider blocking development of any experimental feature in a sidetag by any Fedora community members. @churchyard and @dcantrell and I listed various reasons in the ticket. Let's continue that part of the discussion there.
At the same modular packages remain optional, so other packagers don't have to depend on module-only rpms, and users may or may not use modules as they wish.
Why then we reopened the conversation here? Nothing is proposed under this ticket, default streams on Fedora are still not allowed, and there was no request to change that now.
So what are we trying to achieve by this discussion?
@bookwar: I think we are stuck in a self contradicting loop here. Sorry, but I fail to see the contradiction… And the reason for is the misconception about "module-only" packages. … and I also do not believe that I am misconceiving anything. There is no "modular version" of a package, there is a version of a package packed in a module. There is still a technical difference between a package that is packed in a module (a modular package), and a package that is standalone, not packed in a module (a non-modular package). They are both built differently and delivered differently. And the same package from the same specfile (possibly even from the same git hash?) can also be built and delivered simultaneously in the modular and the non-modular way. So when you say there shouldn't be modular package without non-modular version of it, that's exactly what default streams are doing. Default stream is exactly that "the non-modular version of a package". No, that is not exactly what default streams are doing. Default streams are technically a completely different approach than the non-modular default versions I am asking for. They aim at delivering the same user experience, but they fall short of that in several ways, which have been pointed out repeatedly on the mailing lists, so I do not think it is necessary to repeat them again in this thread, is it?
There is still a technical difference between a package that is packed in a module (a modular package), and a package that is standalone, not packed in a module (a non-modular package). They are both built differently and delivered differently. And the same package from the same specfile (possibly even from the same git hash?) can also be built and delivered simultaneously in the modular and the non-modular way.
"aim at delivering the same user experience, but they fall short" That's exactly what I am talking about. The fail short happens because development in this direction is not allowed by FESCo. And FESCo is not allowing development of this feature because it "fails short".
A default stream means take all packages from this stream of this module and treat them as the default, overriding any non-modular builds of the same packages. They are still built in the modular way, they can still depend on specific versions of specific other modules (and in particular, cause version conflicts with other modules or replace even more non-modular packages), they are still delivered through modular repositories, etc. A non-modular default version (what I also call "banning module-only packages") means to take the default version of each package and to build it directly in the respective fNN branch, outside of any module, as if Modularity did not exist. This does not prevent also offering the same version and/or other versions of the package in modules. It just means the user does neither have to explicitly opt in to modules nor to accept being implicitly opted in by default streams. The reason why we have the debate about non-modular vs modular packages is not that we disagree on the goal, but that we haven't ever implemented the Modularity project to achieve this goal. No, at least for me, that is not the reason. I disagree with the fundamental concepts of Modularity, even if fully implemented, and I do not agree that users should be required to use modules, neither explicitly nor implicitly.
A default stream means take all packages from this stream of this module and treat them as the default, overriding any non-modular builds of the same packages. They are still built in the modular way, they can still depend on specific versions of specific other modules (and in particular, cause version conflicts with other modules or replace even more non-modular packages), they are still delivered through modular repositories, etc. A non-modular default version (what I also call "banning module-only packages") means to take the default version of each package and to build it directly in the respective fNN branch, outside of any module, as if Modularity did not exist. This does not prevent also offering the same version and/or other versions of the package in modules. It just means the user does neither have to explicitly opt in to modules nor to accept being implicitly opted in by default streams.
This is not based on technical merit.
You may say: I don't want people to use packages which can not be updated, or packages which can not be rebuilt, or which can not be installed and removed via dnf, or whatever functional argument you find. This functional arguments may or may not be converted to requirements for default streams.
But you can not just say "I don't want users to use packages from modules, because they come from modules". Module is essentially a script which automates packaging process in some cases. I can not forbid you as package maintainer to use automation, which you wrote to create packages. So you shouldn't forbid automation others have written for other packages, as long as the outcome works for the distribution.
Now, it might be that Modularity just can_not be implemented as it is planned. But currently we are blocked from trying and learning that. We can not deliver the change but we can not move on also, if block ourselves from even moving. The default Fedora distribution with thousands to millions of users is not the right place to do experiments ("trying and learning").
The default Fedora distribution with thousands to millions of users is not the right place to do experiments ("trying and learning").
Fedora has its users because we experiment, try and learn. We should be careful when doing so, of course. That's why we looking for non-disruptive ways to do this.
I do not see an issue with doing experiments if and only if users are only affected when explicitly opting in to those experiments, which clearly means no default streams and no module-only packages.
Again, there is no proposal yet to enable default streams in Fedora. Why we are arguing about it here? With whom are you arguing?
But if you cannot work under those terms, then you need to do your experiments in an entirely separate repository, outside of Fedora infrastructure.
Why then we reopened the conversation here? Nothing is proposed under this ticket, default streams on Fedora are still not allowed, and there was no request to change that now. So what are we trying to achieve by this discussion?
A modular policy for Fedora. Default streams were turned off due to unexpected impact on users and developers and without a clear policy on the scope of modules and what it means to make a module in Fedora, I feel we still need to keep them off.
Specifically what I want answered is: - What will Fedora allow as modules? It obviously can't be a free-for-all as we have seen before where we let anything end up in a module and then that affects installation for other users. - A selling point I have heard of modules is that you disconnect your maintenance of packages from the Fedora release train. OK, fine, but what does that mean for policy around maintenance of modules? What we have seen so far is modules land and rot. Should module maintainers be expected to do the same work as package maintainers? - What does it look like for users? Can they still easily find and install stuff? As a developer, I can't, so I can't imagine it's very clear and not confusing for other users.
And then more specifically: - If I maintain package FOO and a BuildRequire that I need moves in to a module, can I still build my package without becoming a module? The answer here should be yes and should not involve FOO also becoming a module. The policy should indicate how we do this. - Will modules hide subpackages? In RHEL we're using modules to hide subpackages we don't want to support for end users. Fine. But in Fedora that would be super annoying, so I want to know that I can still install BAR-devel even if the module it lives in is hiding it from users that install the module.
As stated in previous comments, a functional prototype still needs to exist as well to answer a lot of these policy questions. I don't think this should happen in the live Fedora packaging system (which I take to mean koji.fedoraproject.org and what builds packages when I type "fedpkg build").
Why then we reopened the conversation here? Nothing is proposed under this ticket, default streams on Fedora are still not allowed, and there was no request to change that now. So what are we trying to achieve by this discussion? A modular policy for Fedora. Default streams were turned off due to unexpected impact on users and developers and without a clear policy on the scope of modules and what it means to make a module in Fedora, I feel we still need to keep them off.
A modular policy for Fedora. Default streams were turned off due to unexpected impact on users and developers and without a clear policy on the scope of modules and what it means to make a module in Fedora,
I feel we still need to keep them off.
There is no argument about that. No one has proposed the change to the decision FESCo made on this topic.
So I still don't understand why we are discussing this.
Specifically what I want answered is: - What will Fedora allow as modules? It obviously can't be a free-for-all as we have seen before where we let anything end up in a module and then that affects installation for other users. - A selling point I have heard of modules is that you disconnect your maintenance of packages from the Fedora release train. OK, fine, but what does that mean for policy around maintenance of modules? What we have seen so far is modules land and rot. Should module maintainers be expected to do the same work as package maintainers? - What does it look like for users? Can they still easily find and install stuff? As a developer, I can't, so I can't imagine it's very clear and not confusing for other users. And then more specifically: - If I maintain package FOO and a BuildRequire that I need moves in to a module, can I still build my package without becoming a module? The answer here should be yes and should not involve FOO also becoming a module. The policy should indicate how we do this. - Will modules hide subpackages? In RHEL we're using modules to hide subpackages we don't want to support for end users. Fine. But in Fedora that would be super annoying, so I want to know that I can still install BAR-devel even if the module it lives in is hiding it from users that install the module.
Why FESCo needs to decide it now?
Why FESCO runs a conversation on the policy for the feature, which is not yet complete and not proposed to FESCo by feature developers?
Do you consider ELN part of the "live" system? If yes, why?
Why then we reopened the conversation here? Nothing is proposed under this ticket, default streams on Fedora are still not allowed, and there was no request to change that now. So what are we trying to achieve by this discussion? A modular policy for Fedora. Default streams were turned off due to unexpected impact on users and developers and without a clear policy on the scope of modules and what it means to make a module in Fedora, I feel we still need to keep them off. There is no argument about that. No one has proposed the change to the decision FESCo made on this topic.
Maybe I misunderstood, but I wanted resolution on this ticket based on @sgallagh's request to enable default module streams for ELN. Did I misunderstand that?
Modularity needs a scope defined and documented for Fedora. It was tried without that and had negative impacts on users. I don't want to repeat that. It is not unreasonable to ask that the scope of modularity for Fedora be defined, documented, and agreed upon.
Specifically what I want answered is: - What will Fedora allow as modules? It obviously can't be a free-for-all as we have seen before where we let anything end up in a module and then that affects installation for other users. - A selling point I have heard of modules is that you disconnect your maintenance of packages from the Fedora release train. OK, fine, but what does that mean for policy around maintenance of modules? What we have seen so far is modules land and rot. Should module maintainers be expected to do the same work as package maintainers? - What does it look like for users? Can they still easily find and install stuff? As a developer, I can't, so I can't imagine it's very clear and not confusing for other users. And then more specifically: - If I maintain package FOO and a BuildRequire that I need moves in to a module, can I still build my package without becoming a module? The answer here should be yes and should not involve FOO also becoming a module. The policy should indicate how we do this. - Will modules hide subpackages? In RHEL we're using modules to hide subpackages we don't want to support for end users. Fine. But in Fedora that would be super annoying, so I want to know that I can still install BAR-devel even if the module it lives in is hiding it from users that install the module. Why FESCo needs to decide it now?
Because it has not happened yet and there is a proposal now to enable modules for ELN. Enabling aspects of modularity now before we have agreed on its scope in Fedora feels like a cart before the horse situation.
Because they are related and dependent.
As stated in previous comments, a functional prototype still needs to exist as well to answer a lot of these policy questions. I don't think this should happen in the live Fedora packaging system (which I take to mean koji.fedoraproject.org and what builds packages when I type "fedpkg build"). Do you consider ELN part of the "live" system? If yes, why?
Yes. ELN builds in Koji in Fedora. You've defined it as a subset build of rawhide packages built with different %{optflags} (perhaps some other macros, but that's the basics as I understand it). In the past we would have called this rhel-rawhide. In fact we did and we pushed it off to the side and pretended it could be developed in a silo and everyone would benefit, but that didn't work. For ELN to work the ELN SIG needs to be able to easily collaborate with Fedora package maintainers to either co-maintain or integrate PRs for ELN builds on an ongoing basis. If we're not considering ELN part of the live Fedora build system and in effect a second type of rawhide, then that will lead to problems. As a Fedora package maintainer I like that ELN will allow me to be a more effective upstream for RHEL and CentOS. From a technical standpoint, it's just another build target like rawhide.
Why then we reopened the conversation here? Nothing is proposed under this ticket, default streams on Fedora are still not allowed, and there was no request to change that now. So what are we trying to achieve by this discussion? A modular policy for Fedora. Default streams were turned off due to unexpected impact on users and developers and without a clear policy on the scope of modules and what it means to make a module in Fedora, I feel we still need to keep them off. There is no argument about that. No one has proposed the change to the decision FESCo made on this topic. Maybe I misunderstood, but I wanted resolution on this ticket based on @sgallagh's request to enable default module streams for ELN. Did I misunderstand that?
I think we have a disagreement here, which causes the overall misunderstanding.
Stephen has proposed the policy specifically for the ELN buidroot. You have extended this proposal to the entire Fedora buildroot, and we ended up discussing Fedora policy for Modularity.
My point is - this is not what Stephen has proposed. We are discussing a straw man, not the actual thing.
Thus before we dive into Fedora Modularity topic, I would like us to go back to the question, whether or not ELN buildroot configuration should be linked to Rawhide buildroot configuration. We need to get to agreement on that topic first. And then if we come to agreement there, and if agreement would be "yes", then we would need a new explicit proposal to discuss.
And I am personally against the scope creep there. See below.
So I still don't understand why we are discussing this. Modularity needs a scope defined and documented for Fedora. It was tried without that and had negative impacts on users. I don't want to repeat that. It is not unreasonable to ask that the scope of modularity for Fedora be defined, documented, and agreed upon. Specifically what I want answered is: - What will Fedora allow as modules? It obviously can't be a free-for-all as we have seen before where we let anything end up in a module and then that affects installation for other users. - A selling point I have heard of modules is that you disconnect your maintenance of packages from the Fedora release train. OK, fine, but what does that mean for policy around maintenance of modules? What we have seen so far is modules land and rot. Should module maintainers be expected to do the same work as package maintainers? - What does it look like for users? Can they still easily find and install stuff? As a developer, I can't, so I can't imagine it's very clear and not confusing for other users. And then more specifically: - If I maintain package FOO and a BuildRequire that I need moves in to a module, can I still build my package without becoming a module? The answer here should be yes and should not involve FOO also becoming a module. The policy should indicate how we do this. - Will modules hide subpackages? In RHEL we're using modules to hide subpackages we don't want to support for end users. Fine. But in Fedora that would be super annoying, so I want to know that I can still install BAR-devel even if the module it lives in is hiding it from users that install the module. Why FESCo needs to decide it now? Because it has not happened yet and there is a proposal now to enable modules for ELN. Enabling aspects of modularity now before we have agreed on its scope in Fedora feels like a cart before the horse situation. Why FESCO runs a conversation on the policy for the feature, which is not yet complete and not proposed to FESCo by feature developers? Because they are related and dependent. As stated in previous comments, a functional prototype still needs to exist as well to answer a lot of these policy questions. I don't think this should happen in the live Fedora packaging system (which I take to mean koji.fedoraproject.org and what builds packages when I type "fedpkg build"). Do you consider ELN part of the "live" system? If yes, why? Yes. ELN builds in Koji in Fedora. You've defined it as a subset build of rawhide packages built with different %{optflags} (perhaps some other macros, but that's the basics as I understand it).
Yes. ELN builds in Koji in Fedora. You've defined it as a subset build of rawhide packages built with different %{optflags} (perhaps some other macros, but that's the basics as I understand it).
We defined ELN as experimental playground to build Fedora content in EL-like way.
"ELN is a new buildroot and compose process for Fedora that will take Fedora Rawhide dist-git sources and emulate a Red Hat Enterprise Linux compose. "
In the past we would have called this rhel-rawhide. In fact we did and we pushed it off to the side and pretended it could be developed in a silo and everyone would benefit, but that didn't work. For ELN to work the ELN SIG needs to be able to easily collaborate with Fedora package maintainers to either co-maintain or integrate PRs for ELN builds on an ongoing basis. If we're not considering ELN part of the live Fedora build system and in effect a second type of rawhide, then that will lead to problems. As a Fedora package maintainer I like that ELN will allow me to be a more effective upstream for RHEL and CentOS. From a technical standpoint, it's just another build target like rawhide.
The defining feature for ELN is to use Fedora content, but allow changes to buildroot and compose process, so that we can experiment on the building and composing without diverging from the sources.
Modules in the buildroot is the property of buildroot, not packages. Same as metadata in the repositories is the property of compose and pungi config.
The same way as we can change comps file for ELN compose, we can change default module streams configuration, without impact on the sources.
This makes it possible to use ELN buildroot for Fedora Modularity.
It may break ELN compose, and probably will, but then it is for the better. We will hit the same issues as EPEL and CentOS and RHEL will hit. Which may have impact on RHEL decisions regarding modules too.
The one concern I find valid is: allowing modules for ELN essentially means allowing branching. We fought hard to ensure that ELN won't be branched from Rawhide, and now we kind of let it back in. So it feels like we have switched places in that conversation.
But I suggest to discuss it further there in #2390
@zbyszek: Yes, I think we can agree on that. But we seem to disagree on how to proceed from here. I don't think that the live Fedora packaging ecosystem should be place where the development of the modular prototype happens. What is a "live Fedora packaging ecosystem"?
@zbyszek:
The packages we deliver to our end users, who have not opted in to becoming alpha-testers.
I think that development of alternative versions of software in modules (as opposed to having rpms that are only available through modules), is a reasonable compromise: it allows developers of Modularity to work on it and have real packages for testing, and it allows packagers who want to test Modularity to build such packages. It does not. It allows packagers to use parts of the Modularity, which have been developed. It doesn't allow to finish development of a complete Modularity feature. As I pointed above - possibility to use default module stream on equal grounds with any other package is the cornerstone of the Modularity. To proceed with Modularity feature we need to work specifically on that part, and development of additional modules somewhere else is not relevant for it.
It does not. It allows packagers to use parts of the Modularity, which have been developed. It doesn't allow to finish development of a complete Modularity feature. As I pointed above - possibility to use default module stream on equal grounds with any other package is the cornerstone of the Modularity. To proceed with Modularity feature we need to work specifically on that part, and development of additional modules somewhere else is not relevant for it.
I do not see why default streams are an essential feature for Modularity, let alone the essential feature as you claim here.
The primary focus ought to be on the end user experience, not the packager experience. And end users will not miss any feature of Modularity by not having default streams, at least as long as a non-modular default build of the package is delivered (as both @zbyszek and me want to require). They can still opt to replace the non-modular version by any version of the module, just as they would be able to with a default stream. They just will not have to deal with the complexity of modules and default streams if they do not want to.
First of all, FESCo has not decided to ban improving the functionality of default streams. FESCo has only decided to not allow default streams at this time, given their current state of implementation. Surely there ought to be a way to work on the implementation in a test repository, not in the repository shipped to thousands to millions of end users.
And secondly, you make the implied assumption that the points in which modular default streams fall short of delivering the same user experience as non-modular default versions can be fixed at all. I do not think that this is possible at all given the core design concepts. In particular, the potential that the default stream of module A and the default stream of module B start depending directly or indirectly on conflicting versions of some module C. Fixing this would require restricting module use so much that I do not see what possible gain would be left over just building the default versions as non-modular packages (which leads to a much simpler user experience).
So I am afraid I do not see the vicious circle that you are trying to paint here.
@kkofler: I disagree with the fundamental concepts of Modularity, even if fully implemented, and I do not agree that users should be required to use modules, neither explicitly nor implicitly. This is not based on technical merit. You may say: I don't want people to use packages which can not be updated, or packages which can not be rebuilt, or which can not be installed and removed via dnf, or whatever functional argument you find. This functional arguments may or may not be converted to requirements for default streams. But you can not just say "I don't want users to use packages from modules, because they come from modules". Module is essentially a script which automates packaging process in some cases. I can not forbid you as package maintainer to use automation, which you wrote to create packages. So you shouldn't forbid automation others have written for other packages, as long as the outcome works for the distribution.
@kkofler:
I disagree with the fundamental concepts of Modularity, even if fully implemented, and I do not agree that users should be required to use modules, neither explicitly nor implicitly.
This is not based on technical merit. You may say: I don't want people to use packages which can not be updated, or packages which can not be rebuilt, or which can not be installed and removed via dnf, or whatever functional argument you find. This functional arguments may or may not be converted to requirements for default streams. But you can not just say "I don't want users to use packages from modules, because they come from modules". Module is essentially a script which automates packaging process in some cases. I can not forbid you as package maintainer to use automation, which you wrote to create packages. So you shouldn't forbid automation others have written for other packages, as long as the outcome works for the distribution.
As explained above: My main functional argument is: I do not want users to be unable to install two unrelated applications A and B due to them requiring (directly or indirectly) conflicting versions of some module C.
You can certainly translate that requirement to a requirement on default streams, but at that point, why do you even need to use default streams at all? If you have to agree on a default version of C anyway, you can just build the default versions of C, A, and B all in the non-modular repository, eliminating all the unnecessary complexity in tools such as DNF and in the resulting user experience that default streams entail.
(There are other functional arguments against default streams in their current state, which may or may not be fixable, but the above is the crucial one, IMHO.)
As for the "modules are just automation" argument, that would be true if we were using something such as Ursa Major on the end-user-facing repositories, putting the packages from the default streams directly in the non-modular repository and making them look exactly like non-modular packages (an idea which, last I checked, was not favored by the Modularity team). But as things are now, users will notice the difference, and it leads to a worse and more complex user experience. So I do not consider it unreasonable to not allow this form of "automation".
In addition, your overgeneralized claim that "I can not forbid you as package maintainer to use automation, which you wrote to create packages." is also not true in several other cases, e.g., it is not allowed to build official Fedora packages on upstream (or third-party) infrastructure (so automating it directly through upstream CI is a form of automation that would not be allowed), it is not allowed to download things from the Internet during build, etc. There are (and rightfully so) restrictions on how a Fedora package is created, for reasons such as security and, yes, user experience. The latter being the reason I am opposed to default streams.
Fedora has its users because we experiment, try and learn.
I strongly disagree that Fedora is supposed to be a community of alpha testers or guinea pigs for experimental functionality.
There are places in Fedora for experimenting, trying and learning. The production repositories (GA and updates) of a stable release are not such a place.
Being an alpha version of RHEL is an explicit non-goal of Fedora.
We should be careful when doing so, of course. That's why we looking for non-disruptive ways to do this.
Requiring an explicit opt-in and ensuring that users get a fully working distribution if they do not opt in is the way to do that. And that is where I want Modularity to get.
But the ban on default streams is completely worthless as long as it is allowed to remove the non-modular default version and hence just not declare a default version at all. That means the user cannot use the package without explicitly opting in to the module, which makes the whole opt in a farce (and is, in fact, an even worse user experience than with the default stream). And the situation is even worse for dependent packages, which are forced to become module-only as well if their dependencies can unilaterally become module-only, as we have seen in real life with avocado.
I think we actually agree that the current situation of allowing module-only packages, but not allowing them to declare a default stream is broken. We just disagree about the way out of this mess. But since the current situation is broken, I think we cannot actually dissociate the discussions on default streams and on module-only packages. Banning default streams necessarily requires also banning module-only packages to get a reasonable and consistent user and packager experience.
Looking at all the open questions with ELN that are coming up in this thread and in #2390, I stay reaffirmed in my position that approving ELN was a mistake.
PS: I also think that it would be extremely hard to enforce the non-conflicting requirement (any two default streams requiring, even transitively, a common module M must agree on the version of M to use) for default streams if we wanted to go down that road, whereas requiring the default version of every package to be non-modular automatically enforces this requirement without even having to do anything.
PPS: The current situation with module-only packages reminds me of those few sites that interpret the GDPR cookie rules in such a way that you actually get locked out completely from the site if you do not "opt in" to cookies.
This was discussed in today's FESCo meeting and we (informally) decided to wait for more info: @sgallagh will work on a policy document, the dnf team is preparing a statement for the council meeting on the 17th (?).
The Council meeting will be 10 June. Details are on the wiki and everyone is welcome to attend.
Unlike what @bookwar wrote in the meeting, the most reliable way to work on Modularity without breaking things is not default streams, because default streams replace the non-modular packages and hence can definitely break things that were previously working. (In fact, several of the issues encountered with Modularity in Fedora were due to default streams, which is why they have been dropped.) The most reliable way to work on Modularity without breaking things is actually a strict "no default streams, no module-only packages" policy. Just maintain your modular version of the package in parallel to the non-modular version and you can experiment as much as you wish in the modular version without possibly breaking anything for the users of the non-modular package. (Yes, it is more work. But it is much safer.)
Similarly, it is not true that "default stream is the feature which removes the viral nature of Modularity" (@bookwar). Quite the opposite, default streams force Modularity onto users, making it very much viral. They eliminate one form of "viralness" (by allowing a package to become module-only without requiring the dependent packages to become module-only as well), but introduce another one (where installing even a non-modular package can force modules onto users' systems).
First of all, let me share the short-term and mid-term plans the Red Hat's Modularity team has. We have a commitment to keep the current implementation of Modularity up and running because it's part of RHEL, CentOS, and EPEL. There are only few people working on it. Our focus is maintenance, but it includes several design changes to fix the most annoying issues such as EOL/Obsoletes, module context upgrade paths, package de-modularization, and creating tools for building and managing modules locally (without dist-git, MBS or anything special).
This should also help us to fix some problems Fedora users have when they upgrade their systems from one version of Fedora to another.
This all is from a technical perspective. We're fixing modularity itself. How Fedora should use modularity? That's a completely different question. Modularity works fine in a relatively small enterprise distro, which doesn't move that fast. The same doesn't seem to work in Fedora. @kkofler is correct about Modularity being too disruptive when some packages are only part of default module streams and non-modular packages depend on them. Switching the default stream to another breaks the distro. From this perspective, it's truly a form of viral behavior when the only option how to fix such non-modular packages is to move them into modules and build them in a number of contexts determined by the number of streams someone else created.
So back to the "How Fedora should use modularity?" question. Modularity team doesn't want to push Fedora to anything. We believe the overall direction should be set by FESCO and individual spin and package maintainers.
We'd like to keep Modularity as an option, so it's used where it makes sense. For example DNF team works on a new upstream version of dnf/libdnf and that will require integration with a lot of other tools based on DNF. Providing such content as a module stream for early adopters/developers and merging it into Rawhide when it's ready makes perfect sense to me.
I hope we'll be able to share some details on the long-term plans in the Jun 10 Council meeting, but we're still in the middle of discussions with other teams at Red Hat. When we have a conclusion, we'll discuss details with Fedora community. Just to make it clear - we're not making any decisions behind the closed door, we're just trying to keep the internal overhead internal.
Thanks @dmach for taking time and posting a comment here.
We have a commitment to keep the current implementation of Modularity up and running because it's part of RHEL, CentOS, and EPEL. There are only few people working on it. Our focus is maintenance, but it includes several design changes to fix the most annoying issues such as EOL/Obsoletes, module context upgrade paths, package de-modularization, and creating tools for building and managing modules locally (without dist-git, MBS or anything special).
If I understand this paragraph correctly: in short, the modularity is not going to be developed (in the sense of new features) by the Red Hat Modularity team, only keeping it alive because of RHEL.
Thanks, this is much appreciated.
We'd like to keep Modularity as an option, so it's used where it makes sense.
So essentially I read this that we do not need default streams. Neither in Fedora nor in ELN because the technology is in the maintenance mode, so we need to start looking for a replacement (or, rather, for a solution that solves problems both for Fedora and RHEL).
Providing such content as a module stream for early adopters/developers and merging it into Rawhide when it's ready makes perfect sense to me.
Sure, as long as somebody is supporting this technology, I have nothing against this. Alternative versions can go to the modules.
I said it's not going to be developed now because the priority is RHEL. I did not say anything about the future. Software development frequently runs in cycles: planning, development, maintenance. Now we're dealing with the maintenance of the current modularity implementation.
Maintenance mode doesn't have anything to do if the default streams should be used or not. It's a completely unrelated problem. My feeling is that Fedora doesn't want the default streams and ELN "spin" wants default streams. It should be their decision to chose what fits their needs the best.
You wrote "we need to start looking for a replacement" - could you elaborate more on who is "we", and why we need the replacement and what are the goals you want to accomplish?
Thanks @dmach for the update.
Our focus is maintenance, but it includes several design changes to fix the most annoying issues such as EOL/Obsoletes, module context upgrade paths, package de-modularization, and creating tools for building and managing modules locally (without dist-git, MBS or anything special).
Sounds reasonable.
@kkofler is correct about Modularity being too disruptive when some packages are only part of default module streams and non-modular packages depend on them. Switching the default stream to another breaks the distro.
This is a good answer to bookwar's statement from the meeting yesterday: bookwar> zbyszek: again, default stream is the feature which removes the viral nature of Modularity, that very "viral" thing which is the key problem of modularity as technology
It is true that having default streams removes a problem (dependent packages don't need to care about modularization of dependencies), but it adds a more significant problem listed above. And the added problem is caused by a fundamental design of Modularity so it's not going away.
For example DNF team works on a new upstream version of dnf/libdnf and that will require integration with a lot of other tools based on DNF. Providing such content as a module stream for early adopters/developers and merging it into Rawhide when it's ready makes perfect sense to me.
True. That would be a valid use case. OTOH, dnf team currently provides the dnf-nightly Copr repo which serves a similar purpose. I have that repo enabled in various places, and it works great. I enabled it at some point to test some upcoming dnf feature, and have mostly forgotten it is there. Apart from a brief conflict when python versions were being switched it just works. And when the conflict was present, I was able to use some simple commandline options to tell dnf to select a different version of itself. Thus, my feeling would be that a Copr repo would serve the above use case just as well or even better. I don't think we should be developing a special tool to handle a case which is handled by another tool that we already have and that solves many other cases too.
We believe the overall direction should be set by FESCO and individual spin and package maintainers.
I disagree. What we have learned in the last two years that this is clearly not something that can be changed in parts of the distro in isolation. I very much believe that we need one choice for the whole distro. And for technical distro-wide decisions FESCo (with input from the whole community and all stakeholders) is the right venue.
The approach of taking this decision at the level of individual package maintainers is what we tried in the past and which clearly doesn't work for maintainers of dependent packages. The idea of making this decision at spin/edition/whatever level is new, but it won't work either. Spins/editions/... are about taking packages out of the common pool and layering a choice of package sets and configuration defaults on top. We don't build packages in different versions for different spins. This is a great strength: work that improves each spin improves the common set of packages and vice versa. It is even possible to transition from one spin to another by dnf, and it is possible to build hybrids, or pluck individual packages. Having some spins switch over to modules would turn them into incompatible fiefdoms.
bcotton has revived the process of verifying that spins are alive. And one of the outcomes is that even keeping spins building over multiple fedora releases is hard. Turning on modules by default in some spins would only exacerbate this issue.
I understand Igor's comment this way: "we" — the community of Fedora, why we need the replacement: the original goals of Modularity.
You wrote "we need to start looking for a replacement" - could you elaborate more on who is "we", and why we need the replacement and what are the goals you want to accomplish? I understand Igor's comment this way: "we" — the community of Fedora, why we need the replacement: the original goals of Modularity.
To me, this basically boils down to: If we declare modularity a dead project, we either admit that we don't need it at all, or we need to find a replacement.
A couple of years ago, we adopted this mission statement:
Fedora creates an innovative platform for hardware, clouds, and containers that enables software developers and community members to build tailored solutions for their users.
Implementation details aside, modularity is specifically designed to enable this, so that developers of Fedora solutions like Spins (or Labs, or Editions, or whatever we call them) can better cater to their audiences.
Choosing different defaults for these outputs is definitely to be desired. For example, a spin designed for classroom use might want to follow a specific version of a programming language over the course of an academic year, or even over the course of several years until there is a curriculum update.
It certainly sounds quite dire for these to become "individual fiefdoms", but I don't think that really follows. All of the regular incentives to collaborate and share work remain. If we don't have a solution, that doesn't mean the user problem goes away. So, people instead standardize on version of the Fedora OS beyond EOL — or choose to use something other than Fedora entirely. This is still "taking packages out of the common pool and layering a choice of package sets and configuration defaults on top" — it's just that we can expand the pool in new ways to meet needs that we're not currently.
I support the decision to disable default modules, because the upgrade problems and other issues were causing real user problems. That was the right call. But I do want the capabilities that this was intended to provide, and I'd like to see those problems worked out — and we should provide a space for that.
@dmach, I feel like "when some packages are only part of default module streams and non-modular packages depend on them. Switching the default stream to another breaks the distro" is a bit over-dramatic.
Rather: if you install packages A, B, and C that depend on a default stream D.1 , and you want to switch to a non-default stream D.2 that isn't compatible with those packages, you can't.
This is the same result as as "the non-default stream D.2 is not available". However now, if you don't need those packages A, B, or C, you actually have an option you didn't have before.
Sure, the packagers of A, B, or C could provide extra functionality to users of D.2 by making modular versions A.2, B.2, C.2 built against D.2, but "you can provide more functionality" doesn't really seem to merit the epithet "viral".
@mattdm
I think we need to define how deep we want to go here, whether it is just programming language + its own package manager or we are going to provide all packages for that programming language (be that python modules, ruby gems, rust crates or something else). The other thing we need to decide whether those should be parallel-installable or not. If so, how does it play with my previous sentence? To be more specific here, rubygems and crates do not "depend" on a compiler / interpreter version. So they are pure noarch packages that can be used with different versions of a compiler / interpreter. But this is not true for python packages due to the bytecode and paths on the file system. I don't think there is one good solution to all ecosystems, so we need to concentrate on some specific things.
I guess what it boils down to is that we need a specific set of use-cases we would like to cover because the mission statement is too generic. Probably post such list to the devel@ and let's see how we can achieve those goals?
Rather: if you install packages A, B, and C that depend on a default stream D.1 , and you want to switch to a non-default stream D.2 that isn't compatible with those packages, you can't. This is the same result as as "the non-default stream D.2 is not available". However now, if you don't need those packages A, B, or C, you actually have an option you didn't have before.
This is not fully correct. We have two choices here:
This all is doable with plain RPMs, without any modules and without any headache. Moreover, with modules option №2 is awkward and non-intuitive for maintainers and users (see Python modules in RHEL8).
@mattdm:
Implementation details aside, modularity is specifically designed to enable this, so that developers of Fedora solutions like Spins (or Labs, or Editions, or whatever we call them) can better cater to their audiences. Choosing different defaults for these outputs is definitely to be desired.
I am worried that this is a very dangerous slippery slope to engage on. What I would really like to avoid is to end up in a situation where we have a GNOME/Workstation module and a KDE Plasma module depending on conflicting library modules, leading to a situation where you cannot install GNOME and KDE Plasma on the same Fedora installation. Such a situation would be completely unacceptable for multi-user machines.
I remember the conflicts over the NetworkManager version that we had for the Fedora 15 release, where GNOME Shell required the (then still unreleased) NetworkManager 0.9, whereas KDE's plasma-nm still only supported NetworkManager 0.8. The solution that was deployed, with the chimera NetworkManager attempting to support both APIs, was very buggy and eventually got replaced in updates, but at least it ensured that both desktops remain parallel installable. Had we had per-spin modules as you propose, the solution that would likely have been deployed would probably have been to let the GNOME/Workstation module bundle or depend on NetworkManager 0.9 and the KDE Plasma module bundle or depend on NetworkManager 0.8, which would have made it impossible for both desktops to coexist on the same installation.
For example, a spin designed for classroom use might want to follow a specific version of a programming language over the course of an academic year, or even over the course of several years until there is a curriculum update.
In this special case, it sounds permissible, but only if the packages depending on that alternate version N of the programming language L are themselves alternate versions and not module-only packages available only in the L-N module or in a module depending on L-N. Instead, there ought to be, for each of those packages, a non-modular default version that is compatible with the main Fedora distribution. Otherwise, those packages are not useful for users not bound by that specific curriculum.
@dmach, I feel like "when some packages are only part of default module streams and non-modular packages depend on them. Switching the default stream to another breaks the distro" is a bit over-dramatic. Rather: if you install packages A, B, and C that depend on a default stream D.1 , and you want to switch to a non-default stream D.2 that isn't compatible with those packages, you can't.
That was not the argument. The argument was: what happens if the default stream in the distro changes to D.2, and most packages in the distro switch to requiring D.2 instead of D.1? Then A, B, and C are suddenly broken.
For that, we do not need a default stream at all. There needs to be a non-modular default version D, and non-modular versions of A, B, and C, which must be compatible with that non-modular default version D. (Or if that is not possible, there must be a compatibility package D1, or a forward-compatibility package D2, which is parallel-installable with D, instead of or in addition to the D.1 or D.2 module.) That default version of D can happen to be identical to the version shipped in the module stream D.1 or in the module stream D.2, but it should be built against and shipped in the non-modular repository. Module streams D.1 and D.2 can be then be offered in addition to the non-modular D (unless Modularity gets replaced with something different entirely).
The main goal here is to prevent conflicts that prevent the packages from coexisting to begin with. Providing maximum functionality by picking the optimal library versions is an additional goal, but it cannot come at the expense of coexistence with other packages in the distribution. Otherwise, what do we need a distribution for to begin with?
And I realize that for maintainers used to modules and default streams (or even module-only packages with no default streams), it may sound like pointless work to also build non-modular default versions of their packages, but it is by far the most efficient way to ensure that there are no dependency version conflicts among default package versions. Trying to enforce that on default streams would be extremely hard due to transitive module dependencies, and if a conflict is found, it would be very painful to coordinate a solution.
And in addition, shipping the default versions in a non-modular repository also leads to a simpler and slicker user experience, at least for the users who stick to those defaults.
Nothing what was proposed by @kkofler goes against what @mattdm said. We can have benefits from modularity (for e.g. the curriculum example) while keeping it fully optional.
Yes, exactly.
Multireply!
@ignatenkobrain SCLs showed that building stacks with non-conflicting paths is actually pretty painful. It was a conscious decision to take a different approach this time around. However, that doesn't mean we're stuck with it. We could certainly have a future version of modularity which also addresses the parallel-install case.
The NetworkManager example Kevin brings up later highlights that this isn't just a "can they both live on disk together?" issue — in many real-world cases that's just the beginning.
The argument was: what happens if the default stream in the distro changes to D.2, and most packages in the distro switch to requiring D.2 instead of D.1? Then A, B, and C are suddenly broken.
This is no different from a big stack update in non-modular Fedora. It needs to be coordinated via a Change and stakeholders (like the packagers of A, B, C) engaged.
I don't think co-installation is the only reason — or even the main reason — for a distribution. The "Everything" option was removed in from Anaconda in 2005 or so. I think it is a good goal for things to be able to co-exist on the same system, but I'd prefer for them to be available but not parallel-installable than not available at all.
If it had happened to be the other way around where the shared dependency needed to be newer for KDE, but GNOME wanted to stick on an old version, and there was no dual-api solution this time around, I don't think we would have wanted to make KDE wait whatever number of releases for the other desktop to catch up. In any case, we can't really force it. This leads to KDE Neon and "GNOME OS", where the desktop environment developers build a whole distro that they control to avoid these conflicts. I think it's better to find a way to provide options and work from there towards coexistence.
Further @kkofler: I agree that the problem you mention about dependency conflicts in the defaults is a real one.
One of the key things about ELN is that it's not supposed to be a separate fork. This means that even if default modules are enabled in ELN, there will also need to be non-module builds for Rawhide.
In turn, that means that the people who are working on ELN will need to make sure to do that extra step, and I think it's fine for that to be policy.
I'd actually like to see an idea from the pre-RHEL-8 modularity plan looked at again, which is tagging all default stream packages into the non modular repo even if they're built with the module build tools — and gating on whether that succeeds. If something like that works, people for find the module tooling more convenient wouldn't have to do the double work.
(As I understand it, this wasn't done exactly because of the possibility of conflicts in the defaults. But if we say that there can't be by policy... tada?)
But things like that need a place to be worked on, and I think it's absolutely better for that to be a Fedora space than a side thing, because a side thing is more likely to diverge than converge back again.
Technical details have been hashed and rehashed many times, so I'll respond briefly with a more general comment: Modularity has significant costs and complications (as listed by @kkofler above and others in many other places), but because of design and implementation choices it offers surprisingly little functionality. I don't see those features as even close to outweighing the costs. For me, to support modularization of more stuff in Fedora, three things would need to happen: modularization wouldn't conflict with the work of other packagers, there would be some significant advantages over other tools, and finally the documentation and usability would improve significantly. Any arguments that boil down to being able to do something with modules almost as well as with a bunch of compat packages are not at all convincing.
@ignatenkobrain wrote above:
we need a specific set of use-cases we would like to cover because the mission statement is too generic. Probably post such list to the devel@ and let's see how we can achieve those goals?
I strongly support this. Let's look at some specific problems with an open mind, considering all possible solutions.
@zbyszek Summarized the proposal again in https://pagure.io/fesco/issue/2406
Decisions from today's meeting: https://pagure.io/fesco/issue/2406#comment-658315
Not sure what to do with this ticket though, but at least now the impact of modularity is very limited so I am dropping meeting tag and we can save this ticket for some better times.
Metadata Update from @ignatenkobrain: - Issue untagged with: meeting
We have discussed the future of Modularity and Red Hat's requirements with our management and I have an update on the Red Hat's Modularity team roadmap:
Short-term and mid-term plan Fix the current modularity and make it work better in RHEL 9. This is aligned with what I wrote earlier in this ticket.
Long-term plan We are allowed to make major changes in Modularity for RHEL 10 which may include developing a different technical implementation. Because the team's capacity is currently spent on fixing the current implementation, I expect us to start working on any bigger changes after RHEL 9 Beta is released.
Please read Josh's email for more details. https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/YIUXL2AWY3GKITU4TBUSKL2IUHDUPB26/
Since this ticket suffers from a certain scope creep, I'm closing it. Please create new tickets to address individual issues such as https://pagure.io/fesco/issue/2406
I apparently cannot close the issue, could someone do it for me?
Metadata Update from @ngompa: - Issue close_status updated to: Accepted - Issue status updated to: Closed (was: Open)
Fix the current modularity and make it work better in RHEL 9.
Sounds good. But we don't know any details.
Also, the plan to fix things before RHEL 9 Beta doesn't answer the original questions about the scope of Modularity in Fedora.
Metadata Update from @zbyszek: - Issue status updated to: Open (was: Closed)
Fix the current modularity and make it work better in RHEL 9. Sounds good. But we don't know any details. Also, the plan to fix things before RHEL 9 Beta doesn't answer the original questions about the scope of Modularity in Fedora.
Sounds good. But we don't know any details. Also, the plan to fix things before RHEL 9 Beta doesn't answer the original questions about the scope of Modularity in Fedora.
In that case, I don't feel I'm the right person to own this ticket. I'm happy to help with technical discussions such as https://pagure.io/fedora-docs/modularity/pull-request/83#comment-122310 but this seems to belong to FESCO, Council or the Fedora leader.
I think this implicitly answers questions in the topic:
this is the kind of thing that we ultimately want to modularize
We do not want.
this is the kind of thing that might be modularized if XYZ conditions are met
Anybody is free to create modular content, according to module guidelines (can't find link right now). We also approved policy that non-modular package must exist and be supported if alternative version would be in a module.
this is the kind of thing that is supposed to stay in Fedora non-modular forever
Until the ~ RHEL9 beta is out, everything ;)
But this is just my understanding :)
Metadata Update from @dmach: - Assignee reset
This is my understanding as well. The scope of modularity is to provide alternate content.
OK.
Login to comment on this ticket.