#1483 Decision on bundling policy in the Fedora Package Collection
Closed None Opened 3 years ago by sgallagh.

= phenomenon =
The Fedora Packaging Guidelines' bundling policies have been identified as a barrier to involvement in the Fedora Project. It is FESCo's responsibility to determine a balance between the difficulty that proposes vs. the security and stability of Fedora as a whole. This issue is significant enough that it is time to revisit it.

= background analysis =
Megathread on devel@lists.fedoraproject.org:
https://lists.fedoraproject.org/pipermail/devel/2015-September/214262.html

= implementation recommendation =
I am revising my original proposal from that thread to read as follows:

=== Mandatory ===
The Fedora Base Working Group has been tasked with defining the base platform of Fedora since its inception. As part of this proposal, we set a deadline for them to provide (and maintain) a specific list of critical path packages. The critical path set is ''not'' required to be self-hosting.
Working Groups for the separate Editions '''may''' voluntarily add packages into the critical path atop the Base WG requirements.
All packages in the critical path '''must''' obey the current strict bundling rules.
All packages not in the critical path whose upstreams allow them to be build against system libraries '''must''' be built against system libraries.
All packages not in the critical path whose upstreams have no mechanism to build against system libraries '''must''' be contacted publicly about a path to supporting system libraries. If upstream refuses, this must be recorded in a link included in the spec file.
All packages not in non-critical path whose upstreams have no mechanism to build against system libraries '''may''' opt to carry bundled libraries, but if they do, they '''must''' include {{{Provides: bundled(<libname>) = <version>}}} in their RPM spec file.

=== Strongly Recommended ===
* Packages in the critical path should be re-reviewed every two years (possibly as a Flock workshop) to avoid unintentional divergence from the policies.


Out of curiosity... I assume this would not be usable for projects that are bundling huge number of libraries and packaging it would require adding another 10 packages into Fedora?

One example is Arduino IDE, which is written in Java and is bundling number of libraries. The version included in Fedora is ancient, because rebasing it would require number of new Java packages to be added into Fedora as well. I know COPR can be used, but then Arduino IDE will not be included in base Fedora.

Replying to [comment:2 thozza]:

Out of curiosity... I assume this would not be usable for projects that are bundling huge number of libraries and packaging it would require adding another 10 packages into Fedora?

One example is Arduino IDE, which is written in Java and is bundling number of libraries. The version included in Fedora is ancient, because rebasing it would require number of new Java packages to be added into Fedora as well. I know COPR can be used, but then Arduino IDE will not be included in base Fedora.

I don't know the specific example. Are you concerned that it falls into the case of "All packages not in the critical path whose upstreams allow them to be build against system libraries must be built against system libraries."? If so, I think that's probably a case for asking FPC for a temporary exception to land the major pieces quickly and plan to split out the bundled packages over the course of one or two releases. I suspect they'd approve that as long as there was a timeline provided.

I have a question and a couple of minor points.

Q: The proposal seems to completely eliminate the FPC from bundling discussions, yet your reply to thozza still suggests that e.g. Arduino IDE needs an exception. Can you elaborate on how (or if) the FPC is supposed to operate with this proposal?

Minor points:

  • Links can and do fail. I would suggest adding a file to the package git repo called 'upstream_bundling.txt' that contains both the link to the discussion (email archive, github issue, whatever), and the primary text of that discussion itself.

  • When it comes to critical path packages, you've allowed WGs to extend the Base WG definition. Do the bundling requirements apply to Base WG critpath + extensions in that case, or only to the Base WG critpath?

  • It is unclear as to whether 'critpath' will be redefined as 'Ring 0' or if they will overlap somehow. In the event that 'Ring 0' ever actually gets defined, we should ensure that all the packages contained within it are also subject to the critpath requirements listed here.

Replying to [comment:4 jwboyer]:

I have a question and a couple of minor points.

Q: The proposal seems to completely eliminate the FPC from bundling discussions, yet your reply to thozza still suggests that e.g. Arduino IDE needs an exception. Can you elaborate on how (or if) the FPC is supposed to operate with this proposal?

For cases where the default policy is still "you must bundle", FPC should retain its current policy for determining when there are acceptable exceptions. Sorry about that; in my first draft I spelled that out explicitly and then somehow lost it in editing.

Minor points:

  • Links can and do fail. I would suggest adding a file to the package git repo called 'upstream_bundling.txt' that contains both the link to the discussion (email archive, github issue, whatever), and the primary text of that discussion itself.

Perfectly sensible.

  • When it comes to critical path packages, you've allowed WGs to extend the Base WG definition. Do the bundling requirements apply to Base WG critpath + extensions in that case, or only to the Base WG critpath?

The whole point of allowing them to extend the definition is to extend the (non)-bundling requirements.

  • It is unclear as to whether 'critpath' will be redefined as 'Ring 0' or if they will overlap somehow. In the event that 'Ring 0' ever actually gets defined, we should ensure that all the packages contained within it are also subject to the critpath requirements listed here.

You caught me. This was kind of a backdoor into forcing a "Ring 0" definition to finally happen. I probably shouldn't have used the term "critical path" above, as it is reuse of an older term we used to have. I was explicitly avoiding using "Ring 0", though. Since I expect that to result in a much more involved (and distracting) conversation.

Replying to [comment:3 sgallagh]:

Replying to [comment:2 thozza]:

Out of curiosity... I assume this would not be usable for projects that are bundling huge number of libraries and packaging it would require adding another 10 packages into Fedora?

One example is Arduino IDE, which is written in Java and is bundling number of libraries. The version included in Fedora is ancient, because rebasing it would require number of new Java packages to be added into Fedora as well. I know COPR can be used, but then Arduino IDE will not be included in base Fedora.

I don't know the specific example. Are you concerned that it falls into the case of "All packages not in the critical path whose upstreams allow them to be build against system libraries must be built against system libraries."? If so, I think that's probably a case for asking FPC for a temporary exception to land the major pieces quickly and plan to split out the bundled packages over the course of one or two releases. I suspect they'd approve that as long as there was a timeline provided.

Yes, I think it would be the case. The problem e.g. with Arduino IDE is that one may want to update it, but doesn't want to maintain another 5 new Java packages. Also using system libraries requires downstream patches. But I guess that can be considered as "upstream don't support using system libraries".

Replying to [comment:6 thozza]:

[...] The problem e.g. with Arduino IDE is that one may want to update it, but doesn't want to maintain another 5 new Java packages. Also using system libraries requires downstream patches. But I guess that can be considered as "upstream don't support using system libraries".

If patching to use system libraries is easy and the application still works without any regressions then there's no excuse not to do it, even if upstream doesn't want to take the patch(es). You're effectively maintaining those 5 new Java packages anyway, you're just not exposing them for consumption by other packages. I know it's a lot of work initially, but it does pay in the long run.

Replying to [comment:5 sgallagh]:

Replying to [comment:4 jwboyer]:

I have a question and a couple of minor points.

Q: The proposal seems to completely eliminate the FPC from bundling discussions, yet your reply to thozza still suggests that e.g. Arduino IDE needs an exception. Can you elaborate on how (or if) the FPC is supposed to operate with this proposal?

For cases where the default policy is still "you must bundle", FPC should retain its current policy for determining when there are acceptable exceptions. Sorry about that; in my first draft I spelled that out explicitly and then somehow lost it in editing.

With my FPC hat on, I don't think cutting FPC out of the process is wise. If anything, we could delegate domain-specific bundling approvals to domain-specific SIGs. Relaxing the policy to the extent being proposed will remove all motivation to convince upstreams not to bundle and I fear the security landscape of Fedora will suffer.

Replying to [comment:5 sgallagh]:

Replying to [comment:4 jwboyer]:

I have a question and a couple of minor points.

Q: The proposal seems to completely eliminate the FPC from bundling discussions, yet your reply to thozza still suggests that e.g. Arduino IDE needs an exception. Can you elaborate on how (or if) the FPC is supposed to operate with this proposal?

For cases where the default policy is still "you must bundle", FPC should retain its current policy for determining when there are acceptable exceptions. Sorry about that; in my first draft I spelled that out explicitly and then somehow lost it in editing.

How is that an improvement on the barrier to entry issue that this is supposed to address? All of the points about critical path/Ring 0 are great and make it blatantly clear bundling isn't allowed there. However, those packages weren't really burdened with this anyway for the most part.

So if we're going to require FPC to still grant exceptions for the packages that this proposal was actually supposed to address, then I'm confused how it is any different from the status quo.

OK, after a conversation on IRC today, I finally realized that I made a critical typo in comment #6. When I said "where the default policy is still "you must bundle" ", I really meant "where the default policy is still "you must '''unbundle'''" ". For clarity, I will restate the complete, revised proposal here (adding a couple out:

=== Mandatory ===
The Fedora Base Working Group has been tasked with defining the base platform of Fedora since its inception. As part of this proposal, we set a deadline for them to provide (and maintain) a specific list of critical path packages. The critical path set is ''not'' required to be self-hosting.
Working Groups for the separate Editions '''may''' voluntarily add packages into the critical path atop the Base WG requirements.
All packages in the critical path '''must''' obey the current strict bundling rules. Packages in the critical path that require bundling '''must''' continue to seek exceptions from the Fedora Packaging Committee.
All packages not in the critical path whose upstreams allow them to be build against system libraries '''must''' be built against system libraries.
All packages not in the critical path whose upstreams have no mechanism to build against system libraries '''must''' be contacted publicly about a path to supporting system libraries. If upstream refuses, this '''must''' be recorded in the spec file using a persistent mechanism to be clarified in the packaging guidelines.
All packages not in the critical path whose upstreams have no mechanism to build against system libraries '''may''' opt to carry bundled libraries, but if they do, they '''must''' include {{{Provides: bundled(<libname>) = <version>}}} in their RPM spec file.

=== Strongly Recommended ===
* Packages in the critical path should be re-reviewed every two years (possibly as a Flock workshop) to avoid unintentional divergence from the policies.

Note: the term "critical path" is reused from a retired Fedora packaging concept, but it is not representing the same set of packages. We may need to come up with a more distinct term, but I can't think of a better choice at this moment. Please avoid repainting this shed.

Replying to [comment:11 sgallagh]:

OK, after a conversation on IRC today, I finally realized that I made a critical typo in comment #6.

Sorry, Trac always confuses me with its positioning of the comment number string. I was referring to comment 5.

This clears things up for me. Thank you for the new summary.

Approved Proposal:
* All packages whose upstreams allow them to be built against system libraries must be built against system libraries.

  • All packages whose upstreams have no mechanism to build against system libraries must be contacted publicly about a path to supporting system libraries. If upstream refuses, this must be recorded in the spec file using a persistent mechanism to be clarified in the packaging guidelines.

  • All packages whose upstreams have no mechanism to build against system libraries may opt to carry bundled libraries, but if they do, they must include {{{Provides: bundled(<libname>) = <version>}}} in their RPM spec file.

Login to comment on this ticket.

Metadata