Generally, I would say packages don't require FESCo approvals to have arbitrary Provides. But I guess the context is in the EPEL issue.

Do you want FESCo to overrule EPEL folks here, or do you seek to have the provide in Fedora but not EPEL?

I seek to have it in both Fedora and EPEL. The EPEL people said I needed to ask FESCo first, because they want to do the same thing as Fedora.

Background: I was originally contacted directly by the EPEL Steering Committee member that created the ticket there. After some back-and-forth, I said that I cared more about the settings in EPEL than Fedora, and since we didn't agree, I asked to bring it to the whole EPEL committee. So that's what he did. I didn't realize at the time that they preferred to ask FESCO first, since I was not familiar with the relationship between the two.

It's also not a matter of overruling; the situation has changed since they made their decision. They didn't want to give permission before the changes were made; they didn't want to make a decision based on a promise that the changes would be made.

Hi - I'm from the company referenced by dwd in his post above, i.e Sylabs. I am also the packager for singularity-ce in Fedora / EPEL. SingularityCE is an alternative fork of Singularity (vs Apptainer), that is primarily maintained and developed by Sylabs.

I'd just like to correct / clarify a couple of points that dwd has made around changes in Apptainer vs Singularity. I personally believe these changes do affect users enough that Apptainer does not provide Singularity (as it was previously used), but I am cognisant of the fact that I have conflicts of interest in this matter.

Because the Singularity project was previously unified under Sylabs, Singularity has incorporated use of Sylabs online services by default, for some time. This is similar e.g. to how Docker has included default use of Docker Hub when pushing/pulling containers etc.

Specifically, the first change was the removal of a company's servers as a default "remote" for using the "library://" protocol; if a user tries to use that, they are now by default pointed to documentation showing how to either use an alternative standards-based protocol or re-enable the company's servers.

Sylabs provides a public library for Singularity images, at https://cloud.sylabs.io, in much the same way as Docker, the company, provides Docker Hub as a default registry for Docker containers.

The change in Apptainer that removed the default server for the library:// protocol is analogous to a fork of Docker changing so that pull / push are not against Docker Hub by default. Given Singularity has defaulted to the Sylabs service since its inception, this is a fairly significant confusing change to users.

The second change was the removal of a "--remote" option to the build command to build containers on the company's servers, for a fee; if a user tries to use that, they are now told to try just leaving out the option, which will work under most of the common circumstances and has advantages over remote builds in addition to not requiring a fee.

This makes it seem as if all use of the remote build option requires paying Sylabs a fee, which is not true. The build service is closed source, but the client side code is open source. There is a public instance that is part of Sylabs' cloud.sylabs.io available to all, with a free allocation of build minutes.

There are some technical limitations to the new build features in Apptainer, mostly related to building containers of new distros on an old host (e.g. building an EL9 container on an EL7 host), when subuid/subgid maps cannot be implemented. The --remote build remains useful for people in these situations.

Many thanks for considering the issues around the Singularity provides. Please let me know if I can clarify anything.

Quick recap:
- singularity project has been replaced by singularity-ce and apptainer.
- Both children are almost-but-not-quite compatible replacements, with plans to further diverge from the parent.
- apptainer (one of the subpackages) has Obsoletes:singularity, so on upgrades, singularity is replaced with that subpackage.
- apptainer had Provides:singularity so that it would be installed when singularity was requested, but this was dropped after EPEL council vote.

The request is to add back Provides:singularity. It only matters for new installations, as upgrades are already handled. If we consider the renaming guidelines, it could be added or not, depending on how important the incompatible changes are judged to be. EPEL members mostly seemed to support "no", but after some changes to restore compatiblity were made in the latest release, the arguments for "yes" are stronger. But there is a second consideration: there are two daughter projects. Fedora should not be in the business of picking one project or the other, since both are suitable for packaging and available to users. If the Provides is not present, users have to make an explicit choice between the projects. I think this is the best way to proceed.

Proposal: FESCo does not support adding of Provides:singularity to either of the two projects.

+1

+1 for zbyszek's proposal.

+0

I think that both projects could have the provides. However, that does not allow users to make an explicit choice, as said.

They could also both provide alternative-for(singularity) and dnf would report both packages. This was a change done for dnf install python in RHEL 8 and apparently it made it to mainline dnf:

https://github.com/rpm-software-management/dnf/blob/e50875b3f5790f70720bdb670e1dd2bf4d828744/dnf/cli/commands/install.py#L44

Not sure if this is to be kept in DNF 5.

$ mock -r epel-8-x86_64 install python
...
No match for argument: python
There are following alternatives for "python": python2, python36, python38, python39

Ah, that's nice. It seems that dnf5 does support this:
https://github.com/rpm-software-management/dnf5/blob/main/libdnf/base/transaction.cpp#L144

Amended proposal:

Proposal: Both projects should add alternative-for(singularity) to allow users to make an explicit choice for new installations.

How does that interact with the Obsoletes: singularity that is already on apptainer-suid?

Please keep in mind that there is a difference between the two projects. Apptainer is a rename and SingularityCE is a fork. The Fedora policy is about renames and whether or not they are compatible enough, nothing about forks. Does the policy need to be changed to also talk about the existence of forks?

How does that interact with the Obsoletes: singularity that is already on apptainer-suid?

I expect that Obsoletes takes effect during upgrades, and Provides takes effect during new installations (as shown by @churchyard above).

Apptainer is a rename and SingularityCE is a fork.

That is true, but it's also somewhat beside the point. Because of changes, users might want one or the other, and the latest proposal makes the choice explicit. The goal is to make users' lives nicer, not to follow the guidelines literally.

Background: I was originally contacted directly by the EPEL Steering Committee member that created the ticket there. After some back-and-forth, I said that I cared more about the settings in EPEL than Fedora, and since we didn't agree, I asked to bring it to the whole EPEL committee. So that's what he did. I didn't realize at the time that they preferred to ask FESCO first, since I was not familiar with the relationship between the two.

This is a misrepresentation of our communications. I contacted you originally about resolving the file conflicts between apptainer and singularity-ce, per the conflicts guidelines. You immediately tried to pivot into disparaging singularity-ce and arguing that apptainer must provide singularity. I basically gave you the same advice that @zbyszek did in this issue about Fedora picking one project or the other and that users making an explicit choice would be the best outcome. That also aligns with the renaming guidelines, specifically if a replacing package isn't a sufficiently compatible replacement that it should only obsolete and not provide the previous package name. You agreed to remove the provides for Fedora branches, but wanted to diverge in EPEL branches and continue to provide singularity. That is why it was brought to the EPEL Steering Committee. The decision from us was that there wasn't justification to diverge. Because of that, I advised you to bring the issue to FESCO for guidance, and EPEL would follow whatever FESCO decides.

Carl: I don't see how what I wrote is a misrepresentation. You asked me to make several changes, and the only one I objected to was the Provides in EPEL, because that's what matters most to the vast majority of the users.

Zbigniew: seriously, if FESCo wants to do something different than the written policy, it's time to revise the policy. I have in good faith been responding to every objection based on the compatibility portion of the policy, and in the end it turns out that is irrelevant, the only thing that matters is the existence of a fork.

The goal is to make users' lives nicer, not to follow the guidelines literally.

I believe that it does make user's lives nicer if they continue to get the package from the project they always got it from before. The SingularityCE fork has never used the rpm package name "singularity". I think that people who have done yum update singularity on one machine will expect to get the same package if they do yum install singularity on another machine. Since the removal of the Provides we have also had two reports (on bugzilla and github) of people who had either Requires: singularity in their own package or dnf install singularity in their automated container rebuild and now have suddenly had to change them.

I'm adding this to the agenda for the meeting today (17:00 UTC, #fedora-meeting on LibreChat).

@dwd, @dctrud, @carlwgeorge: you are all invited.

Last week's meeting was cancelled because of lack of quorum. Let's try again today.

Last week's meeting was cancelled because of lack of quorum. Let's try again today.

I was lurking in IRC the past 2 weeks... :-) ... but unfortunately I have a conflict today at 17:00 UTC so won't be able to attend.

The only thing I'd anticipate wanting to possibly raise / clarify in the meeting is to reinforce that at this point in time, singularity-ce is closer to the functionality of the old singularity package than apptainer-suid is, due to...

• apptainer from 1.0 removing remote build functionality entirely (was present in singularity, and remains in singularity-ce).
• apptainer from 1.0 removing the default configuration for pulling containers from cloud.sylabs.io (analogous to changing docker, podman etc. to remove the default of pulling from Docker Hub).

... which I'd argue are not things users would reasonably expect, and are not "changes of magnitude that are commonly found in version upgrade changes".

On the other incompatibility points raised during the history of this topic, it's clear that @dwd has performed quite a bit of work, in good faith, to clean things up w.r.t the incompatible upgrade from singularity to apptainer. E.g. configuration migration. Also, putting Provides: singularity on apptainer-suid instead of apptainer negates a large subset of arguments against the provides.

I haven't asked for singularity-ce to be able to take Provides: singularity, despite it being closer to the old singularity package in behaviour than apptainer is. That would be a stretch, given the lineage of the package that @dwd has been maintaining, and the fact that singularity-ce and apptainer are both diverging, or plan to diverge, over time from the old 3.8 singularity.

However, I do personally feel that apptainer-suid doesn't truly provide singularity as it was up to 3.8, and the features that it lacks / changes it makes are significant to users.

Many thanks again to FESCO and those involved in the EPEL discussions for considering this matter carefully, and to @dwd for the work that has been done to resolve some aspects of the incompatible upgrade.

This was discussed during yesterday's meeting:
* LINK: https://pagure.io/fesco/issue/2934#comment-839521 claims that both have diverged or plan to diverge
* LINK: https://pagure.io/fesco/issue/2934#comment-836095
* AGREED: both packages add Provides: alternative-for(singularity) (+6,0,-0)

Just noting here that the Provides: alternative-for(singularity) was applied to singularity-ce in rawhide (now f38) and epel branches yesterday, and so will reach epel stable next week.

Many thanks again for everyone's patience with this issue.