So ... this means that any user who has an affected package installed on Fedora 37 and upgrades to Fedora 38 will basically get a broken packager manager? That's not good :(

I think we have to block on this problem (Beta ideally, but Final definitely), because it's a major black eye if popular packages don't work properly on Fedora and you upgrade to a broken setup.

Proposal: FESCo requests this be handled as a Beta Blocker, as the functional and reputational damage caused by this is too significant to ignore.

So ... this means that any user who has an affected package installed on Fedora 37 and upgrades to Fedora 38 will basically get a broken packager manager? That's not good :(

Yes, but please note there's a difference between weak signatures in RPM packages and weak signatures in repo keys. Weak signatures in RPM packages (Chrome) block system updates and you can't even uninstall the package using standard means. Weak signatures in repo keys also block system updates, but you can uninstall the package (e.g. VSCode), which will then unblock system updates (because an affected package will no longer be in the transaction).

+1 to Neal's text.

I'm open to rewordings and adjustments. The important choice is whether to block on this (my answer: "yes"), and whether to block beta (also: "yes"). This is a very significant issue that will prevent many users from really testing Beta. It can be worked around, but that requires manual steps and manual adjustments on the package set, making Beta testing harder, but also making the results of such testing much less useful.

For me the gold standard is that the proverbial "grandma" in front of the computer with Fedora can press "Update to latest version" twice per year and this must succeed (without any custom operations in the terminal or tweaking of boot options or such). Anything else is a bug.

+1 to Neal's proposal.

+1 to Neal's proposal.

Another +1 for Neal’s proposal.

+1 here as well for Neal's proposal

+1 (reluctantly)

I'm a bit worried about how this will be unblocked. It sounds like it's difficult to fix in rpm, so the other obvious alternative would be to just relax the entire crypto-policy, but thats a very big hammer indeed.

I'm a bit worried about how this will be unblocked.

Proposal: require that dnf remove packages with old signatures? We certainly need upgrades to not fail. Probably don't need newer Fedoras to be compatible with third-party packages built for older Fedoras.

Probably don't need newer Fedoras to be compatible with third-party packages built for older Fedoras.

That is not how these third party repositories work. Especially not Google Chrome. There's only one RPM for everyone.

So to clarify what I meant: Kevin is concerned we'll never land the change if we block on third parties, and that seems likely, so instead of waiting for them to fix their RPMs, I suggest we just ensure that the RPMs get uninstalled during upgrade. GNOME Software already has a nice dialog to warn the user when an upgrade will result in uninstallation of applications (where an "application" is defined to be something with appstream metadata).

Alternatively, FESCo might decide we want Fedora to continue to allow use of these third-party RPMs, but if so I predict the change will probably never land again, because good luck convincing third-party repos to change what they're doing before their packages stop working (and how to define which third-party repos we care about anyway?).

Michael raises a great question, and this is exactly what I'd like to decided by FESCo. If affected third-party apps are not installable on F38, and we prevent people from reaching a broken state with upgrades (either by uninstalling those rpms or showing an error dialog with information), is that an acceptable solution? (Note that I'm not sure if we can technically do this, but at least we know whether to attempt it).

As far as I am concerned, cleanly forcing uninstallation of third-party RPMs with weak signatures would be an acceptable resolution—unfortunate for users, but acceptable under the circumstances. I agree that most third parties won’t change anything proactively, and I do not want the stricter signature policy deferred indefinitely.

BTW Google Chrome is extremely widely used, so if we force uninstallation that will unfortunately be disruptive. OK, I think that's really all I have to contribute here... good luck with this problem. :(

Can we get RPM (or dnf at a higher level) to print a more useful error message instead of the current cryptic one? Users should be clearly presented with the choice to either weaken their system's crypto policy or remove the offending software.

As far as I am concerned, cleanly forcing uninstallation of third-party RPMs with weak signatures would be an acceptable resolution—unfortunate for users, but acceptable under the circumstances. I agree that most third parties won’t change anything proactively, and I do not want the stricter signature policy deferred indefinitely.

Fully agreed.
I understood Neal's proposal to include this as a possible way to handle this issue.

BTW Google Chrome is extremely widely used, so if we force uninstallation that will unfortunately be disruptive.

Yeah. But I think it's acceptable to uninstall chrome and describe the issue in Common Bugs. We can't really block on third party repos. E.g. for Chromium the issue was reported 2022-12-07 (https://bugs.chromium.org/p/chromium/issues/detail?id=1398429).

I understood Neal's proposal to include this as a possible way to handle this issue.

I would not accept that as a way to handle the issue. Uninstalling Google Chrome, for example, would be a serious problem for people using Chrome for their corporate laptops. It would probably result in Fedora being dropped in such environments.

Google Chrome has been available in the Fedora Workstation third-party repositories for a number of releases. In a GNOME survey: https://blogs.gnome.org/aday/2023/01/18/gnome-info-collect-what-we-learned/ 35% of users had Google Chrome installed and 11% had it as their default browser. Fedora is going to be roughly similar. (VS Code was installed on 28% of machines.)

If we ship Fedora Workstation with this policy applied to RPMs a large fraction of users are going to struggle and eventually permanently weaken their crypto policies.

At least for Workstation, I think we need to back off this policy until Fedora 39 and work hard to get the most high profile RPMs and repositories updated. Perhaps we can find to apply a different crypto policy for this RPM/dnf than for other users.

And shouldn't we consider it a bug that having an RPM previously installed with a non-comformant signature makes RPM misbehave? (as I understand the reported symptoms). How does that help anybody?

Uninstalling these RPMs would definitely be disruptive and frustrating. I would prefer to see a solution that neither does this nor indefinitely postpones distrusting SHA-1 signatures. I am not sure how much ability we have to influence these high-profile third-party sources or get them to commit to a particular timeline. I hope that more information will soon be available.

I will say that for Chrome in particular, I think allowing it to remain installed but prohibiting further security updates would be more harmful than either uninstalling it or continuing to pemit SHA-1 signatures across the board.

Permitting SHA-1 signatures across the board is absolutely a valid option. We need messaging for getting everyone on board with upgraded, but breaking everything without directly contacting at least our third party repository vendors and working with them to upgrade things first is not acceptable. It is hugely disruptive.

Just a clarification, this is not just about SHA1. sudo update-crypto-policies --set DEFAULT:SHA1 is not enough to allow Chrome to update. Its repo gpgkey https://dl.google.com/linux/linux_signing_key.pub is still rejected, until I do sudo update-crypto-policies --set LEGACY. Some security-aware person can tell us which additional algorithm is being blocked here.

I don't want to add unnecessary technical details to this discussion, just please know that this is not just SHA1-related problem.

E.g. for Chromium the issue was reported 2022-12-07 (https://bugs.chromium.org/p/chromium/issues/detail?id=1398429).

And on Nov 11 2022: https://bugs.chromium.org/p/chromium/issues/detail?id=1383526

Given that this is time-sensitive, I'm putting this on the agenda for today's FESCo meeting.

BTW Google Chrome is extremely widely used, so if we force uninstallation that will unfortunately be disruptive. OK, I think that's really all I have to contribute here... good luck with this problem. :(

Well, Workstation WG wound up discussing this today. We have consensus on the following proposal for FESCo to consider:

• RPM security change should be permanently reverted in Fedora 38. We request more time for us to work with third-party repos to adjust to this change
• The change should be reverted in rawhide too until uninstallation of affected RPMs is possible.
• Once uninstallation is possible, RPM developers should feel free to proceed with all desired changes in rawhide regardless of impact on third-party repos. There is sufficient time between now and Fedora 39 for third-party repos to adapt, and no need to block the change indefinitely on third-party repos that we do not control. (In particular, we expect Google Chrome repo should be updated well before Fedora 39 release.)

• AGREED: FESCo agrees to block Beta for this issue. In order to unblock, RPM must accept SHA-1 hashes and DSA keys for Fedora 38, ideally with a deprecation warning that it will be disabled in F39. FESCo strongly advises against allowing these algorithms elsewhere, but will accept that for F38 if it's the only achievable, timely solution. (+8, 0, -0) (sgallagh, 17:33:23)

This was discussed during today's FESCo meeting. The update that resolves the issue is going through testing. No action is required from our side.