Issue #2116: Moving forward with zchunk metadata for F30? - fesco

fesco

#2116 Moving forward with zchunk metadata for F30?

Closed: Accepted 5 years ago by sgallagh. Opened 5 years ago by jdieter.

Currently, none of the F30 repositories have zchunk metadata enabled, and, after the little fiasco on Sunday, I'd like FESCo to decide how to move forward with re-enabling it.

To make a long story short, a librepo update (1.9.6-1) was released that segfaults dnf when it checks a repository that has zchunk metadata. More background is available in the next comment.

librepo-1.9.6-2, which fixes this bug, has been released and is on its way to stable in the next compose, but the broken librepo also made it to stable and is currently in the fedora repo.

Needless to say, I'm quite leery at the moment of turning on zchunk metadata without getting a green light, and, after talking it over with Adam Williamson, he suggested that FESCo should decide when and how it happens.

As I see it, the options we have are:

Turn on zchunk metadata within the next week or so (after everyone has hopefully updated to librepo-1.9.6-2) with a note that anyone who has librepo-1.9.6-1 will need to set zchunk=False in /etc/dnf/dnf.conf, update librepo, and then remove that line from dnf.conf.

Given that we had a zchunk-enabled base repo for a few weeks before we disabled it due to compose problems, and we didn't have any user bug reports against zchunk, I don't anticipate that this should cause a problem (but, then again, I didn't anticipate that librepo would get broken, so...)
Say that zchunk by default isn't quite ready yet but will be available as a "Technology Preview". Update libdnf so that zchunk=False is the default. Sometime after GA, when we're sure most people have the new librepo and a version of libdnf that won't automatically download zchunk metadata, we re-enable zchunk metadata for updates-testing and updates.

Anybody who wants to use zchunk can set zchunk=True in /etc/dnf/dnf.conf, which will give us a base of testers in preparation for F31. Everyone else will get the standard xz metadata. In Rawhide, libdnf will leave zchunk=True as the default.

Adam, quite reasonably, pointed out that the number of people who will actually set zchunk=True is probably pretty minimal, so this option might not be as useful as I thought it might be.
Say that zchunk isn't ready at all. Leave the F30 repositories without zchunk metadata. Continue with zchunk metadata enabled in Rawhide and get the feature into F31 without generating zchunk metadata in F30.

jdieter commented 5 years ago

Some background:

We disabled zchunk metadata on the fedora repository for about a week before Beta due to a bug that the compose process was hitting. We fixed the bug and re-enabled zchunk metadata on Rawhide, but left it off for the F30 composes.

On Saturday, we enabled zchunk metadata on the updates-testing repo since the compose bug was fixed and unlikely to affect users anyway. Unfortunately, a few days earlier an update of librepo was released that segfaulted when zchunk metadata was available in any repositories.

We discovered the bug Saturday night in the Rawhide composes, but didn't realize it affected F30 until Sunday morning (after updates-testing was pushed with zchunk metadata). We quickly turned off zchunk metadata in updates-testing, sent out an email explaining how to work around the problem, and published a patch to fix the bug.

churchyard commented 5 years ago

I think that at least Fedora contributors would like to have the possibility to have 2. This should give you plenty of people to test it, so I definitively prefer 2 over 3.

Important information: it looks like Fedora 30 Beta got librepo-1.9.5-1.fc30 - that makes me think that we should be ok with 1.

(Also note that the update should probably be pushed sooner, with lower karma limit and urgent severity.)

jdieter commented 5 years ago

(Also note that the update should probably be pushed sooner, with lower karma limit and urgent severity.)

I think it got pushed as quickly as it could. It got 3 karma before it even made it into the testing repository and is on its way to stable right now. I do agree that the severity should be urgent, but I'm not sure what practical effect it has.

BTW, I realized that I never linked to the update: https://bodhi.fedoraproject.org/updates/FEDORA-2019-07c3e09858

kevin commented 5 years ago

I'm ok with 1. It's only going to affect folks who updated at just the wrong time pre-beta. Hopefully pre-beta people know to look for problems like this and will see the solution.