Time for a Fedora 39 Final candidate compose. We have some outstanding blockers, but we (@mattdm, @kevin and I) decided it would be reasonable to do a candidate now and discuss whether to make special provisions for the outstanding blockers (which are all Raspberry Pi related, aside from the longstanding shim issue which we already intend to waive) at the go/no-go meeting.
Please make sure all necessary configs are adjusted for a final release.
Please include:
Untag all previous builds:
koji list-tagged f39-compose --quiet | awk '{print $1}' | xargs -rt koji untag-pkg f39-compose
and then tag the new builds:
blockers=( mutter-45.0-11.fc39 ) fes=( libnvme-1.6-2.fc39 libblockdev-3.0.4-1.fc39 ) koji tag-build f39-compose ${blockers[@]} ${fes[@]}
Compose started: https://kojipkgs.fedoraproject.org/compose/39/Fedora-39-20231025.0/
Compose finished, it's syncing now.
All synced out. :)
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
OK, time for RC2, with (we hope) the Pi bugs fixed with a uboot-tools rollback. Please include:
blockers=( mutter-45.0-11.fc39 bcm283x-firmware-20231017-1.ce3a0b4.fc39 uboot-tools-2023.07-3.fc39 ) fes=( libblockdev-3.0.4-1.fc39 libnvme-1.6-2.fc39 ) koji tag-build f39-compose ${blockers[@]} ${fes[@]}
Metadata Update from @adamwill: - Issue status updated to: Open (was: Closed)
Compose fired off. :checkered_flag:
Compose finished, syncing now.
Looks to me the RC artifacts has incorrect names, they include "39_RC-1.2", while previously they had only "39-1.2" in the past Fedora GA releases. See eg. https://kojipkgs.fedoraproject.org/compose/39/Fedora-39-20231026.0/compose/Everything/ppc64le/iso/
IIRC there were some changes made to the naming for the Beta(?) or was it for F-38(?) ...
and it should be fixed in https://pagure.io/pungi-fedora/c/8cf94a7b9f5666938d2b26a4f6dfc6c6de74c9de?branch=f39
Thanks :-)
Yes, rc-1.2 had incorrect image / checksum names. ;(
It's all done and synced tho.
I fixed that issue in pungi config and fired off a rc 1.3
Metadata Update from @phsmoura: - Issue tagged with: medium-gain, medium-trouble, ops
rc 1.3 blew up due to me not running it with the correct config. ;(
I'm going to close this for now and QE can request 1.4 once we have sorted out when/what/if we need one.
OK, we're gonna need another candidate now. Please include:
blockers=( arm-image-installer-3.9-2.fc39 bcm283x-firmware-20231017-1.ce3a0b4.fc39 uboot-tools-2023.07-3.fc39 anaconda-39.32.6-2.fc39 dracut-059-15.fc39 ) fes=( mutter-45.0-12.fc39 kitty-0.30.1-2.fc39 golang-github-seancfoley-ipaddress-1.5.5-1.fc39 golang-github-seancfoley-bintree-1.2.3-1.fc39 golang-github-altree-bigfloat-0.2.0-1.fc39 ) koji tag-build f39-compose ${blockers[@]} ${fes[@]}
note, I added the kitty update manually because unfortunately Bodhi refuses to let me edit the update and mark it as fixing the bug (I think because the side tag has been garbage-collected, or something). Please double-check I got it right.
alternatively we could push the kitty update stable before composing, I guess. It's queued. I'll file a stable push.
rc-1.4 fired off.
1.4 finished, syncing.
There were a few failures... the same as the nightly ones we have been seeing. ;(
all done last night while I was asleep. ;)
OK, let's do another, since we need to pull in Firefox 119. this hopefully gives us a chance to fix up some failing atomic images too. Please include:
blockers=( firefox-119.0-2.fc39 nss-3.94.0-2.fc39 ) fes=( ) koji tag-build f39-compose ${blockers[@]} ${fes[@]}
RC-1.5 started.
RC-1.5 finished and synced.
The Onyx/ and Sericea/ dirs in https://dl.fedoraproject.org/pub/alt/stage/39_RC-1.5/ are empty. (I see those dirs don't exist at all in https://kojipkgs.fedoraproject.org/compose/39/Fedora-39-20231031.1/compose/ .)
runroot.log from https://koji.fedoraproject.org/koji/taskinfo?taskID=108388617 shows they failed likely due some network/infra issue :-(
There are currently two heisenbugs affecting composes which mean we are basically never going to get a full set of deliverables ATM. Something is always going to fail. We've been trying to debug them for a while but they're both proving to be very tricky.
Would it make sense / be possible to do another RC build with the exact same set of packages and then take the failed images from the new RC (hoping thet succeed there)? So we'd mix and match the two composes to get a full set of images?
We could, but there's no time to test it or finish composing it before the go/no-go.
I was going to advocate that we try and take versions of those images that failed in rc 1.5 from the final nightly (tomorrow, after we push all the updates stable). Of course they could also fail in tomorrows nightly.
They would also be named weirdly
Yep. We would need to put them somewhere else, they would be named weirdly (YYYYMMDD.n.0 instead of 1.5) Alternately, we could just say we dont have those images for f39 and advise folks to install f38 and rebase. But we can discuss at the meeting.
I don't know if it matters, but the signed checksum files in https://kojipkgs.fedoraproject.org/compose/39/Fedora-39-20231031.1/compose/ haven't been synced to https://dl.fedoraproject.org/pub/alt/stage/39_RC-1.5/ .
They don't generally get synced to alt, they get synced to the final release location.
Log in to comment on this ticket.