Considering the thread "Fedora 27 mass rebuild at risk" on the devel mailing list, I propose that we consider postponing the mass-rebuild by two days to July 14th to give the glibc and gcc teams enough time for the dust to settle and ensure that things are working properly.
Given the already-constrained timeframe for Fedora 27, I further propose that we do not move out any other date to accommodate this; instead, we assume the risk involved in having a shorter period between mass-rebuild and branching (which was already longer than in previous releases due to the "No Alphas" change).
FESCo members: please chime in on this ticket immediately, as a decision on this proposal must be made today.
CCing @codonell @fweimer @jakub @mohanboddu @pbrobinson
+1
In the thread on devel, Go 1.9 was quoted as another reason for the mass rebuild. However, Go 1.9 has not yet landed in Fedora. Furthermore, Go 1.8 currently FTFBS on multiple architectures:
So Go would not benefit from the mass rebuild.
This means that the s390x import is the only reason left for a mass rebuild in Fedora 27. Can we do it without one?
the time for mass rebuild this time is what it always was supposed to be. releng will not start the massrebuild until everything is ready.
I'm +1 to waiting until things are ready for sure. If we end up not needing one, all the better....
sgallagh codonell: (Not rushing you) What is the current status RE: mass-rebuild? codonell sgallagh, gcc is done, glibc in-progress, go has an s390x fix, and go is waiting for the glibc fix so it can get a clean test run. sgallagh, I'm talking to IBM right now about the glibc fixes, and what we can do to make it happen now sgallagh OK, cool codonell sgallagh, I expect in 1-2 hours we have glibc done, then rebuild go, and when both green light, we can do the mass rebuild. sgallagh Let me know if there's anything I can do to grease the gears. codonell sgallagh, Prepare for the worst, a 1 day delay. sgallagh, 2 days if you want to be generous. I don't see use having a problem, we think we've ironed everything out. sgallagh, But preparing for the worse would be the best plan. sgallagh codonell: OK, sounds good. Mind if I quote this conversation on the FESCo ticket? codonell sgallagh, Quote away please.
I'm +1, given RelEng's opinion on the matter.
Is there still anything that blocks the mass rebuild start? The last (at least arch specific) issue (https://bugzilla.redhat.com/show_bug.cgi?id=1470692), I'm aware of, got fixed today.
So, apparently rel-eng hasn't started the mass-rebuild yet because they were waiting on a decision from this ticket.
Somehow, the message got lost that they should have started on the 14th at the latest. I've just confirmed with the tools team that they are indeed green-lit. We should get the rebuild started ASAP, @mohanboddu
One thing that needs to be done before starting the mass rebuild is to merge all the koji side tags (ocaml, boost). Maybe a good plan would be to give a heads up to the side tags owners that they have until tomorrow to finish the work and then merge the side tags tomorrow and kick off the mass rebuild, so that it can run over the weekend?
@jwakely @rjones
another thing to mention here... binutils has/had bug on s390x, so if we will build whatever is in dist-git we might get that problem back.
binutils has been fixed with binutils-2.28-14.fc27
@mohanboddu mentioned on IRC that he's on PTO Friday and Monday. I wonder if it would even be possible to start the mass rebuild today so that we don't fall behind on the schedule that much.
f27-boost koji side tag just got merged; that shouldn't be blocking the rebuild any more.
f27-ocaml2 is still unmerged, but that contains just a single ocaml-4.05.0-1.fc27 build and the ocaml rebuild apparently hasn't started yet.
Given the constraints, maybe it would make sense to add an exception in the mass rebuild script to skip the 'ocaml' package , and run the mass rebuild without merging the f27-ocaml2 tag? This way the mass rebuild hopefully doesn't conflict with the ocaml work and @rjones can finish the ocaml rebuilds after the mass rebuild is done.
Anyone else that could keep an eye on the mass rebuild script if it falls over while @mohanboddu is gone? @kevin @pbrobinson ?
I don't know if there was a decision or not, but the OCaml build is delayed by a high priority RHEL 7.4 bug that turned up today.
After discussion about this on IRC, I'll wait for the Fedora 27 Mass Rebuild to finish before starting the OCaml 4.05 rebuild. See also: https://pagure.io/releng/issue/6906#comment-449668
Metadata Update from @maxamillion: - Issue tagged with: meeting
Metadata Update from @maxamillion: - Issue untagged with: meeting
Mass rebuild is complete
Metadata Update from @sgallagh: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.