In order to complete pkgconf as system pkg-config implementation, builders need to apply update of dnf and hawkey, which is already in stable.

Unfortunately bug in dnf (promoting 'install a' to 'install b' when b obsoletes a) was opened for long time and no one really fixed it. Yum behaved exactly how dnf behaves in this update for obsoletes, so there's no special hacks or workaround, just fix of obsoleting issue. In short, if you have X, Y which obsoletes X and you are trying to (Build)?Require X, you will always get X (while proper and expected behavior is to get Y).

Hopefully this is clear and no one has objections for doing this.

@jakub what is the current status of gcc7? will we have builds and be ready to run the mass rebuild next week?

So far we have scratch packages, e.g. https://kojipkgs.fedoraproject.org/scratch/jakub/task_17356239/
I'll do the first non-scratch build likely later today.
Marek has performed a test mass rebuild, but we still need to finish analysis.

@jakub Thank you. Can you please update the issue when you have done the analysis to let us know if it is ready for the mass rebuild.

If possible, please exclude pkgconfig package from mass rebuild, so that it doesn't go past the versioned Obsoletes+Provides set in pkgconf-pkg-config. If not possible, let me know so that I can bump the Obsoletes+Provides to account for that.

One easy way to avoid having to bump Obsoletes, is introduce an epoch so that it will always be newer (and if pkgconfig ever comes back, then it can use a newer epoch).

I really, really, really want to avoid Epoch bumping, since Fedora policy currently is that Epochs are permanent.

eol'ing and Obsoleting a package is also usually permanent (to be clear, I meant introduce epoch only in the Obsoletes/Provides: pkgconfig, not pkgconfig itself)

so, typo in the end, not pkgconf itself

as I have it the changes needed for the mass rebuild are as follows

https://fedoraproject.org/wiki/Changes/GCC7
https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo
https://fedoraproject.org/wiki/Changes/StaticLibraryDebuginfo
https://fedoraproject.org/wiki/Changes/Fedora26CFlags
https://fedoraproject.org/wiki/Changes/golang-buildmode-pie
https://fedoraproject.org/wiki/Changes/golang1.8

We need the boost side tag to be completed it is being tracked in #6605
a stretch goal is to get s390 merged into koji.fp.o before we build but we will not delay for it @kevin @sharkcz do we have any chance of the infra changes being done in time?

we need feedback/help from the mainframe eng-ops guys

it might be as simple as editing system.config file with different VLAN

00273 / define vswitch /
00274 define vswitch vsw1 rdev 0900 ETHERNET VLAN 1
00275

but I wouldn't dare to change it myself without being sure.

Current status as I know it

Done:
https://fedoraproject.org/wiki/Changes/pkgconf_as_system_pkg-config_implementation
https://fedoraproject.org/wiki/Changes/GCC7
https://fedoraproject.org/wiki/Changes/Fedora26CFlags
https://fedoraproject.org/wiki/Changes/golang1.8
https://fedoraproject.org/wiki/Changes/golang-buildmode-pie

Not Done
https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo
* https://bugzilla.redhat.com/show_bug.cgi?id=1340819 will not be ready until some time next week
https://fedoraproject.org/wiki/Changes/StaticLibraryDebuginfo
* waiting on https://bugzilla.redhat.com/show_bug.cgi?id=1395280

We need the boost side tag to be completed, it is being tracked in #6605

once boost is done we need to decide if we wait on the other two or start without them. of the two the bigger impact is the parallel installable debuginfo change while we can do the mass rebuild without and people will be able to install debuginfo still they will not get the benefits of the change.

Regarding the "Parallel Installable Debuginfo" Change I asked FESCo to provide a guidance how to move forward: FESCo issue #1669.

Do you think the mass rebuild will start already tomorrow, or during the weekend, or only next week? We are still fighting a couple of silent wrong-code issues in GCC, e.g.
http://gcc.gnu.org/PR79327
http://gcc.gnu.org/PR79352
http://gcc.gnu.org/PR79354
and would like to be able to find out how quickly we need to add fixes or workarounds.
I hope all 3 issues will be resolved during tomorrow or Saturday and we can do the build on Sat or Sunday, unfortunately armv7hl is as usually terribly slow, so gcc builds doesn't take 3-5 hours that it takes on sane architectures, but 22+ hours, sometimes more than a day.

At this point it's going to be next week. I'd rather wait and ensure GCC is ready.

We are updating to the final released glibc to 2.25 right now, and this will be in F26.

The schedule says the mass rebuild is starting today?

The mass rebuild will start once we are sure we have all dependencies ready. do you want to ensure that glibc 2.25 was included in the rebuild? if so we have a little time afaik, however your change should have been submitted and ready to go already

As of right now glibc 2.25 for Fedora 26, based on the final upstream 2.25 (released 3 days ago) is building in rawhide (https://koji.fedoraproject.org/koji/taskinfo?taskID=17682458), and scratch builds all passed already.

It is important that we use the final glibc 2.25 for mass rebuilds, just as important as using the right compiler.

The issue with glibc is that it forms the basis of the lowest-level ABI and API guarantees we have in the distribution. As the final glibc release approaches we keep rawhide in sync as close as possible. Therefore day-to-day we know exactly where we stand. The final touches are that we move Rawhide to the official release tag such that upstream provides Fedora all the assurances of an ABI guarantee. In reality we are 99.99% assured anyway, but we like to meet the letter of the guarantee by being on the right branch. That's what we're doing here.

So I'm OK with the mass rebuild as long as glibc-2.25-1 is used.

@jakub do you consider gcc ready yet?

I'd say yes, the silent wrong-code issues I've been aware should be resolved by now, if there is some ICE or similar bug left, it can be resolved post mass rebuild (the package will fail to build).

The glibc-2.25-1 is built for all arches except s390x, the s390-koji seems to be behind and has not built glibc-2.25-1 yet.

<dhorak> codonell: the shadow process is running, the right glibc version will be used when the mass rebuild will be processed in the s390 koji instance

Looks like s390x is good for glibc.

@codonell where is the approved change for glibc? if its something that we need to be tracking as part of the dependencies or reasons why we do a mass rebuild we have to have it as a change so we can track and consider it.

@ausil the Change is here: https://fedoraproject.org/wiki/Changes/GLIBC225 , currently in the "Announced" state, waiting till Monday Feb-13 to be moved to 'Ready for FESCo' state.

@jkurik then it can not be a factor or consideration in the mass rebuild as that deadline passed already, it may just be semantics. However we need to do better about making sure that we report correctly what is needing the mass rebuild. half the changes we have listed are only listed because I went though all of the changes and plucked them out. they had working like "releng does not need to do anything as a mass rebuild is already planned"

@codonell I am really confused, your change has
"Release engineering: In general coordination with release engineering is not required. A mass rebuild is not required. "
but you demanded that we wait until you are ready for the mass rebuild. it is one or the other, you need it or not.

@codonell I am really confused, your change has
"Release engineering: In general coordination with release engineering is not required. A mass rebuild is not required. "
but you demanded that we wait until you are ready for the mass rebuild. it is one or the other, you need it or not.

My apologies this is a mistake in the template. You always need to wait for an ABI-stable glibc. We can ignore this, but then we might suffer serious ABI consquences in a released version of Fedora.

@codonell I am really confused, your change has
"Release engineering: In general coordination with release engineering is not required. A mass rebuild is not required. "
but you demanded that we wait until you are ready for the mass rebuild. it is one or the other, you need it or not.

My apologies this is a mistake in the template. You always need to wait for an ABI-stable glibc. We can ignore this, but then we might suffer serious ABI consquences in a released version of Fedora.

I want to make it clear that I consider it the responsibility of the Fedora glibc team to ensure that everything works smoothly, and we can help iron out any process issues. However, fundamentally, like the kernel, glibc needs to be lined up before the distribution is released.

I have clarified the system-wide change request to read:
https://fedoraproject.org/wiki/Changes/GLIBC225#Scope
Release engineering: All Fedora releases must be released using a released version of glibc. The Fedora glibc team is responsible for ensuring that Fedora Rawhide stabilizes ABI before a Fedora release, or that after the branch that the Fedora release is rebased (a very small rebase) to the final released version. This is a requirement for Fedora to inherit the ABI and API guarantees provided by upstream. If a mass rebuild is required by glibc or other components, the Fedora glibc team will ensure coordination with release engineering such that a mass rebuild uses the released version of glibc to fix any last minute ABI changes. The GNU C Library (glibc) does not require a mass rebuild for this release.

I will use this as our template text going forward.

Output of mass-tag.py

https://mohanboddu.fedorapeople.org/mass-tag-f26-rebuild.out