#2165 F31 System-Wide Change: Golang 1.13
Closed: Accepted 4 years ago by ignatenkobrain. Opened 4 years ago by bcotton.

Rebase of Golang package to upcoming version 1.13 in Fedora 31, including rebuild of all dependent packages(pre-release version of Go will be used for rebuild, if released version will not be available at the time of the mass rebuild).


+1

@jcajka please make sure that packages are still buildable and are not broken due to this GO111MODULE stuff and such. I am pretty sure Go SIG is aware, but just to make sure :)

@ignatenkobrain actually that is my only big concern. The noted conf changes in the proposal are actually changes that are needed atm, but it is in the @nim @eclipseo and @qulogic hands as I no longer have any direct access to any bits of the macros stack and they made it clear to me that they don't want my input.

TBH I have one more concern regarding testing this change(compared to the past devel cycles) as I have currently rather limited means of testing Go 1.13 due to the rawhide breakage related to the fesco#2120 although they have requested side tag, all the changes(most notable retires) are landing directly in rawhide breaking deps of the dependent packages. Due to this I'm strongly considering landing 1.13beta1 in to the rawhide asap so I can collect packagers feedback and react to it, while having chance to pass the feedback to the upstream prior to the release(mass-rebuild). Even though Go upstream is doing usually great job and releasing relatively bug free. I will appreciate any opinions on this approach.

And for the record I will unfortunately not be able to make it for this week's FESCO meeting. If you have any questions I will be happy to answer them here.

the meeting this week has been canceled (today is holiday in the US, tomorrow in CZ).

+1
@jcajka please make sure that packages are still buildable and are not broken due to this GO111MODULE stuff and such. I am pretty sure Go SIG is aware, but just to make sure :)

GOMODULE is OFF with the macros, there should not be any issue. Everything was rebuilt for 1.12. I don't know if the Notary thing needs to be disabled too?

@jcajka I'm finishing off the remaining packages today or tomorrow ( I need to do a check that I haven't missed any package), so the side tag will be able to be merged soon.

@jcajka The list of packages will need to be updated first as we have renamed lots of them. Note that none of the binary packages have been updated so we may experience failures with them.

@jcajka
You need to tell nim to add this to the macros I guess

By setting GOSUMDB=off and GOPROXY=direct, instead of them set to the default Google's services(or any other provider).

@jcajka
You need to tell nim to add this to the macros I guess

By setting GOSUMDB=off and GOPROXY=direct, instead of them set to the default Google's services(or any other provider).

I will do this at the compiler level(so it will be visible by all end users), but it will make sense to add it explicitly in to the stack too(if we ever change it at compiler level) please suggest this to @nim.

+1
@jcajka please make sure that packages are still buildable and are not broken due to this GO111MODULE stuff and such. I am pretty sure Go SIG is aware, but just to make sure :)

GOMODULE is OFF with the macros, there should not be any issue. Everything was rebuilt for 1.12. I don't know if the Notary thing needs to be disabled too?
@jcajka I'm finishing off the remaining packages today or tomorrow ( I need to do a check that I haven't missed any package), so the side tag will be able to be merged soon.
@jcajka The list of packages will need to be updated first as we have renamed lots of them. Note that none of the binary packages have been updated so we may experience failures with them.

This is already observable in the wild(for some time, thus my concerns). For example https://bugzilla.redhat.com/show_bug.cgi?id=1727674 just landed. Could you(as in all change proposal owners) please work on fixing those with the respective maintainers?

@jcajka
You need to tell nim to add this to the macros I guess
By setting GOSUMDB=off and GOPROXY=direct, instead of them set to the default Google's services(or any other provider).

@jcajka

Adding such flags makes no sense since right know we disable modules altogether because the tooling is not ready both Fedora and upstream side.

Just to make things 100% clear, moving to Go modules in Fedora would ideally require fixing
https://gist.github.com/nim-nim/9f72827d7cba9637dfe1492364ad9adf

upstream, in Golang 1.14 or golang/x/mod

In particular, if the test requirements are not separated in the upstream module design, as requested in
https://github.com/golang/go/issues/31300

there will be a huge fallout Fedora-side – switching to modules will make all test deps mandatory.

And projects are assuming Go modules now so unless upstream module stack is fixed, we will have this mass Fedora Go package breakage within a year.

@ignatenkobrain but it is in the @nim @eclipseo and @qulogic hands as I no longer have any direct access to any bits of the macros stack and they made it clear to me that they don't want my input.

@jcajka

You're admin of
https://pagure.io/go-rpm-macros
and
https://src.fedoraproject.org/rpms/go-rpm-macros

And even if you weren’t nothing would stop you from submitting PRs like I do for redhat-rpm-config.

NOTHING stops you from improving things, your input is taken into account, just because we’re all too busy to implement every single idea you can’t be bothered with yourself does not mean we do not read your messages.

@ignatenkobrain but it is in the @nim @eclipseo and @qulogic hands as I no longer have any direct access to any bits of the macros stack and they made it clear to me that they don't want my input.

@jcajka
You're admin of
https://pagure.io/go-rpm-macros
and
https://src.fedoraproject.org/rpms/go-rpm-macros
And even if you weren’t nothing would stop you from submitting PRs like I do for redhat-rpm-config.

Thank you for adding me and others today, although I can't at the moment commit to maintain your stack/change. Also I'm not sure if Go-SIG(any Go sig member can access/commit in to it) level access to such a "core" component is good idea. As it has potential to break whole distribution(build root). But I guess it is just matter of opinion.

NOTHING stops you from improving things, your input is taken into account, just because we’re all too busy to implement every single idea you can’t be bothered with yourself does not mean we do not read your messages.

I'm sure that you are reading my messages because they usually trigger some responses.

@jcajka
You're admin of
https://pagure.io/go-rpm-macros
and
https://src.fedoraproject.org/rpms/go-rpm-macros
And even if you weren’t nothing would stop you from submitting PRs like I do for redhat-rpm-config.

Thank you for adding me and others today

I added you for the spec part, the macro files themselves, where compiler settings are set, are in https://pagure.io/go-rpm-macros and you've been admin of this for months now.

@jcajka
You're admin of
https://pagure.io/go-rpm-macros
and
https://src.fedoraproject.org/rpms/go-rpm-macros
And even if you weren’t nothing would stop you from submitting PRs like I do for redhat-rpm-config.
Thank you for adding me and others today

I added you for the spec part, the macro files themselves, where compiler settings are set, are in https://pagure.io/go-rpm-macros and you've been admin of this for months now.

This is not a part of Fedora, but I would rather stop here as we are going a way of topic for this ticket.

@jcajka
You need to tell nim to add this to the macros I guess
By setting GOSUMDB=off and GOPROXY=direct, instead of them set to the default Google's services(or any other provider).

@jcajka
Adding such flags makes no sense since right know we disable modules altogether because the tooling is not ready both Fedora and upstream side.

I will respectfully disagree(and technically nitpick, those are env vars). IMO distribution macros/flags/envars should be as much as explicit as possible, rather than implicit to allow unambiguous interpretations, avoid unintended breakages(when defaults changes) and to clearly document intended behavior and context for it.

Just to make things 100% clear, moving to Go modules in Fedora would ideally require fixing
https://gist.github.com/nim-nim/9f72827d7cba9637dfe1492364ad9adf
upstream, in Golang 1.14 or golang/x/mod
In particular, if the test requirements are not separated in the upstream module design, as requested in
https://github.com/golang/go/issues/31300
there will be a huge fallout Fedora-side – switching to modules will make all test deps mandatory.
And projects are assuming Go modules now so unless upstream module stack is fixed, we will have this mass Fedora Go package breakage within a year.

Metadata Update from @churchyard:
- Issue tagged with: meeting

4 years ago

I see there is a huge list of dependencies but there is no proposed way how they will be dealt with, there are not part of Contingency plan or "Upgrade/compatibility impact".

Please add a bit more information on this.

Will Change Owners take care of fixing dependencies? If maintainers expected to do it, what will happen with packages not fixed by the time of release deadlines?

Are there known packages out of the list of dependencies with higher risks of incompatibility? Can you mention them in the Change Proposal? Are there any packages which considered to be critical for the entire effort, and which if not ported by deadline will cause the revert of the entire change?

@nim @jcajka @eclipseo Can one of you also summarize the current discussion, should it be considered blocker for a change?

Summary: we need @nim to explicitly set GOSUMDB=off and GOPROXY=direct at the macros level. Both me and @jcajka have access to the macros but I would prefer @nim to do it as he has a more complete understanding of the macros stack. This is not a blocker per se, because it will be also set at the compiler level by @jcajka .

We have discussed this today on FESCo meeting:

AGREED: Wait a week and get answers for questions in the ticket (ignatenkobrain, 15:11:00)

Metadata Update from @ignatenkobrain:
- Issue untagged with: meeting

4 years ago

I see there is a huge list of dependencies but there is no proposed way how they will be dealt with, there are not part of Contingency plan or "Upgrade/compatibility impact".
Please add a bit more information on this.
Will Change Owners take care of fixing dependencies? If maintainers expected to do it, what will happen with packages not fixed by the time of release deadlines?
Are there known packages out of the list of dependencies with higher risks of incompatibility? Can you mention them in the Change Proposal? Are there any packages which considered to be critical for the entire effort, and which if not ported by deadline will cause the revert of the entire change?

Those are not mentioned as there shoudn't be an issue(and haven't been many in past, compare to something like C), assuming there are no packages that are not relying on undocumented or undefined Go behavior. More in general I'm not sure what are you asking here. Could you bit more clarify/propose improvements to the proposal?

For record I'm open for improvements of the change proposal, but be honest in context of the past proposal to rebase Go(and their executions) and for example fesco#2120 , which doesn't deal with any details that you are mentioning(not even listing all package that will be affected by it and hand wavy contingency plan, not corresponding to the change scope) , and is in fact bigger in scope than this change proposal. FESCO is/has been fine with it(even accepting it as self contained change). This seems to me kind of arbitrary.

Summary: we need @nim to explicitly set GOSUMDB=off and GOPROXY=direct at the macros level. Both me and @jcajka have access to the macros but I would prefer @nim to do it as he has a more complete understanding of the macros stack. This is not a blocker per se, because it will be also set at the compiler level by @jcajka .

I second that especially as GOMODULE is/should be OFF in the new macros stack(this is really only thing that has potential to cause any issues at distro level). Although I have been given access to the package(I could have open PR anyway). I don't feel confident to contribute to it. I would defer to @nim implement it in the way he likes/accepts(and in short term it is not issue as I will resolve that at "compiler" level).

For the record @eclipseo has merged back the fesco#2120 side tag now, which should reduce the amount of "artificially" broken packages. This allows me to again gather more information on what could be broken by the rebase of Go. Hopefully there will be not much fallout left.

I see there is a huge list of dependencies but there is no proposed way how they will be dealt with, there are not part of Contingency plan or "Upgrade/compatibility impact".
Please add a bit more information on this.
Will Change Owners take care of fixing dependencies? If maintainers expected to do it, what will happen with packages not fixed by the time of release deadlines?
Are there known packages out of the list of dependencies with higher risks of incompatibility? Can you mention them in the Change Proposal? Are there any packages which considered to be critical for the entire effort, and which if not ported by deadline will cause the revert of the entire change?

Hm... maybe you have asked how broken package will be fixed after the contingency being triggered? In that case by simply rebuilding them(against the contingency version of Go) as Go is "statically"(in Go sense) linked and ABI has no practical impact in this case(and compiler level/stdlib API is stable and backwards compatible). Is that what have you been asking?

Those are not mentioned as there shoudn't be an issue(and haven't been many in past, compare to something like C), assuming there are no packages that are not relying on undocumented or undefined Go behavior. More in general I'm not sure what are you asking here. Could you bit more clarify/propose improvements to the proposal?
For record I'm open for improvements of the change proposal, but be honest in context of the past proposal to rebase Go(and their executions) and for example fesco#2120 , which doesn't deal with any details that you are mentioning(not even listing all package that will be affected by it and hand wavy contingency plan, not corresponding to the change scope) , and is in fact bigger in scope than this change proposal. FESCO is/has been fine with it(even accepting it as self contained change). This seems to me kind of arbitrary.

Summary: we need @nim to explicitly set GOSUMDB=off and GOPROXY=direct at the macros level. Both me and @jcajka have access to the macros but I would prefer @nim to do it as he has a more complete understanding of the macros stack. This is not a blocker per se, because it will be also set at the compiler level by @jcajka .

I second that especially as GOMODULE is/should be OFF in the new macros stack(this is really only thing that has potential to cause any issues at distro level). Although I have been given access to the package(I could have open PR anyway). I don't feel confident to contribute to it. I would defer to @nim implement it in the way he likes/accepts(and in short term it is not issue as I will resolve that at "compiler" level).

And yet you feel confident enough to invent problems and post them authoritatively everywhere, without checking your facts, and without listening to the answers we give you when we learn of it (another example here: “GOMODULE is/should be OFF in the new macros stack” → if you can’t take at face value what we tell you in previous messages, can’t you at least read current package build logs instead of FUD-ing in FESCO tickets?).

In fact the only reason this change that you are the owner of is stuck is because you could not resist the urge to throw dirt about fesco#2120 in another change proposal, confusing FESCO.

And you feel entitled to request work from others that are not in the change owner list, while not only you refused to contribute to fesco#2120 but you made every possible effort to make the work of the people who worked on it more difficult, and disparaged it in every communication channel you could think of.

Can't you understand the problem? Community work is a two-way street.

and for example fesco#2120 , which doesn't deal with any details that you are mentioning(not even listing all package that will be affected by it and hand wavy contingency plan, not corresponding to the change scope) , and is in fact bigger in scope than this change proposal.

The only reason you feel fesco#2120 over-reached is that you repeatedly refused to believe Go packages in Fedora had degraded to a very sorry state, making a deep and painful clean up inevitable. IIRC you accused me of FUD-ing when https://fedoraproject.org/wiki/More_Go_packaging stated it plainly for everyone to see. fesco#2120 is just an updated way to make this change happen, it took a year and a half of work, a lot of which was consumed by obstructions.

Summary: we need @nim to explicitly set GOSUMDB=off and GOPROXY=direct at the macros level. Both me and @jcajka have access to the macros but I would prefer @nim to do it as he has a more complete understanding of the macros stack. This is not a blocker per se, because it will be also set at the compiler level by @jcajka .
I second that especially as GOMODULE is/should be OFF in the new macros stack(this is really only thing that has potential to cause any issues at distro level). Although I have been given access to the package(I could have open PR anyway). I don't feel confident to contribute to it. I would defer to @nim implement it in the way he likes/accepts(and in short term it is not issue as I will resolve that at "compiler" level).

And yet you feel confident enough to invent problems and post them authoritatively everywhere, without checking your facts, and without listening to the answers we give you when we learn of it (another example here: “GOMODULE is/should be OFF in the new macros stack” → if you can’t take at face value what we tell you in previous messages, can’t you at least read current package build logs instead of FUD-ing in FESCO tickets?).
In fact the only reason this change that you are the owner of is stuck is because you could not resist the urge to throw dirt about fesco#2120 in another change proposal, confusing FESCO.
And you feel entitled to request work from others that are not in the change owner list, while not only you refused to contribute to fesco#2120 but you made every possible effort to make the work of the people who worked on it more difficult, and disparaged it in every communication channel you could think of.
Can't you understand the problem? Community work is a two-way street.

and for example fesco#2120 , which doesn't deal with any details that you are mentioning(not even listing all package that will be affected by it and hand wavy contingency plan, not corresponding to the change scope) , and is in fact bigger in scope than this change proposal.

The only reason you feel fesco#2120 over-reached is that you repeatedly refused to believe Go packages in Fedora had degraded to a very sorry state, making a deep and painful clean up necessary. IIRC you accused me of FUD-ing when https://fedoraproject.org/wiki/More_Go_packaging stated in plainly for everyone to see. fesco#2120 is just an updated way to make this change happen, it took a year and a half of work, a lot of which was consumed by obstructions.

Could we please focus on the technical issues and facts?

@jcajka I’m posting facts. You are not. You can’t spend your time raising problems, and dismiss answers whenever they don’t go your way. Don’t raise problems if you can not deal with answers.

@jcajka I’m posting facts. You are not. You can’t spend your time raising problems, and dismiss answers whenever they don’t go your way. Don’t raise problems if you can not deal with answers.

I'm really sad that this is slipping in to the personal level. I don't want to and will not engage in flame war with you. And will reply to you with citing you "You can’t spend your time raising problems, and dismiss answers whenever they don’t go your way. Don’t raise problems if you can not deal with answers." Pleas keep this discussion technical and on topic.

@jcajka I don't want to and will not engage in flame war with you.

Then stop peppering your messages with negative hints. No one else is doing it. Ignoring you does not work because you just raise the inflammatory level in the next message.

+1 for the Change

It seems that there's general technical agreement how to proceed. Further discussion is not going to help the maintainers.

Metadata Update from @ignatenkobrain:
- Issue tagged with: meeting

4 years ago

We have discussed this on today's FESCo meeting:

AGREED: APPROVED (+7, 2, -0) (ignatenkobrain, 15:10:52)

Metadata Update from @ignatenkobrain:
- Issue close_status updated to: Accepted
- Issue status updated to: Closed (was: Open)

4 years ago

Login to comment on this ticket.

Metadata