go-rpm-macros

Clone
Source Code
GIT

#43 rpm/macros.d: use linkmode=external
Merged 3 months ago by ngompa. Opened 4 months ago by bluca.
bluca/go-rpm-macros external_linker  into  master

file modified
+1 -1 
@@ -29,7 +29,7 @@ 
 

  #
 

  #    %make GOBUILDFLAGS="%gobuildflags"
 

  #
 

- %gobuildflags() %{expand:%{gocompilerflags} -tags="rpm_crashtraceback ${BUILDTAGS:-}" -ldflags "${LDFLAGS:-} %{?currentgoldflags} -B 0x$(head -c20 /dev/urandom|od -An -tx1|tr -d ' \\n') -compressdwarf=false -extldflags '%{build_ldflags} %{?__golang_extldflags}'" -a -v -x}
 

+ %gobuildflags() %{expand:%{gocompilerflags} -tags="rpm_crashtraceback ${BUILDTAGS:-}" -ldflags "${LDFLAGS:-} %{?currentgoldflags} -B 0x$(head -c20 /dev/urandom|od -An -tx1|tr -d ' \\n') -compressdwarf=false -linkmode=external -extldflags '%{build_ldflags} %{?__golang_extldflags}'" -a -v -x}
 

  

  # Turn off Go modules
 

  %gomodulesmode GO111MODULE=off

The golang linker does not support a variety of flags and features,
in particular it does not allow us to set add a .note.package section.
Use linkmode=external so that it works out of the box.

RFC, as I am really not a golang expert, and I do not know if this is the right thing/acceptable thing to do. Looking for feedback.
For context: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/O6HNXXSILRE7WJPZQTDV3JGHEOXK7SGG/

For reference, appears to be the recommended setup in Arch: https://wiki.archlinux.org/title/Go_package_guidelines#Flags_and_build_options

@alexsaezm and @jcajka, is there any reason that we shouldn't add this to Fedora? It looks like it's already present in the c9s go-rpm-macros package.
Edited 4 months ago by gotmax23

I don't think that universally forcing external linker is the right thing. @bluca What issue are you trying to solve? It is not clear to me from the context messages.

This is regarding this change for F36: https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects

The golang linker doesn't seem to support linker scripts (or if it does, it doesn't seem to work, without any visible feedback), so the flags set by the rpm macro is simply ignored.

By changing to use the default distro linker, like other distros are doing, the problem is solved and the .note.package is added as expected to golang binaries.

@bluca do you have upstream issue for the Go linker issue?

Sorry, no - I am really not a golang expert or user. A quick google search would suggest it's just not supported:

https://stackoverflow.com/questions/69111979/using-custom-linker-script-with-go-build
https://groups.google.com/g/golang-nuts/c/Upp5MO0hsXA

Sorry, no - I am really not a golang expert or user. A quick google search would suggest it's just not supported:

https://stackoverflow.com/questions/69111979/using-custom-linker-script-with-go-build
https://groups.google.com/g/golang-nuts/c/Upp5MO0hsXA

IMO this probably should be fixed on the Go's link side. Thinking about the change proposal all you would need is a way to set/add arbitrary note section to some value. More generic analogue to the -B option. And the value can be easily generated by RPM macro, outside linker right? @dbenoit @alexsaezm What do you think?

From experience, trying to have dozens of different implementations, one for each "special case", ends up in an unmanageable mess. With the approach we have taken, everything uses the same mechanism, and for the vast majority it "just works", and when it doesn't it's most of the times because of pre-existing hidden issues that are brought to light - which is a good thing, as then they can be fixed (eg: variables leaking into pkg-config-like executables).
For example in Rust too the macros were changed so that this option would work as expected: https://pagure.io/fedora-rust/rust2rpm/pull-request/179

What issues do you see in using the default distro linker instead of golang's? I think it was mentioned that this already happens for c9s?

@bluca I don't think that rust change is relevant as rust is clang based
ldscript for this seems to me a bit of over-engineering, but I guess the ship has already sailed. I think it would be good to further discuss this in next Go sig meeting, if external linker is acceptable for all. I will open ticket.
@gotmax23 @dbenoit Do you know why c9s is defaulting to external linking? I guess FIPS/boringcrypto?

@bluca I don't think that rust change is relevant as rust is clang based

Sure, it was just meant as an example of how similar issues were fixed elsewhere.

ldscript for this seems to me a bit of over-engineering, but I guess the ship has already sailed.

Well, actually this is quite simple and straightforward in the end, it would have been over-engineering to design a different solution for every language, package class, build system, build flavour, compiler, linker, etc :-)

ldscript for this seems to me a bit of over-engineering, but I guess the ship has already sailed.

Well, actually this is quite simple and straightforward in the end, it would have been over-engineering to design a different solution for every language, package class, build system, build flavour, compiler, linker, etc :-)

But it is still bound to particular tool-chain technology. My naive took would be that there should be more generic way, adding the note to already built ELFs, not saying that it would be easy(ier) :).

Sadly, I think this implementation is as simple as it gets. It was more complex initially. :sweat_smile:

Yep, in practice post-processing or other inline alternatives were all more complex and subject to breakages due to the differences in setup/build/etc across the whole package base.

I did a test rebuild with this enabled with packages that BR:go-rpm-macros and are archful:
https://copr.fedorainfracloud.org/coprs/zbyszek/go-package-notes/builds/

360 packages total (including go-rpm-macros themselves)
54 packages failed initially.
2 out of those were using ld.gold and I pushed a commit to add %global _package_notes_linker gold, which effectively disables package notes and fixes the build issue: rclone and golang-k8s-sample-cli-plugin.
Other failures are clearly unrelated, except for four packages for which I wasn't sure of the reason. Those are marked as "unclear error". I'd be grateful if someone who groks go could take a look.

cadvisor    fedora-rawhide-x86_64   3529253 failed  0.40.0-3
cadvisor    fedora-rawhide-aarch64  3529253 failed  0.40.0-3
%build:
cd /builddir/build/BUILD/cadvisor-0.40.0
/usr/lib/golang/pkg/tool/linux_amd64/compile -o $WORK/b562/_pkg_.a -trimpath "$WORK/b562=>" -p github.com/google/cadvisor/utils/cloudinfo/aws -complete -installsuffix shared -buildid IDSVCJHPW7L9JSohE-DZ/IDSVCJHPW7L9JSohE-DZ -goversion go1.18beta2 -shared -c=2 -nolocalimports -importcfg $WORK/b562/importcfg -pack ./_build/src/github.com/google/cadvisor/utils/cloudinfo/aws/aws.go
/usr/lib/golang/pkg/tool/linux_amd64/buildid -w $WORK/b562/_pkg_.a # internal
cp $WORK/b562/_pkg_.a /builddir/.cache/go-build/d1/d190520cf7782537a37c7ff34936888f16cc6574da2c77f694290155193e12d1-d # internal
error: Bad exit status from /var/tmp/rpm-tmp.d88LDT (%build)
unclear error, I have no idea what is going wrong here.
f36-rebuild passed

deepin-gir-generator    fedora-rawhide-aarch64  3529262 failed  2.0.2-11
deepin-gir-generator    fedora-rawhide-x86_64   3529262 failed  2.0.2-11
%prep: sed -i 's#go build#go build -buildmode pie -compiler gc -tags=rpm_crashtraceback' ' -ldflags ' -X pkg.deepin.io/gir/version.tag=2.0.2 -X pkg.deepin.io/gir/version=2.0.2 -B 0xcf6ce87e43fe3789ea4e6245cf79202e47a4f39b -compressdwarf=false -linkmode=external -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/.package_note-deepin-gir-generator-2.0.2-11.fc37.x86_64.ld  -a -v -x#' Makefile
sed: invalid option -- 'X'
I expect that this is related to the the change to set flags early, not package notes.

etcd    fedora-rawhide-x86_64   3529267 failed  3.5.0-5
etcd    fedora-rawhide-aarch64  3529267 failed  3.5.0-5
%check: FAIL    go.etcd.io/etcd/api/etcdserverpb    0.008s

exercism    fedora-rawhide-x86_64   3529268 failed  3.0.13-7
exercism    fedora-rawhide-aarch64  3529268 failed  3.0.13-7
%check: FAIL    github.com/exercism/cli/cmd 0.027s

git-lfs fedora-rawhide-x86_64   3529273 failed  3.0.2-2
%check: fails in tests on on arch

goddi   fedora-rawhide-x86_64   3529279 failed  1.2-6
goddi   fedora-rawhide-aarch64  3529279 failed  1.2-6
%check: FAIL    github.com/tinygo-org/tinygo    0.143s

golang-gioui    fedora-rawhide-x86_64   3529349 failed  0-0.4
golang-gioui    fedora-rawhide-aarch64  3529349 failed  0-0.4
%check: pkg-config --cflags  -- egl wayland-egl wayland-client wayland-cursor x11 xkbcommon xkbcommon-x11 x11-xcb xcursor xfixes
Package wayland-egl was not found in the pkg-config search path.
Perhaps you should add the directory containing `wayland-egl.pc'
to the PKG_CONFIG_PATH environment variable
Package 'wayland-egl', required by 'virtual:world', not found
Package 'wayland-client', required by 'virtual:world', not found
Package 'wayland-cursor', required by 'virtual:world', not found
Package 'xkbcommon-x11', required by 'virtual:world', not found
Package 'xcursor', required by 'virtual:world', not found
Package 'xfixes', required by 'virtual:world', not found

golang-github-acme-lego fedora-rawhide-x86_64   3529351 failed  4.4.0-2
golang-github-acme-lego fedora-rawhide-aarch64  3529351 failed  4.4.0-2
%build: unclear error

golang-github-aliyun-cli    fedora-rawhide-aarch64  3529355 failed  3.0.104-3
golang-github-aliyun-cli    fedora-rawhide-x86_64   3529355 failed  3.0.104-3
%check: ./metas.go:7577:25: fmt.Errorf call has arguments but no formatting directives

golang-github-appc-goaci    fedora-rawhide-aarch64  3529360 failed  0.1.1-7
golang-github-appc-goaci    fedora-rawhide-x86_64   3529360 failed  0.1.1-7
%check: ./asset.go:179:11: fmt.Errorf call needs 1 arg but has 2 args

golang-github-atotto-clipboard  fedora-rawhide-aarch64  3529363 failed  0.1.4-2
golang-github-atotto-clipboard  fedora-rawhide-x86_64   3529363 failed  0.1.4-2
%check: clipboard_test.go:18: No clipboard utilities available. Please install xsel, xclip, wl-clipboard or Termux:API add-on for termux-clipboard-get/set.
With xsel added, the tests fail on unexpected empty output. Maybe settings up a fake server would help…

golang-github-cheekybits-genny  fedora-rawhide-aarch64  3529375 failed  1.0.0-8
golang-github-cheekybits-genny  fedora-rawhide-x86_64   3529375 failed  1.0.0-8
%check: FAIL    github.com/cheekybits/genny/parse   0.075s

golang-github-cloudflare-cfssl  fedora-rawhide-aarch64  3529383 failed  1.6.0-2
golang-github-cloudflare-cfssl  fedora-rawhide-x86_64   3529383 failed  1.6.0-2
%check: sign_test.go:48: {"code":9300,"message":"x509: cannot verify signature: insecure algorithm SHA1-RSA (temporarily override with GODEBUG=x509sha1=1)"}

golang-github-dave-dst  fedora-rawhide-aarch64  3529396 failed  0.26.2-3
golang-github-dave-dst  fedora-rawhide-x86_64   3529396 failed  0.26.2-3
%check: util_test.go:13:2: cannot find package "gopkg.in/src-d/go-billy.v4/memfs" in any of:

golang-github-deislabs-oras fedora-rawhide-aarch64  3529399 failed  0.11.1-4
golang-github-deislabs-oras fedora-rawhide-x86_64   3529399 failed  0.11.1-4
%build: "github.com/docker/distribution/registry/client/auth/challenge".Manager does not implement "github.com/distribution/distribution/v3/registry/client/auth/challenge".Manager (wrong type for GetChallenges method)

golang-github-distribution-3    fedora-rawhide-aarch64  3529401 failed  2.8.0~beta.1-1
%check: FAIL    github.com/distribution/distribution/v3/registry/handlers   600.054s
arch-specific, timeout?

golang-github-docker-distribution   fedora-rawhide-aarch64  3529402 failed  2.7.1-8
golang-github-docker-distribution   fedora-rawhide-x86_64   3529402 failed  2.7.1-8
%check: FAIL    github.com/docker/distribution/registry/storage 6.931s

golang-github-docker-slim   fedora-rawhide-x86_64   3529403 failed  1.36.4-5
golang-github-docker-slim   fedora-rawhide-aarch64  3529403 failed  1.36.4-5
%build: ./wsclient.go:187:4: fmt.Print call has possible formatting directive %v

golang-github-docker-swarmkit   fedora-rawhide-aarch64  3529404 failed  1.12.0-21
golang-github-docker-swarmkit   fedora-rawhide-x86_64   3529404 failed  1.12.0-21
%check: FAIL    github.com/docker/swarmkit/manager/orchestrator/jobs/replicated [build failed]

golang-github-facebook-ent  fedora-rawhide-x86_64   3529417 failed  0.5.4-4
golang-github-facebook-ent  fedora-rawhide-aarch64  3529417 failed  0.5.4-4
%check: FAIL    github.com/facebook/ent/dialect/gremlin/encoding/graphson   0.009s

golang-github-gogo-protobuf fedora-rawhide-x86_64   3529429 failed  1.3.2-4
%check:   stdtypespb_test.go:611: 1:39: expected operand, found '<' (and 1 more errors)
(But it's arch specific?)

golang-github-gopherjs  fedora-rawhide-x86_64   3529441 failed  0-0.16
golang-github-gopherjs  fedora-rawhide-aarch64  3529441 failed  0-0.16
%build: ./utils.go:220:12: conversion from int to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)

golang-github-grpc-ecosystem-gateway    fedora-rawhide-aarch64  3529443 failed  1.16.0-4
golang-github-grpc-ecosystem-gateway    fedora-rawhide-x86_64   3529443 failed  1.16.0-4
%check: FAIL    github.com/grpc-ecosystem/grpc-gateway/codegenerator    0.003s

golang-github-jung-kurt-gofpdf  fedora-rawhide-x86_64   3529462 failed  2.17.2-4
golang-github-jung-kurt-gofpdf  fedora-rawhide-aarch64  3529462 failed  2.17.2-4
%check: Get "https://github.com/jung-kurt/gofpdf/blob/master/image/gofpdf.png?raw=true": dial tcp: lookup github.com: Temporary failure in name resolution

golang-github-krishicks-yaml-patch  fedora-rawhide-aarch64  3529466 failed  0.0.10-6
golang-github-krishicks-yaml-patch  fedora-rawhide-x86_64   3529466 failed  0.0.10-6
%check: patch_test.go:9:2: cannot find package "github.com/onsi/ginkgo" in any of:

golang-github-latex fedora-rawhide-x86_64   3529468 failed  0-0.4
golang-github-latex fedora-rawhide-aarch64  3529468 failed  0-0.4
%check: drawimg_test.go:12:2: cannot find package "github.com/go-fonts/latin-modern/lmroman12bold" in any of:

golang-github-maruel-panicparse fedora-rawhide-x86_64   3529479 failed  1.6.0-4
golang-github-maruel-panicparse fedora-rawhide-aarch64  3529479 failed  1.6.0-4
%check: FAIL    github.com/maruel/panicparse/internal   0.448s

golang-github-mattermost-xml-roundtrip-validator    fedora-rawhide-x86_64   3529480 failed  0-0.4
golang-github-mattermost-xml-roundtrip-validator    fedora-rawhide-aarch64  3529480 failed  0-0.4
%check: FAIL    github.com/mattermost/xml-roundtrip-validator   0.008s

golang-github-mmarkdown-mmark   fedora-rawhide-x86_64   3529487 failed  2.2.10-4
golang-github-mmarkdown-mmark   fedora-rawhide-aarch64  3529487 failed  2.2.10-4
%check: FAIL    github.com/mmarkdown/mmark  0.042s

golang-github-opencontainers-runtime-tools  fedora-rawhide-x86_64   3529507 failed  0.9.0-7
golang-github-opencontainers-runtime-tools  fedora-rawhide-aarch64  3529507 failed  0.9.0-7
%check: * Get "https://raw.githubusercontent.com/opencontainers/runtime-spec/v1.0.2-dev/schema/config-schema.json": dial tcp: lookup raw.githubusercontent.com: Temporary failure in name resolution

golang-github-quay-clair-4  fedora-rawhide-aarch64  3529527 failed  4.1.5-5
golang-github-quay-clair-4  fedora-rawhide-x86_64   3529527 failed  4.1.5-5
%check: ./server.go:101:23: undefined: trace.Config

golang-github-quay-claircore    fedora-rawhide-x86_64   3529528 failed  0.5.4-4
golang-github-quay-claircore    fedora-rawhide-aarch64  3529528 failed  0.5.4-4
%check: ./cvss_test.go:214:5: (*testing.common).Errorf does not support error-wrapping directive %w

golang-github-rakyll-statik fedora-rawhide-x86_64   3529530 failed  0.1.7-4
golang-github-rakyll-statik fedora-rawhide-aarch64  3529530 failed  0.1.7-4
%build: ./statik.go:376:2: fmt.Println arg list ends with redundant newline

golang-github-rogpeppe-internal fedora-rawhide-aarch64  3529534 failed  1.8.0-2
golang-github-rogpeppe-internal fedora-rawhide-x86_64   3529534 failed  1.8.0-2
%build failed, no idea why

golang-github-theupdateframework-notary fedora-rawhide-x86_64   3529550 failed  0.7.0-2
golang-github-theupdateframework-notary fedora-rawhide-aarch64  3529550 failed  0.7.0-2
%build: "github.com/docker/distribution/registry/client/auth/challenge".Manager does not implement "github.com/distribution/distribution/v3/registry/client/auth/challenge".Manager (wrong type for GetChallenges method)

golang-github-uber-jaeger-client    fedora-rawhide-aarch64  3529558 failed  2.25.0-4
golang-github-uber-jaeger-client    fedora-rawhide-x86_64   3529558 failed  2.25.0-4
%check: ./numeric.go:72:12: conversion from int64 to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)

golang-github-yuin-gopher-lua   fedora-rawhide-aarch64  3529566 failed  0-0.18
golang-github-yuin-gopher-lua   fedora-rawhide-x86_64   3529566 failed  0-0.18
%check: ./lexer.go:258:19: conversion from int to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)

golang-gvisor   fedora-rawhide-aarch64  3529579 failed  20211129.0-3
golang-gvisor   fedora-rawhide-x86_64   3529579 failed  20211129.0-3
%build: proc/exec.go:241:114: not enough arguments in call to e.parent.Platform.CopyConsole

golang-honnef-tools fedora-rawhide-aarch64  3529580 failed  2021.1-2
golang-honnef-tools fedora-rawhide-x86_64   3529580 failed  2021.1-2
%check: FAIL    honnef.co/go/tools/analysis/facts/nilness   0.448s

golang-k8s-sample-cli-plugin    fedora-rawhide-aarch64  3529588 failed  1.22.0-2
arch-specific
/usr/bin/ld.gold: error: /builddir/build/BUILD/sample-cli-plugin-kubernetes-1.22.0/.package_note-golang-k8s-sample-cli-plugin-1.22.0-2.fc37.aarch64.ld:47:8: syntax error, unexpected STRING
Disabled.

golang-modernc-golex    fedora-rawhide-aarch64  3529590 failed  1.0.1-4
golang-modernc-golex    fedora-rawhide-x86_64   3529590 failed  1.0.1-4
f36-rebuild failed
%check: --- FAIL: Example_completeGeneratedProgram (0.00s)

golang-nanomsg-mangos-3 fedora-rawhide-x86_64   3529593 failed  3.2.1-3
arch-specific
%check: must.go:28: Failed at /builddir/build/BUILD/mangos-3.2.1/_build/src/go.nanomsg.org/mangos/v3/internal/test/certs_test.go:26: Error is not nil: x509: cannot verify signature: insecure algorithm SHA1-RSA (temporarily override with GODEBUG=x509sha1=1)

golang-sigs-k8s-application fedora-rawhide-x86_64   3529597 failed  0.8.3-4
golang-sigs-k8s-application fedora-rawhide-aarch64  3529597 failed  0.8.3-4
f36-rebuild failed too
unclear error

golang-sigs-k8s-kustomize   fedora-rawhide-x86_64   3529599 failed  4.0.3-3
golang-sigs-k8s-kustomize   fedora-rawhide-aarch64  3529599 failed  4.0.3-3
f36-rebuild failed too
%check: exec: "git": executable file not found in $PATH
adding git doesn't solve the issue

golang-vitess   fedora-rawhide-x86_64   3529604 failed  11.0.1-2
golang-vitess   fedora-rawhide-aarch64  3529604 failed  11.0.1-2
%check: ERROR: logging before flag.Parse: E0218 19:12:13.563544  132874 syslogger.go:149] can't connect to syslog

golang-x-build  fedora-rawhide-x86_64   3529605 failed  0-0.20
golang-x-build  fedora-rawhide-aarch64  3529605 failed  0-0.20
%check: ./store_test.go:61:3: (*testing.common).Errorf does not support error-wrapping directive %w

golang-x-mobile fedora-rawhide-aarch64  3529608 failed  0-0.10
golang-x-mobile fedora-rawhide-x86_64   3529608 failed  0-0.10
%build: x11.c:40:9: error: unknown type name 'XVisualInfo'; did you mean 'VisualID'?

golie   fedora-rawhide-aarch64  3529283 failed  0.2.1-4
golie   fedora-rawhide-x86_64   3529283 failed  0.2.1-4
%check: ./xml.go:899:24: conversion from int to string yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)


httpdump    fedora-rawhide-x86_64   3529293 failed  0-0.5
httpdump    fedora-rawhide-aarch64  3529293 failed  0-0.5
%build: _build/src/github.com/hsiafan/httpdump/http_traffic_handler.go:282:16: undefined: filex.WriteAllFromReader

hugo    fedora-rawhide-aarch64  3529295 failed  0.80.0-5
hugo    fedora-rawhide-x86_64   3529295 failed  0.80.0-5
%check: FAIL    github.com/gohugoio/hugo/hugolib    7.778s

rclone  fedora-rawhide-aarch64  3529319 failed  1.57.0-2
arch-specific, gold, fix submitted

reg fedora-rawhide-x86_64   3529320 failed  0.16.1-6
reg fedora-rawhide-aarch64  3529320 failed  0.16.1-6
%build: _build/src/github.com/genuinetools/reg/clair/layerutil.go:73:42: cannot use mf.Layers[i] (variable of type "github.com/distribution/distribution/v3".Descriptor) as type "github.com/docker/distribution".Descriptor in assignment

syncthing   fedora-rawhide-aarch64  3529328 failed  1.18.6-2
syncthing   fedora-rawhide-x86_64   3529328 failed  1.18.6-2
unclear error
f36-rebuild failed too

tinygo  fedora-rawhide-aarch64  3529332 failed  0.22.0-5
tinygo  fedora-rawhide-x86_64   3529332 failed  0.22.0-5
%check: main.go:1118: error: requires go version 1.15 through 1.17, got go1.18
and lots of other errors.
Edited 4 months ago by zbyszek

ping - any update?

This needs to be rebased, it can't be merged due to conflicts.

rebased onto 50b3fcc
4 months ago

thank you, done

:thumbsup:

If there are no further objections, I'll merge this later today.

@ngompa. are you still planning to merge this?

Pull-Request has been merged by ngompa
3 months ago
Metadata
None
No Tags
Changes Summary 1
+1 -1
file changed
rpm/macros.d/macros.go-compilers-golang
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.