#63 Use $SOURCE_DATE_EPOCH instead of random bytes
Merged 4 months ago by gotmax23. Opened 5 months ago by zbyszek.
zbyszek/go-rpm-macros build-reproducibility  into  master

@@ -44,7 +44,7 @@ 

  #     $LDFLAGS is supported as a deprecated alias to $GO_LDFLAGS to add additional

  #     arguments to `go build -ldflags "..."`.

  #     Please set $GO_LDFLAGS instead of $LDFLAGS.

- %gobuild_ldflags ${GO_LDFLAGS-${LDFLAGS-}} %{?currentgoldflags} -B 0x$(head -c20 /dev/urandom|od -An -tx1|tr -d ' \\n') -compressdwarf=false -linkmode=external -extldflags '%{build_ldflags} %{?__golang_extldflags}'

+ %gobuild_ldflags ${GO_LDFLAGS-${LDFLAGS-}} %{?currentgoldflags} -B 0x$(echo "%{name}-%{version}-%{release}-${SOURCE_DATE_EPOCH:-}" | sha1sum | cut -d ' ' -f1) -compressdwarf=false -linkmode=external -extldflags '%{build_ldflags} %{?__golang_extldflags}'

  # A shell-escaped version of %gobuild_ldflags.

  # Can be used to set an environment or make variable.

  # Example from the aerc package:

Using random bytes makes the builds obviously irreproducible. Use instead
$SOURCE_DATE_EPOCH as the source of the identifier, so that rebuilds of the same
sources are the same. $SOURCE_DATE_EPOCH is augmented with
%{name}-%{version}-%{release} and hashed with sha1sum to get a number that is 40
hexadecimal bytes (same as the one generated from the random bytes before). Any
hash function would do, we only need something that has good hashing properties,
there is no security component here since the input is public.

Tested on podman-5.0.0~rc3-3.fc41.x86_64.rpm. Together with
https://github.com/rpm-software-management/rpm/pull/2930 and
https://src.fedoraproject.org/rpms/podman/pull-request/131, the build becomes
reproducible.

This PR and https://pagure.io/go-rpm-macros/pull-request/62 conflict, so one of them will have to be rebased.

If #62 is merged first, I'll rebase…

Is it possible for $SOURCE_DATE_EPOCH to be undefined? Should this have a fallback or at least change to ${SOURCE_DATE_EPOCH-} (in case a packager uses set -u or such)?

rebased onto cac1a22

5 months ago

I think in the Fedora case, SOURCE_DATE_EPOCH would always be defined. But you're right, it's probably better to provide a fallback like for other variables. I changed it to use :-.

Cool. Thanks! Can you please rebase this now that other PR has been merged? This flag has moved to the %gobuild_ldflags definition.

rebased onto 1980932

5 months ago

Pull-Request has been merged by gotmax23

4 months ago

I tested this with osbuild-composer-102-1.fc41. I still get a difference, but I think it must be caused by something else. Looking at the build log [1]:
+ go build -buildmode pie -compiler gc '-tags=rpm_crashtraceback ' -a -v -x -ldflags ' -X '\''github.com/osbuild/osbuild-composer/internal/common.RpmVersion=osbuild-composer-102-1.fc41.x86_64'\'' -X github.com/osbuild/osbuild-composer/version=102 -B 0xf36d4ab9ce0d4697e7c7f22d606899e6b457eff8 -compressdwarf=false -linkmode=external -extldflags '\''-Wl,-z,relro -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld-errors -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes '\''' -o _bin/osbuild-composer github.com/osbuild/osbuild-composer/cmd/osbuild-composer
and locally
+ go build -buildmode pie -compiler gc '-tags=rpm_crashtraceback ' -a -v -x -ldflags ' -X '\''github.com/osbuild/osbuild-composer/internal/common.RpmVersion=osbuild-composer-102-1.fc41.x86_64'\'' -X github.com/osbuild/osbuild-composer/version=102 -B 0xf36d4ab9ce0d4697e7c7f22d606899e6b457eff8 -compressdwarf=false -linkmode=external -extldflags '\''-Wl,-z,relro -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld-errors -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes '\''' -o _bin/osbuild-composer github.com/osbuild/osbuild-composer/cmd/osbuild-composer

So I think the build flags are working as expected.

[1] https://kojipkgs.fedoraproject.org//packages/osbuild-composer/102/1.fc41/data/logs/x86_64/build.log

Metadata