#1315 Allow choosing chroot where the imported src.rpm sources are built
Opened 2 years ago by churchyard. Modified a year ago

I was debugging a build failure of https://src.fedoraproject.org/rpms/python-apsw in https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/python-apsw/ (example: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/build/1311094/)

Funny thing: The spec file has version that is different in different Fedora versions (simplified):

%if 0%{?fedora} >= 32
Version: 3.31.1.r1
%else
Version: 3.29.0.r1
%endif

An intermezzo: :scream: :scream: :scream:

After heavy breathing I have realized that this is perfectly fine in Fedora, however, copr blows up. It creates the SRPM on Fedora 31 with:

cmd: ['rpkg', 'srpm', '--outdir', '/tmp/copr-rpmbuild-k6aemxj5', '--spec', '/tmp/copr-rpmbuild-k6aemxj5/obtain-sources/python-apsw']

That outputs:

Output: ['python-apsw.spec', 'apsw-3.29.0-r1.zip', 'python-apsw-3.29.0.r1-1.fc31.src.rpm']

Yet then the rawhide build (obviously) fails with:

error: Bad source: /builddir/build/SOURCES/apsw-3.31.1-r1.zip: No such file or directory

Can the SRPM be built for the given Fedora version, on the given Fedora version (or at least with an equivalent of: fedpkg --release <release> srpm?

I realize that this would require a fundamental change in Copr design, as currently the SRPM is built just once. As an alternative, there is a "Mock chroot: what chroot to run the script in" option with the "custom" source type. Can we please have that option for SCM builds as well?

Thanks


It seems that "Mock chroot: what chroot to run the script in" only affects the downloading of the source and the SRPM creation happens on Feodra 31 anyway. In order to make this work it would need to create the SRPM in the chroot.

Hmm, I don't think we want to regress back to the time when we - for each
target chroot - generated one src.rpm which was later imported. It is both
wrong (because people don't know that they build different sources into
different chroots), but it is also non-economical; buliding the SCM sources
consumes some resources (not only IO/CPPU, but for large SRC.RPMs even
network).

I'd say that -- if user really wants such hacks -- multiple builds have to be
submitted, with different --chroot sub-sets.

Meh, yes ... but we could allow you to somehow pick what chroot is
is used to build the (later imported) src.rpm in. That sounds like valid RFE.

Packages building from SCM create SRPM on arbitrary Fedora version

The "arbitrary" version is the host Fedora version (/etc/mock/default.cfg is used for that).

but we could allow you to somehow pick what chroot is is used to build the (later imported) src.rpm in

That sounds like a reasonable solution.

BTW the rpmbuild fails because it cannot find the zip file. It is in lookaside cache, but it is not fetched because it is not in sources file.

I bet it will start working if the user write:

%if 0%{?fedora} >= 32
Version: 3.31.1.r1
%else
Version: 3.29.0.r1
%endif
Source0: apsw-3.29.0-r1.zip
Source1: apsw-3.31.1.r1.zip

Hmm, but the %prep section would be a pain....

Having this in the spec file is asking for trouble. It just made me realize that we make the SRPMs on the target platform in Fedora. So I would like Copr to do the same. But I realize that it is a major redoing of how things work + waste of resources. So instead, being able to select 1 particular chroot to create the SRPMs in solves the problem for "we are rebuilding rawhide content" repos like @python/python3.9.

I agree with what you said @churchyard , but

It just made me realize that we make the SRPMs on the target platform in Fedora. So I would like Copr to do the same.

s/in Fedora/in Koji/. But Copr actually does the same. We first re-build src.rpm in target
chroot, and then we build RPMs from it. The problem is how to package is imported to
Copr's dist-git ... we
- on one builder machine download or build the SRC.RPM sources
- on dist-git machine we download ^^^, and commit it to dist-git
- on different builder we build SRC.RPM from dist-git for the target chroot
- and on that ^ builder we build RPM for the target chroot

@msuchy wrote

Hmm, but the %prep section would be a pain....

In %prep section you can use %version macro already, I believe.

The entire "fetch from fedora dist-git - create srpm - import to copr dist-git" loop seems like an overkill (in this use case). Being able to build from fedora dist-git directly would be awesome (out of scope here, just a side note).

It looks complicated, but dist-git is actually "proxy" between outside world and copr.
Except that we want to make sure that we use the same sources (byte-by-byte) for
building for all the chroots in one build -- we also don't want to DoS the outside world
(fedora dist-git, gitlab, github, etc.).

Btw., Mirek documented the dist-git purpose in blog post https://fedora-copr.github.io/posts/dist-git but now I realized that it doesn't mention the consistency POV (and DoS protection).
I think we should dump the reasons why we have dist-git, even to downstream docs.

Metadata Update from @praiskup:
- Issue tagged with: RFE

2 years ago

Btw., #534 would probably help here (all methods should go through the same abstraction
as custom method).

That said, you can try "custom" method, where you can set what chroot is used to build
sources (i.e. pick rawhide chroot, install fedpkg, run fedpkg clone -a .. && fedpkg sources.

"custom" method only fetches the sources in the selected chroot, creates SRPM in "default" chroot

Indeed. I forgot about that, we didn't want to do manual rpmbuild -bs ... so we re-used
mock for automating this. Meh, the great irony is that we use the src.rpm only to transport
the sources to copr-dist-git - and plain rsync, or tarball in worst case would be just fine for that.

Side note: https://pagure.io/packaging-committee/issue/959 (should conditional version be forbidden in Fedora?)

Another one: https://src.fedoraproject.org/rpms/mycli/blob/master/f/mycli.spec

%if 0%{?fedora} > 32
Patch0:         mycli-1.21.1-prompt-toolkit.patch
%endif
%if 0%{?fedora} < 32
Patch0:         mycli-1.21.1-sqlparse.patch
%endif

Copr:

RPM build errors:
error: Bad file: /builddir/build/SOURCES/mycli-1.21.1-prompt-toolkit.patch: No such file or directory
    Bad file: /builddir/build/SOURCES/mycli-1.21.1-prompt-toolkit.patch: No such file or directory

Another one is libreoffice:

%if 0%{?fedora} > 33 || 0%{?rhel} > 8
Patch5: 0001-Upgrade-liborcus-to-0.16.0.patch
%endif

Copr:

error: Bad file: /builddir/build/SOURCES/0001-Upgrade-liborcus-to-0.16.0.patch: No such file or directory

https://bugzilla.redhat.com/show_bug.cgi?id=1898093

Another one is libreoffice:

Something like this happens from time to time, and people are asking why they should
or shouldn't conditionalize Version/Source/Patch statements.

I don't think it can be considered a valid use case because maintainers rarely use
mock for generating source RPMs. They just do 'fedpkg srpm' or 'rpmbuild -bs', and
such commands shouldn't depend on the system they are run on.

Should we try to forbid this by Fedora guidelines for everyone's good?

In theory, we could try to fix RPM so the Patch/Source statements are applied
unconditionally during rpmbuild -bs (even though they are behind %if 0).
But that's just protecting people from willingly shooting into foot, and so it
would stop building source RPMs for existing source RPMs.

Perhaps we could add a warning into /bin/rpmlint about conditional
Patch/Source/Version?

Should we try to forbid this by Fedora guidelines for everyone's good?

Yes, but people will still do it.

Perhaps we could add a warning into /bin/rpmlint about conditional Patch/Source/Version?

That's a hard problem. Some conditionals are OK (they are only defined in the spec file) and some are not (they are impacted by the environment).

As an example, I consider this to be OK:

%global prerelease beta4
...
%if %{defined prerelease}
Source0:  https://sources.tld/development/%{name}-%{version}-%{prerelease}.tar.xz
%else
Source0:  https://sources.tld/stable/%{name}-%{version}.tar.xz
%endif

This to be sometimes necessary:

%global freeworld 0
...
%if %{freeworld}
Source0:  https://sources.tld/releases/%{name}-%{version}.tar.xz
%else
# Removed illegal codec
Source0:  %{name}-%{version}-stripped.tar.xz
%endif

This to be dangerous and better to be avoided:

%if %{with bootstrap}
Source0:  https://sources.tld/releases/%{name}-%{version}.tar.xz
%else
Source0:  https://sources.tld/releases/%{name}-%{version}_with_tests.tar.xz
%endif

This one really bad:

%if 0%{rhel}
Source0:  https://sources.tld/releases/%{name}-%{version}_everything_is_bundled.tar.xz
%else
Source0:  https://sources.tld/releases/%{name}-%{version}.tar.xz
%endif

And this one a heresy:

%if 0%{fedora} > 33
Version: 15.4
%else
Version:  15.0
%endif
Source0:  https://sources.tld/releases/%{name}-%{version}.tar.xz

(I could probably do the lawful-chaotic good-evil chart if I wanted to.)

Login to comment on this ticket.

Metadata