#954 Prohibit use of `rpm` command from specfile.
Opened 5 months ago by vondruch. Modified 6 days ago
vondruch/packaging-committee dont-use-rpm-in-specfile  into  master

@@ -1754,6 +1754,10 @@ 

  

  If (and only if) the script needs to remain executable and cannot be modified to pass the checks, then the maintainer MAY elect to disable the checks and modifications. It is also possible to disable the functionality for specific paths or for specific shebang lines by setting `+%__brp_mangle_shebangs_exclude_from+` and `+%__brp_mangle_shebangs_exclude+`, respectively, using the same syntax as the settings described in xref:AutoProvidesAndRequiresFiltering.adoc[Packaging:AutoProvidesAndRequiresFiltering]. It is also possible to disable the functionality entirely by adding `+%undefine __brp_mangle_shebangs+` near the beginning of the specfile.

  

+ == Do not call `rpm` command from specfile

+ 

+ `rpm` command in specfile MUST NOT be used, because it couldn't be guaranteed to work in the case that the buildroot was populated using a different version of rpm that used a different backend database. If that is the case, call to `rpm` might result in build failure.

+ 

  == BRP (BuildRoot Policy) Scripts

  

  BRP scripts are injected at the end of `+%install+` (via the `+%__os_install_post+` macro) and perform some automatic sanity checks of, or adjustments to, files installed in the build root.

This might be problematic, if buildroot was populated using a different version of rpm that used a different backend database.

Original discussion at https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/QKTLQ2AITO5MVSX77GYDHTD24ZW7MLFO/

-1 until proven problematic

+1

That is whole reason why mock is using --nodeps all over the place because rpmdb of system does not have to be readable by chroot's RPM.

-1 until proven problematic

I think this was always problematic. Just remember the days building RHEL6 packages on more recent Fedora.

And as @kevin pointed out, it could be even more problematic when RPM is migrated to Sqlite backend [1] proposed by @pmatilai.

It does not appear to be problematic, because luckily, it is not widely used. I think the only problem is that the reasons are not correctly documented, what this PR should cover.

I was following that discussion. And before we ban %requires_eq without a replacement, I want to see it actually blow up.

I was following that discussion. And before we ban %requires_eq without a replacement, I want to see it actually blow up.

I don't think we apply guidelines retroactively. Also, banning %requres_eq is different topic [1] and there might be exceptions to this rule based on "use their own best judgment" guideline (shouldn't the "use their own best judgement" guideline deserve its own chapter?)

There's nothing particularly wrong with calling rpm from within spec files, as such. However doing that in the kind of cross-version, cross-chroot build-environment that Fedora has been using IS wrong and will behave erratically in some cases. I'm a bit surprised this hasn't been banned in Fedora since the beginning because it's quite widely known fact at least among us old-timers. Packagers' best judgement doesn't work because this typically works fine when you try it on your own system, it's in the buildsystem that it fails.

Note that IF mock was used in the (recently added) full bootstrap-image mode where the host rpm is never used for anything inside the chroot, then calling rpm from spec wouldn't be a problem.

Is it possible to get some macro in rpm that does this "the safe way" so we can ban it with an actual replacement?

You can't do it safe this way.. What we could do is to generate automatically some macro based on package versions and inject them into chroot somewhere in mock...

We could inject list of all versions in a text file into chroot and provide an API to read it, or something like that.

Is it possible to get some macro in rpm that does this "the safe way" so we can ban it with an actual replacement?

I don't really think there is something like this necessary. There is enough provides and what not already generated. But there are always exceptions. And since this is exception, it should be handled exceptionally.

It would even not be that bad to maintain that hard dependency manually, because it must be all the time broken. But in the Samba case, there could be dependency on %{_libdir}/libldb.so.2.x.y which is easy to extract from current build root. Where for the ccls, it could be just parsing clang --version or ls /usr/lib64/clang/

Anyway, samba just internalized the marco [1] which is probably fine based on packagers discretion. While it was recently introduced to ccls [2] by @defolos

You can't do it safe this way.. What we could do is to generate automatically some macro based on package versions and inject them into chroot somewhere in mock...

That sounds pretty much like generating alternative RPM database :)

The catch is that rpm(build) itself cannot know whether its being used in a "safe" manner or not.
So basically the choice is between defining it illegal or fixing the infrastructure to make such usage safe.

OK, so I've done this:

Mock config:

include('/etc/mock/fedora-rawhide-x86_64.cfg')

config_opts['root'] = 'fedora-rawhide-rpm-snapshot'

config_opts['use_bootstrap'] = False

config_opts[f'{config_opts.package_manager}.conf'] += """

[rpm-snapshot]
name=Copr repo for rpm-snapshot owned by pmatilai
baseurl=https://download.copr.fedorainfracloud.org/results/pmatilai/rpm-snapshot/fedora-rawhide-$basearch/
type=rpm-md
skip_if_unavailable=False
gpgcheck=1
gpgkey=https://download.copr.fedorainfracloud.org/results/pmatilai/rpm-snapshot/pubkey.gpg
repo_gpgcheck=0
enabled=1
enabled_metadata=1
cost=10
"""

And I've run this from Fedora 32:

$ mock -r fedora-rawhide-rpm-snapshot init
$ mock -r fedora-rawhide-rpm-snapshot shell
<mock-chroot> sh-5.0# rpm --version
RPM version 4.15.90
<mock-chroot> sh-5.0# rpm --eval '%_db_backend'
bdb
<mock-chroot> sh-5.0# echo %_db_backend sqlite > /etc/rpm/macros.db
<mock-chroot> sh-5.0# rpm --eval '%_db_backend'
sqlite
<mock-chroot> sh-5.0# rpm -q rpm
warning: Found bdb Packages database while attempting sqlite backend: using bdb backend.
rpm-4.15.90-0.git14971.1.fc33.x86_64
<mock-chroot> sh-5.0# rpm --eval '%requires_eq rpm'
warning: Found bdb Packages database while attempting sqlite backend: using bdb backend.
Requires: rpm = 4.15.90

That doesn't sound that bad.

IIRC, with BDB that simple thing will already have rendered the database inaccessible from outside of the chroot however. And if you swap the places (as will eventually happen) and you need to build for an older release that doesn't talk sqlite, it'll only see an empty database directory and initialize a new BDB db in there, so you'll get incorrect results and confusion on top. There are plenty of weird scenarios in this general direction.

Metadata Update from @churchyard:
- Pull-request tagged with: meeting

4 months ago

From this weeks meeting (https://meetbot-raw.fedoraproject.org/fedora-meeting-1/2020-05-14/fpc.2020-05-14-16.00.txt):

FYI I was requested a few years ago to add buildroot content to the srpm , so people could check what a package was built with without needing to hunt down the info in koji-like web portals

That implies running rpm -qa in build and I implemented it as part of
https://fedoraproject.org/wiki/Changes/rpm_level_auto_release_and_changelog_bumping

I could easily switch the rpm -qa call so something else if the something else was available (after all the spec is executed by rpm, so it can call rpm lua methods).

Not sure how relevant the rpm incompatibility issue is today now that mock has a container mode, however.

Not sure how relevant the rpm incompatibility issue is today now that mock has a container mode, however.

Oh, that's a good point. If mock bootstrap-mode is used, the reason for avoiding rpm queries from the spec simply vanishes.

Oh, that's a good point. If mock bootstrap-mode is used, the reason for avoiding rpm queries from the spec simply vanishes.

That would be actually terrific, even more if it would be somehow possible to check if a package was e.g. built in bootstrap mode or not.