I have an idea for improving module disttag generation.
Now, the disttag is a random string. That means building the same component in various modules results in random package NEVRA. That's fine until you get a stack of two modules that provide the same RPM component. DNF in Koji then picks up the highest NEVRA of the package. That means a random one because they differ in disttag only and disttag is random.
I'll show you example:
I have perl-bootstrap module that builds perl-srpm-macros. But the perl-srpm-macros is already provided by MBR module. That means I have no clue which one appears in the build root. Fine, If I need different perl-srpm-macros build, I will probably use different commit ID of that component and I can carefully bump NVR.
But than I have another "perl" module that build-requires perl-bootstrap module. And "perl" module rebuilds some of the components that already exists in perl-bootstrap. The components point to the same commit ID, but they have different binary RPM output because of different macros in perl-srpm-macros file.
If I had more buildorder groups in "perl" module, I have the problem with unpredictable NEVRAs.
In case of perl, it does not matter. But I can image there can be cases when it matters (i.e. I want to use the last rebuild of a component).
Maintaining two sets of component IDs just only because of NEVRA is no-go. Especially if you have module with 170 components.
I propose change in module disttag generation. The disttag should be highest of all build-required modules. This can be implemented by adding an integer prefix to the disttag, gather all build-required disttags, and computing a new interger prefix higher than all of them.
Example: MBR has disttag 0.SOMEHASH, perl-bootstrap build-requires MBR, thus disttag for perl-boostrap will be 1.ANOTHERHASH. "perl" module that buildrequires MBR and "perl" will get assigned 2.YETANOTHERHASH because it has 0.SOMEHAS and 1.ANOTHERHASH in the build-root. When rebuilding "perl" module again, it will get assigned 2.YETYETANOTHERHASH. So the dist-tags will not grow indefinitely because a stack always contains finite number of modules.
This would still not solve multiple inheritance like this:
C / \ A B \ / MBR
where both A an B modules provided the same component because A and B are incomparable to each other. But still it would be big help because C were above A and B and MBR. This would ease bootstrapping module stacks where the intermediary modules exists for the sole purpose of bootstrapping and are not distributed to users at all.
Hey @psabata and @jkaluza, when you have time can you comment on this one?
Instead of solving this problem with changed dist-tags, this should probably raise an error at build time indicating that there are two conflicting modules providing a conflicting component.
We could look into solving that in a couple of places:
C
The problem is that packages provided by more modules are normal.
E.g. perl-srpm-macros package is run-required by rpm-build package. Thus perl-srpm-macros exists in base-runtime module. But the package naturally belongs to perl module because it provides RPM macros needed for building source RPM packages. And the same applies to *-srpm-macros packages for python, go, rust etc.
If you ban overlapping packages , you will break these package ecosystems.
And it breaks local building of layered modules now https://bugzilla.redhat.com/show_bug.cgi?id=1487117.
See also #823.
I think this is fixed with #823.
Metadata Update from @ralph: - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.