Learn more about these different git repos.
Other Git URLs
If you add an external repo to a build tag, and the external repo uses the same xml:base trickery that Koji itself uses (that is, if the external repo you are pointing it is produced by another Koji instance for a tag which itself has an external repo configure), the xml:base attribute is lost by /usr/libexec/kojid/mergerepos and the resulting merged repodata ends up with incorrect URLs.
To give a hypothetical example, imagine your Koji is producing a repo from a build tag at:
http://thirdpartykoji.example.com/kojifiles/repos/beaker-harness-rhel-8-build/latest/x86_64/repodata/
It uses an external repo to pull packages from another Koji instance at:
http://download.bigcorp.com/brewroot/repos/rhel-8.0-build/latest/x86_64/repodata/
But that repo itself is pulling packages from an external repo:
https://kojipkgs.fedoraproject.org/repos/module-bootstrap-rawhide/latest/x86_64/repodata/
The download.bigcorp.com repodata will refer to the remote packages using xml:base, like this:
<location xml:base="https://kojipkgs.fedoraproject.org/repos/module-bootstrap-rawhide/latest/x86_64/toplink/packages/audit/2.7.7/5.fc27/x86_64" href="audit-libs-2.7.7-5.fc27.x86_64.rpm"/>
Everything is fine so far. But now, when mergerepos runs on thirdpartykoji.example.com, I expect it to preserve the same <location/> so that the packages can be correctly downloaded no matter how many layers of external repos are involved. However mergerepos on thirdpartykoji.example.com instead produces this:
<location xml:base="http://download.bigcorp.com/brewroot/repos/rhel-8.0-build/latest/x86_64" href="audit-libs-2.7.7-5.fc27.x86_64.rpm"/>
which fails to download because the package is not there. The original value of xml:base has been lost.
The end result is build failures with errors like this in mock_output.log as yum/dnf tries to download the packages from the wrong URL:
http://download.bigcorp.com/brewroot/repos/rhel-8.0-build/latest/i386/audit-libs-2.7.7-5.fc27.i686.rpm: [Errno 14] HTTP Error 404 - Not Found Trying other mirror.
I narrowed the problem down to this line in createrepo/init.py around line 621, where it unconditionally overwrites the package basepath:
def save_keptpkgs(upto): while keptpkgs and (upto is None or keptpkgs[-1][0] < upto): filename, po = keptpkgs.pop() # reset baseurl in the old pkg po.basepath = self.conf.baseurl
even though in this case, there is no "baseurl" configured and so it just replaces it with None.
I think that code probably should be:
if self.conf.baseurl: po.basepath = self.conf.baseurl
that is, only replace the base url if a base url is explicitly passed into createrepo. But the code is in createrepo not Koji and it's hard for me to tell if that might break things elsewhere somehow.
Does this work correctly with createrepo_c enabled and using mergerepo_c --koji mode?
createrepo_c
mergerepo_c --koji
Is there a config option or some other thing I can do to make kojid use mergerepo_c instead?
When you use 'use_createrepo_c=yes' in kojid.conf, both (createrepo_c + mergerepo_c) will be used.
Ah ha, that's what I was missing.
Yes I can confirm that when mergerepos_c is used, the xml:base attribute is correctly preserved and everything works as expected. So this bug is only in the /usr/libexec/kojid/mergerepos implementation.
My suggestion is that the koji internal mergerepos code is ripped out and that we only depend on mergerepo_c. That will not only solve the correctness problem, but speed things up considerably.
mergerepo_c
What about starting with change 'use_createrepo_c=yes' as a default? @mikem?
So, I would propose following - we're not fixing it in bundled mergerepo and change usecreaterepo_c to default in 1.18 with possible removal of bundled code and option in some future release (1.20+).
Metadata Update from @tkopecek: - Issue set to the milestone: 1.18
Metadata Update from @tkopecek: - Issue tagged with: deprecation
PR #1278
Commit d226111 fixes this issue
Commit be1a566 fixes this issue
Login to comment on this ticket.