#311 comps-sync mechanism does not work with arch-dependent package specifiers
Closed: Fixed by siosm. Opened by adamwill.

So, we have a bit of a problem with Kinoite right now. The kde-desktop group in comps has this line:

      <packagereq arch="aarch64,x86_64" type="default">kio-gdrive</packagereq>

that means "include kio-gdrive, but only on aarch64 and x86_64". This is because it's not built on other arches (it is only built on the arches for which qt5-qtwebview is built).

However, comps-sync.py entirely ignores comps' arch filtering. So in kde-desktop-pkgs.yaml we have kio-gdrive listed in the packages list, and this causes Kinoite build to fail on ppc64le:

DEBUG util.py:445:  error: Packages not found: kio-gdrive

Having poked at this for an hour or so, I realized there doesn't seem to be any good solution for this right now.

It's difficult/impossible to handle these arch-dependent specifiers in comps-sync.py because of how libcomps actually handles them. The libcomps object for a package entry you get from a comps query doesn't include this arch property. Instead, you're supposed to put an arch filter on your queries if you want the results for a specific arch. So if you do this (using the Python bindings), you'll get kio-gdrive in the results:

packages = comps.groups_match(id="kde-desktop")[0].packages

because you're not filtering at all. If you do this, you'll also get it:

filtered = comps.arch_filter(["x86_64"])
packages = filtered.groups_match(id="kde-desktop")[0].packages

because you're filtering for x86_64. But if you do this:

filtered = comps.arch_filter(["ppc64le"])
packages = filtered.groups_match(id="kde-desktop")[0].packages

because you're filtering for ppc64le. However, this mechanism is just not very helpful for something like comps-sync.py. You can make it work, but it'd be very inefficient and rather hard to keep track of. Still, it might come down to doing that, unfortunately, because other options are also lacking.

There are situations like this in the core group, and there's already a kind of coping mechanism in place for those. They are excluded in the exclude_list in comps-sync-exclude-list.yml, and added back in fedora-common-ostree.yaml, in the packages-(arch) lists.

However, I can't see a way to do something like this for an arch-dependent package that's not in a core group and so not safe to list in fedora-common-ostree.yaml. The logical thing to do is add a packages-aarch64 and a packages-x86_64 to kde-desktop-pkgs.yaml and list it there, but that file is the output of comps-sync.py, and will be overwritten by future runs of it, so that won't work. There does not appear to be any other place I could 'manually' include the package for the two arches we know it should be included for.

So, it's a conundrum. All I can do for the short term is send a PR to exclude the package from all Kinoite builds, even though it should be included on aarch64 and ppc64le.

I note there's already a case where this is kind of "wrong". thermald is a similar situation where it's x86_64 only. The 'fix' that's been used there is to exclude it in comps-sync-exclude-list.yaml and include it back in fedora-common-ostree.yaml - even though it is only in workstation-product and so shouldn't be in Kinoite etc. We can hardly do that for a kio thing, though, which will result in large chunks of KDE being pulled into the other desktops.

I don't know if the best fix here is to do the awful surgery on comps-sync.py to make it run like five times with different arch filters and somehow keep track of the combined results, or enhance how the includes/excludes overrides are done somehow so we can do desktop-specific workarounds for these lines. But right now it's clearly a problem.


so writing that essay inspired me to go back and see how awful the surgery on comps-sync.py would be and, well, it's pretty awful, but I think it's just about doable. I'm in the middle of it now. There's blood and organs everywhere.

Closing as fixed by https://pagure.io/workstation-ostree-config/pull-request/312. Next step in https://pagure.io/workstation-ostree-config/pull-request/314 which we'll likely soon merge too.

Thanks!

Metadata Update from @siosm:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

Metadata