#3626 "koji import-comps" skips environment groups
Opened 2 years ago by ngompa. Modified 2 years ago

When attempting to build CentOS Hyperscale live media, the builds fail because none of the environment groups from CentOS Stream nor EPEL are present:

The GNOME image failure in the logs is:

DEBUG util.py:444:  [ ERROR   ]: 14:10:03 | KiwiInstallPhaseFailed: System package installation failed: Module or Group 'workstation-product-environment' is not available.

The KDE image failure in the logs is:

DEBUG util.py:444:  [ ERROR   ]: 14:17:47 | KiwiInstallPhaseFailed: System package installation failed: Module or Group 'kde-desktop-environment' is not available.

According to my local environment, these both exist:

ngompa@fedora ~> podman run --rm -it quay.io/centoshyperscale/centos:stream9
bash-5.1# dnf grouplist -v
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync
DNF version: 4.14.0
cachedir: /var/cache/dnf
User-Agent: constructed: 'libdnf (CentOS Stream 9; hyperscale; Linux.x86_64)'
repo: downloading from remote: centos-hyperscale
countme: event triggered for centos-hyperscale: bucket 2
CentOS Stream 9 - Hyperscale                                                                                                                                                463 kB/s | 248 kB     00:00
centos-hyperscale: using metadata from Thu Dec  8 00:04:34 2022.
repo: downloading from remote: centos-hyperscale-experimental
countme: event triggered for centos-hyperscale-experimental: bucket 1
CentOS Stream 9 - Hyperscale Experimental                                                                                                                                   5.5 MB/s | 2.4 MB     00:00
centos-hyperscale-experimental: using metadata from Wed Dec  7 18:00:01 2022.
repo: downloading from remote: centos-hyperscale-spin
countme: event triggered for centos-hyperscale-spin: bucket 1
CentOS Stream 9 - Hyperscale Spin                                                                                                                                            42 kB/s |  36 kB     00:00
centos-hyperscale-spin: using metadata from Wed Apr  6 23:34:08 2022.
repo: downloading from remote: baseos
countme: no event for baseos: budget to spend: 3
CentOS Stream 9 - BaseOS                                                                                                                                                     13 MB/s | 6.0 MB     00:00
baseos: using metadata from Mon Dec  5 14:09:20 2022.
repo: downloading from remote: appstream
countme: event triggered for appstream: bucket 2
CentOS Stream 9 - AppStream                                                                                                                                                  10 MB/s |  16 MB     00:01
appstream: using metadata from Mon Dec  5 14:11:11 2022.
repo: downloading from remote: crb
countme: event triggered for crb: bucket 1
CentOS Stream 9 - CRB                                                                                                                                                       1.4 MB/s | 5.1 MB     00:03
crb: using metadata from Mon Dec  5 14:12:03 2022.
repo: downloading from remote: extras-common
countme: no event for extras-common: budget to spend: 1
CentOS Stream 9 - Extras packages                                                                                                                                            20 kB/s | 9.2 kB     00:00
extras-common: using metadata from Fri Nov 25 07:08:56 2022.
repo: downloading from remote: epel
countme: no event for epel: budget to spend: 1
Extra Packages for Enterprise Linux 9 - x86_64                                                                                                                              6.5 MB/s |  12 MB     00:01
epel: using metadata from Sat Dec 10 00:29:42 2022.
repo: downloading from remote: epel-next
countme: no event for epel-next: budget to spend: 2
Extra Packages for Enterprise Linux 9 - Next - x86_64                                                                                                                       1.2 MB/s | 1.4 MB     00:01
epel-next: using metadata from Thu Nov 24 03:28:46 2022.
Completion plugin: Generating completion cache...
Available Environment Groups:
   Server with GUI (graphical-server-environment)
   Server (server-product-environment)
   Minimal Install (minimal-environment)
   Workstation (workstation-product-environment)
   KDE Plasma Workspaces (kde-desktop-environment)
   Custom Operating System (custom-environment)
   Virtualization Host (virtualization-host-environment)
Available Groups:
   Legacy UNIX Compatibility (legacy-unix)
   Console Internet Tools (console-internet)
   Container Management (container-management)
   Development Tools (development)
   .NET Development (dotnet)
   Graphical Administration Tools (graphical-admin-tools)
   Headless Management (headless-management)
   Network Servers (network-server)
   RPM Development Tools (rpm-development-tools)
   Scientific Support (scientific)
   Security Tools (security-tools)
   Smart Card Support (smart-card)
   System Tools (system-tools)
   Fedora Packager (fedora-packager)
   Xfce (xfce-desktop)
bash-5.1# exit

But Koji doesn't have them in the build tag:

ngompa@fedora ~> cbs taginfo hyperscale9s-spin_media-experimental-el9s-build
Tag: hyperscale9s-spin_media-experimental-el9s-build [2665]
Arches: x86_64 aarch64
Groups: additional-devel, anaconda-tools, backup-client, base, base-x, build, conflicts-appstream, conflicts-baseos, console-internet, container-management, core, crb, critical-path-kde, debugging, desktop-debugging, development, dial-up, dns-server, dotnet, emacs, fedora-packager, file-server, firefox, fonts, ftp-server, gnome-apps, gnome-desktop, graphical-admin-tools, graphics, guest-agents, guest-desktop-agents, hardware-monitoring, hardware-support, headless-management, infiniband, input-methods, internet-applications, internet-browser, java-development, java-platform, kde-apps, kde-desktop, kde-education, kde-media, kde-office, kde-software-development, kf5-software-development, kiwi-build, large-systems, legacy-unix, legacy-x, mail-server, mainframe-access, multimedia, network-file-system-client, network-server, network-tools, networkmanager-submodules, office-suite, ostree-support, performance, platform-devel, platform-kvm, platform-microsoft, platform-vmware, print-client, python-web, remote-desktop-clients, remote-system-management, rpm-development-tools, scientific, security-tools, server-product, smart-card, smb-server, srpm-build, standard, system-tools, texlive, virtualization-client, virtualization-hypervisor, virtualization-platform, virtualization-tools, web-server, workstation-product, xfce-desktop
Tag options:
  mock.new_chroot : 0
  mock.package_manager : 'dnf'
  mock.yum.module_hotfixes : 1
  rpm.macro.vendor : 'CentOS Hyperscale SIG'
This tag is a buildroot for one or more targets
Current repo: repo#992704: 2022-12-12 11:49:43.189377+00:00
Targets that build from this tag:
  hyperscale9s-spin_media-experimental-el9s
External repos:
    5 centos9s-baseos (http://mirror.stream.centos.org/9-stream/BaseOS/$arch/os/, merge mode: bare), arches: inherited from tag
   10 centos9s-appstream (http://mirror.stream.centos.org/9-stream/AppStream/$arch/os/, merge mode: bare), arches: inherited from tag
   15 centos9s-crb (http://mirror.stream.centos.org/9-stream/CRB/$arch/os/, merge mode: bare), arches: inherited from tag
   20 epel9 (https://cbs.centos.org/kojifiles/repos/epel/9/Everything/$arch/, merge mode: bare), arches: inherited from tag
   25 epel9-next (https://cbs.centos.org/kojifiles/repos/epel/next/9/Everything/$arch/, merge mode: bare), arches: inherited from tag
Inheritance:
  0    .... hyperscale9s-packages-main-release [2378]
  5    .... buildsys9s-release [2363]
  10   .... hyperscale9s-spin_media-experimental-candidate [2662]
  15   .... hyperscale9s-packages-hotfixes-release [2414]
  20   .... hyperscale9s-packages-spin-release [2382]
  25   .... hyperscale9s-packages-experimental-release [2410]
  30   .... extras9s-extras-common-candidate [2553]

@arrfab imported the comps.xml from CentOS and EPEL into here, so the only way they could be missing is if they're all being skipped.


Yes, we're ignoring other group types. They didn't make sense so far. We can either fake it (import environment as a regular group) or implement environments. Not sure if it could be of some use for other image types. @mikem ? Other option is to create kiwi-specific option for uploading its own comps and use it but it would be complicated to combine with koji repo (something like locally merged repo used just for given build).

Metadata Update from @tkopecek:
- Custom field Size adjusted to None

2 years ago

These would make sense for image builds to have, since the CentOS/Fedora image kickstarts and the kiwi descriptions reference environment groups.

How is this working in Fedora Koji for the images built from fedora-kickstarts? They use environment groups too.

For example: https://pagure.io/fedora-kickstarts/blob/main/f/fedora-kde-common.ks

There is an inherent translation in this command. Koji's groups data model was based on the structure of comps in 2005. Comps has continued to change, but it's not nearly as easy to change this part of the data model.

This command is best understood as "import koji groups data from a comps file".

If we extend this, we need to make sure we do so in a backwards compatible way.

How is this working in Fedora Koji for the images built from fedora-kickstarts? They use environment groups too.

KS-based images use additional repo (from compose) as e.g. here.

That doesn't apply to images built in CBS, since we don't have composes...

Log in to comment on this ticket.

Metadata