#1431 uploaded comps.xml doesn't get used by repodata of the chroot build results
Closed: Fixed 2 years ago by praiskup. Opened 4 years ago by loise.

When uploading a comps.xml into a chroot configuration, I expect the comps.xml not only to be used inside the chroot (mock) but also to be used in the repodata of the build results so the build results have a comps.xml groups file that gets used when calling createrepo with that file.

The upload location can stay the way it is, just when calling createrepo it should add -g with the location of the comps.xml file.

Result: you have a repo with your own groups created which helps users installing by a group from that repo.

Example of what I mean here:
Uploading comps.xml:
https://i.imgur.com/9T9ZRGb.png

After regenerating the repodata see directory contents of repodata of the chroot results:

https://i.imgur.com/Puzg02p.png


Metadata Update from @praiskup:
- Issue tagged with: bug

4 years ago

Metadata Update from @praiskup:
- Issue priority set to: High

4 years ago

Metadata Update from @praiskup:
- Issue assigned to praiskup

3 years ago

Commit 88a2db6 relates to this ticket

I found this issue and would like to understand what comps.xml actually is. The documentation in copr only has a link to this issue being fixed and nothing more (see https://docs.pagure.org/copr.copr/release-notes/2021-06-16.html?highlight=comp%20xml#bugfixes). A hint to where the actual documentation for this file format is would be helpful. Maybe even putting a link to the documentation inside of the copr UI would be helpful as well. Then things become less esoteric.

I found this issue and would like to understand what comps.xml actually is. The documentation in copr only has a link to this issue being fixed and nothing more (see https://docs.pagure.org/copr.copr/release-notes/2021-06-16.html?highlight=comp%20xml#bugfixes). A hint to where the actual documentation for this file format is would be helpful. Maybe even putting a link to the documentation inside of the copr UI would be helpful as well. Then things become less esoteric.

comps.xml is the groups file in a repo where you can collect sets of rpms to groups for yum group install. That way, you can just use yum group install kde-plasma-workspaces to install KDE.

The reason I noticed was that I tried to upload a changed epel groups file to add groups like developer tools for KDE and test my copr repo using yum group install instead of installing each rpm separately.

comps.xml is the groups file in a repo where you can collect sets of rpms to groups for yum group install. That way, you can just use yum group install kde-plasma-workspaces to install KDE.

Okay thanks @loise . That explains the idea but not the format.

We could add some useful link(s) to the page in question if we have some? Please open a separate issue.

Please open a separate issue.

I think there is no need. Modified in PR#2118

Commit 3ad7a0c fixes this issue

And that still doesn't work :(

Copr backend puts the file at chroot/comps.xml, as shown in the testcase:

         file = os.path.join(self.opts.destdir,
                             "praiskup/ping/fedora-rawhide-x86_64",
                             "comps.xml")

copr-repo looks foor chroot/repodata/comps.xml instead,

def run_createrepo(opts):
    mb_comps_xml_path = os.path.join(opts.directory, "repodata", "comps.xml")
    if os.path.exists(mb_comps_xml_path):
        createrepo_cmd += ['--groupfile', mb_comps_xml_path, '--keep-all-metadata']

fails to find it and produces repomd.xml without references to comps.xml.

Something like this would fix the repodata generation, but it obviously doesn't address stale files at the previous location or all the references to chroot/comps.xml and tests:

diff --git a/backend/copr_backend/actions.py b/backend/copr_backend/actions.py
index a686efc0..cb21a4c9 100644
--- a/backend/copr_backend/actions.py
+++ b/backend/copr_backend/actions.py
@@ -307,7 +307,7 @@ class CompsUpdate(Action):
         remote_comps_url = self.opts.frontend_base_url + url_path
         self.log.info(remote_comps_url)

-        path = os.path.join(self.destdir, ownername, projectname, chroot)
+        path = os.path.join(self.destdir, ownername, projectname, chroot, "repodata")
         ensure_dir_exists(path, self.log)
         local_comps_path = os.path.join(path, "comps.xml")
         result = ActionResult.SUCCESS

A call to call_copr_repo to regenerate repodata could be a nice addition as well.

Interesting that backend/tests/test_modifyrepo.py:test_comps_present uses correct destination and ensures that comps can be published to the repodata :smile:

Meh, thank you for reporting this.

Something like this would fix the repodata generation, but it obviously doesn't address stale files at the previous location

Those files stay unused, so do we care?

or all the references to chroot/comps.xml and tests:

Not sure I understand this, can you elaborate?

A call to call_copr_repo to regenerate repodata could be a nice addition as well.

Hm, indeed.

Interesting that backend/tests/test_modifyrepo.py:test_comps_present uses correct destination and ensures that comps can be published to the repodata 😄

That test case is syntetic, mocking the actual directory.... but it actually proves that
once we install the file on an appropriate place, it will work finally?

Metadata Update from @praiskup:
- Issue status updated to: Open (was: Closed)

2 years ago

Something like this would fix the repodata generation, but it obviously doesn't address stale files at the previous location

Btw., I am proposing a different fix: Keep the comps.xml in the old place, and fix the logic that searches for it.

Possible fix in #2239.
Btw., I am proposing a different fix: Keep the comps.xml in the old place, and fix the logic that searches for it.

Thanks! I haven't thought about changing path in copr-repo, but it actually resolves all my concerns about migrating old comps.xml and fixing tests. #2239 looks good to me.

Thank you for the feedback. Fix is going to be shipped in the next release.

Log in to comment on this ticket.

Metadata
Related Pull Requests
  • #2239 Merged 2 years ago
  • #1841 Merged 3 years ago