#289 RFE: support for including scratch module builds
Opened 2 years ago by merlinm. Modified 9 months ago

As an extension to the way ODCS is frequently used to test modules after they have been built, it would be desirable to also be able to create composes that include scratch module builds.

MBS does not create the content generator build in Koji for scratch module builds. It also does not move build from "done" to "ready". Although ODCS has been modified to also include builds in "done" state, pungi will fail later one because it expects the content generator build in Koji.

We either need a pungi change, or an MBS change (to create content generator builds for scratch builds) before this can be worked on.

+1 having this feature would be cool and would safe developers time for preparing their own solutions.

Oh wow :) so how should one now download and install the scratch module? I thought this works :(

@hhorak do you have a reference to the custom stuff people use now?

We just wanted to start testing module scratch builds and this is like a showstopper for us now :(

@merlinm is there anyway to prioritize this please?

@lucarval is there anybody who could help with this?

@jkaluza ? :D

Copying my answer to this topic from other ticket:

ODCS can be changed easily to find out scratch module builds. The issue is in Pungi which will not find the module build in Brew, because there simply isn't such build for scratch module builds. Pungi needs that Koji module build to find out modulemd metadata. This metadata can also be found in the MBS, but it is not generated in a way which could be used in the compose - the artifacts.rpms are not populated, licences are not populated, ...

So, this RFE should start with RFE for MBS to provide expanded modulemd files for scratch module builds somewhere. Then Pungi can find them in that new location (that's RFE for pungi) and then ODCS can pass these modules to Pungi (RFE for ODCS).

@hhorak do you have a reference to the custom stuff people use now?

This is what I did the last time:

for p in $(koji list-tagged --quiet scrmod-mariadb-10.3-8020020200212122856-6a468ee4_1+5716 | awk '{print $1}') ; do koji download-build -a x86_64 $p ; done

You can also do something like:

odcs create tag scrmod-mariadb-10.3-8020020200212122856-6a468ee4_1+5716

You will get the repository, but without modular metadata (which is the same results as for @hhorak's command above ^).

Probably also with --sigkey none to allow unsigned packages...

$ odcs --redhat create tag scrmod-mariadb-10.3-8020020200212122856-6a468ee4_1+5716 --sigkey none

@jkaluza could not be the modular metadata somehow generated from an existing module instead? We could provide this "baseline module" ... ?

The modular metadata contains the NVRs of built modules included in the module. This list of NVRs respects multilib, module filters, "whitelists" in modulemd and so on. It's not something you could directly take from older version of module.

The code which generates that modulemd file is here: https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/builder/KojiContentGenerator.py#_678.

The _fill_in_rpms_list is the most important method there: https://pagure.io/fm-orchestrator/blob/master/f/module_build_service/builder/KojiContentGenerator.py#_496.

I will create ticket in fm-orchestrator asking for some solution how to run that code for scratch-builds and make it available for Pungi to consume. Let's see.

@jkaluza thanks, we had discussion and this will become a blocker until we are able to reasonably consume this content ...

Any additional issue will definitely help for tracking, so we can follow and whine :dollar:

^ For a moment I though you are offering me a "wine" if I code it :).

@jkaluza wine also works for me, I make it even 2!

Would be nice if the .repo created by odcs had the module_hotfix=True on it, while this feature is not implemented and we have to keep using the repo without modular data.

But only if it's a low hanging fruit. I rather wait for the ability to run odcs compose on scratch buidls tough =)

Login to comment on this ticket.