#126 Automatically track refs in a module yaml file
Opened 4 months ago by rcritten. Modified 4 months ago

As a module developer I want to have more automated control when rebuilding a module to pick up package changes.

For example, during development I'd respin the module frequently to pick up whatever the latest packages are. Eventually I go into production and want to lock those package versions so if I need to rebuild the module to pick up a change in one package I don't want to pull in any other packages that may have been rebuilt as well unless I explicitly request it.

There is a mechanism now, by specifying the refs, but it is a manual process to pull the modulemd.txt and manually update the yaml file.

It would be helpful to have a tool that takes a latest tagged module a production build and makes a module.yaml update with the refs for it.

No, we want to have an integration with fedpkg/rhpkg that takes this information and updates existing module.yaml file.

How about approaching this from the other side and tagging the commits you're interested in in the rpms namespace? You can then just move the tag when you want to update that component.

This is a big no-no, sorry. Moving tags breaks builds reproducibility.

@abbra That's true if you're expecting that sources are everything you need to reproduce a build. But that's not the case. I'd argue that changing the buildroot (=building any other packages in Fedora) breaks reproducibility as well if you're only using the source information to reproduce builds. To reproduce builds correctly, I'd suggest using the information in Koji as well including package NEVRAs in the buildroot, the modulemd I linked above that has all the refs, etc.

As I said, we are looking for automation that allows to avoid stupid human errors. You have all the means in place, please add the automation.

Having to look up manually in Koji/collecting NEVRAs, etc. does not scale and is prone for mistakes. Reviewing generated update is easier for a human, so please help poor humans. :)

Please note it is not just Fedora that is affected. For example, in RHEL case while there is stricter control on what can be committed to a particular release, building a module creates a large uncertainty for zstreams exactly because a module will pick up everything in the components that was changed since the last build if we would not pin the references. Having to do that manually hurts a lot. ;)

Metadata Update from @nphilipp:
- Issue tagged with: Meeting

4 months ago

I'll add this to the agenda of the team IRC tomorrow in #fedora-meeting-3 at 15:00 UTC. We'd gladly have you there and discuss this.

Correction: because of conflicts we'll cancel the meeting tomorrow and have the discussion the week after (if nothing gets in between by then :wink:).

So, writing a tool that does what you want would be trivial. Finding the right place for that code, that's a different story :)

I was also thinking we could add a feature to MBS, an extension to the rebuild strategies, that would allow you to list components that you want to build and ignore the rest. Basically just limiting the only-changed set even further. Something like fedpkg module-build --components foo,bar. Would you find that useful or even preferable?

Metadata Update from @asamalik:
- Issue untagged with: Meeting

4 months ago

@rcritten Would the thing from the previous comment work for you?

Login to comment on this ticket.