#97 RFE (and policy change request): automatic creation of module repos and branches for modules which comprise a single RPM
Opened a year ago by mattdm. Modified 4 months ago

If we're going to a "hybrid" model where the platform/base is the traditional "everything" release, a lot of the dangers of unreviewed modules go away. I'd like to propose that when creating an arbitrary branch with fedrepo, an additional flag be available (and perhaps be the default) which would cause a matching repo and branch to be created in the module namespace.

This module repo would automatically be populated with a minimal module.md, which is pretty simple and can have all of the things that need to be unique auto-filled, because it's just srpm name, summary, and description.

document: modulemd
version: 1
data:
   summary: $RPMSUMMARY
   description: >-
        $RPMDESCRIPTION
    license:
        module:
            - MIT
    components:
        rpms:
            $SRPMNAME:
                 rationale: The single package in this module.

Module review would be triggered if the module.md is edited.

This would significantly reduce the barrier to getting people to create modules — basically, for simple cases, creating the branch would do it. And it will also reduce reviewer backlog, because there's no need for human review of trivial things like this.


(Also, I guess, a way to do the same thing for the arbitrary branches which already exist. Like for the current ones, or if someone doesn't use the flag and later changes their mind.)

This seems like a completely reasonable proposal and I'm behind it 100%.

Note specifically that the api and filter sections are excluded, which means that it will ship the complete package with all of its subpackages, but will not assert that any part of it is "API".

@mprahl - does this seem do-able to you (in the New Year, of course...)?

@mprahl - does this seem do-able to you (in the New Year, of course...)?

I think so. Any ideas on how to programatically get the license though? My only thought now is to have the user manually enter it in as an option in their fedpkg request-repo command or we can scrape the Bugzilla bug comments for the URL of the spec file and get the license from it.

@mprahl Do you mean the content license? (The RPM license?) That's optional, right? I guess this indicates that the generated file should include the comments. :)

This does bring up something for me, though. I was thinking of this from the perspective of adding arbitrary branches to existing dist-git repositories, not the situation where a new repository is created in the new modular world intended for the module reop only and there are only these branches.

In the already-existing case, you could get all of this information from the master branch of rpms/dist-git/$PACKAGENAME/$PACKAGENAME.spec. And also there isn't a bugzilla URL.

We might actually want the template module.md to include some other fields commented out, to make extension easier. But we can figure that out later.

@nphilipp could you review and respond to whether "fedmod rpm2module" does this?

@ralph would "creating the module repo" be something that we could add to fedpkg? In the case of modules with fedora approved rpms, I think we can argue creating the repos unreviewed is ok because: a) the srpm(s) has already been reviewed b) the module is not actually included in any release until a separate step of adding it to the "variants-modular.xml" is performed.

@mprahl Do you mean the content license? (The RPM license?) That's optional, right? I guess this indicates that the generated file should include the comments. :)
This does bring up something for me, though. I was thinking of this from the perspective of adding arbitrary branches to existing dist-git repositories, not the situation where a new repository is created in the new modular world intended for the module reop only and there are only these branches.
In the already-existing case, you could get all of this information from the master branch of rpms/dist-git/$PACKAGENAME/$PACKAGENAME.spec. And also there isn't a bugzilla URL.

You are right, I thought the content licenses were required but they are in fact optional.

Right, the content licenses are automatically generates by MBS, so they'll get populated by the build process.

@langdon, sorry for the late reply but Pagure didn't ping me when you mentioned my name. Or I missed it somehow.

Anyway, in theory fedmod rpm2module would do even more (like: add dependencies which aren't in host&platform) as long as you run fedmod fetch-metadata first. But right now either one fails for me ('/home/nils/.cache/fedmod/f27-module-contents-cache.json' does not exist...), so I can't give you a 100% answer. I guess the failure is related to recent changes with the new hybrid approach, no idea if that requires us to accommodate to it in fedmod (when the changes have landed fully).

@nphilipp I want it super simple. I keep hearing "this modulemd thing is sooooo complicaaaatted"... and with the full thing, it's a fair complaint.

@mattdm that was just me thinking out loud. If all works well when the necessary repodata is back, I expect the current code to figure out that everything else is available (yay hybrid model) and only add the package itself.

Just please add option to opt-out for this. I'm not interested for using this for any 600+ of my packages.

I think it would be an extra manual flag when requesting the new package/new branch.

@ignatenkobrain What exactly do you want to opt out of? You want to create stream branches for those 600 packages, but you want to create a module for each one manually, and you want to have to file a separate ticket to get the module repo created, and you don't want to have a simple module.md in place to start from?

The situation where one would want to opt-out would be when stream branches exist for leaf packages which are intended to only be built as part of a bigger module. That might indeed be the case for many of @ignatenkobrain's packages, but from @sgallagh and @langdon I think we expect 80% of modules to correspond closely to packages, so I think it's probably good to be on by default and opt-out where desired.

This request is based on @sgallagh and @langdon's estimation that 80% of modules will be basically single-package, combined with the idea that having a module is the only way to enable a stream branch. At DevConf, we're talking about enabling some way to link stream branches to builds without having a module at all.

That'd make sense for single packages where we expect to only make a single stream available. I need to wrap my head around that a bit — I think the case where there are multiple options is going to be more common than people are thinking right now, but maybe I'm wrong. If I am wrong, we probably want this to be still available, but not the default when creating a stream branch after all.

Sooooo, what came out of those stream-branches-without-modules discussions? Are we interested going that way? If we're not, I'd like to move forward with this, so that it's really easy for people to create new modules.

So, while I don't think modulemd is complicated and so far people haven't seem to be having much problems with it, I still consider this relevant.

@nphilipp is now working on restructuring fedmod so that it is more useful for the hybrid model. It could then be integrated into fedpkg as noted above. This will still take some time, however.

To @mattdm 's question, it seems the idea of package stream branches without modules is dead. But as @psabata points out, creating modules in a simple way is relevant. The docs have it covered: https://docs.fedoraproject.org/fedora-project/subprojects/fesco/en-US/making-modules/adding-new-modules.html

I believe this is now done and documented in the Fedora Docs / Making Modules / Adding New Modules

When requesting a new stream branch for a package, the module repo + branch requests are automatically created:

$ fedpkg request-branch --repo=NAME --sl rawhide:2020-12-01 BRANCH

To create a new stream branch for a package without the module repo + branch requests:

fedpkg request-branch --repo=NAME --sl rawhide:2020-12-01 --no-auto-module BRANCH

More info in the link above.

Should we close this?

Login to comment on this ticket.

Metadata