Learn more about these different git repos.
Other Git URLs
@jgreguske has put a ton of work into "signed repos" in koji. It works, is cool, and solves a lot of other problems for us.
We can leverage it relatively easily in bodhi's workflow. The signed-repos task takes a tag as its input, and it generates and signs a repo based on that tag. Bodhi needs to create testing and updates repos for each release. It already manages tags for those repos, so it should "just work" :tm:.
With pungi, we're in a mess. We have the variants/editions that we need to consider: Server, Workstation, Cloudliness.
The definition of what goes into those can be defined either in comps.xml, or in a JSON mapping. Pungi reads in one of those two, depending on its configuration, and uses that to sort out what goes into each of the variants/editions.
One idea we had, was to pass comps.xml (or the json mapping) along into koji, and into the signed repo task. The signed repo task would then take all of the rpms in the everything tag, and then smartly sort them out into different repos after reading the big comps list we passed it. That feels wrong.
Another option would be to have pungi orchestrate some ephemeral tags for its work. It would read in the comps/json definition (whichever) and then automatically tag everything into some temporary tags, ask the signed repo task to create and sign repos for those tags, then go and do the rest of its work. At the end of the compose, it would untag everything from those temporary tags to clean up after itself.
One issue with this is that we wouldn't be able to run two composes at the same time. To solve that, we should have the tag names prefixed or suffixed with the compose ID, so they'll be unique.
There are use cases where a single build is split across variants in RHEL. Some subpackages go to Server, some go to Desktop, and others go to Optional. I'm not sure if Fedora already does this across editions. In either case the suggestion of having a tag-per-compose would not work because the most granular control you have is by build, which means all subpackages or none of them.
The method in which pungi accepts package lists needs to be re-designed. Comps needs to go too, because a translated XML file is very difficult to integrate between services. We need comps-the-API, comps-as-a-service, Internet-of-Comps, or whatever the current buzzword is that describes accessing comps remotely in a database.
I personally would rather yum/dnf not be involved at all when generating repositories or media. Instead, the complete package list would be known in advance before build time. That would enable us to actually QA our tools to ensure the right content is being included. Today we depend on yum/dnf to do the right thing, and when they do not, we have no way of knowing in advance of delivering the compose to QA.
I need to do an interview during lunch but when I come up with a solution I'll post it here. I know all I've done so far is point out problems here.
This would be even more difficult: pungi can add additional packages or filter some packages out. It would be necessary to produce something else apart from the actual comps/json files.
Agreed. Let's not go that route.
Yeah, even if Fedora doesn't do this now I think its reasonable to expect that we would want to some day. Let's plan to support it.
We talked in IRC about if it would be okay to include the whole build in the tag (and therefore in the signed repo) for pungi. All of the subpackages would be included in the repo, and signed, but then subsequent pungi phases could just ignore those extra subpackages and leave them out of the built artifacts. We decided that this would be okay for most media, (since the %packages section in the kickstart file will let us exclude stuff we don't need), but that for install media this might be a blocker. We still need to figure out if it is actually a blocker or not.
%packages
Agreed, but this is really another issue - something to file under the Factory2 bonanza. Something beyond our immediate F25 timeframe.
To make it even more complicated, I expect that the comps-as-a-service service would need to be able to specify what goes into each variant/edition in terms of modules, not in terms of packages. So, down the road a bit.
Interesting. This was not on my radar, but makes sense.
I like the idea of abolishing depsolving and using a full package list as input. That would probably need some other tools to generate the list, though, as it is not feasible to do manually. E.g. Fedora 24 Workstation has 12 comps groups with 811 packages in total, but on x86_64 with depsolving it ends up at 2790 rpms.
See #713 for more information about using this koji feature.
Log in to comment on this ticket.