@ausil We thought the major issues with this approach is signing and koji-gc. But @mikeb confirmed that any packages tagged into -overrides should already be signed, because they're older packages and koji-gc shouldn't gc packages that are "latest" in a tag and anything tagged into -overrides would be latest. So, he thinks that with current configuration, it just works. We need your opinion on this.

FYI from todays meeting:

#info The plan seems okay so far, but we will get another review from dgilmore.

Discussed in today's meeting. Mohan is going to try to get Dennis to review this.

+1

from today's releng meeting:

13:41:44       dustymabe | #info waiting another week for dgilmore's input. the third time is the charm

There is a probability that the older updates have had the signed copies cleaned up on disk before hand, assuming it was the previous build.

Rather than using dist-repos we should use pungi to make the repo. A side effect of this change is that you are unlikely to get the result that you want. as when anaconda sees the updates in the updates repo witha higher nvr it will choose that. you will need to manage the entire chain of packages in the pungi compose to get your intended behaviour.

There is a probability that the older updates have had the signed copies cleaned up on disk before hand, assuming it was the previous build.

I believe @mikeb did not think this was a problem because if we tag them into f27-atomic-overrides they won't be gc by koji??

Rather than using dist-repos we should use pungi to make the repo.

dist repo was because we'd only need to change this repo few times and we can just run the create the dist repo command right after we tagged a package into the tag. The ostree creation (that runs during bodhi call to pungi) would pick up rpms from f27 release, the f27 updates run that was just done as part of the bodhi run, AND the dist-repo. The goal is to prevent having to create the atomic overrides repo as part of the bodhi run every day.

A side effect of this change is that you are unlikely to get the result that you want. as when anaconda sees the updates in the updates repo witha higher nvr it will choose that. you will need to manage the entire chain of packages in the pungi compose to get your intended behaviour.

The installer would come from the f27-atomic-installer tag which inherits from f27 release tag. Similar to today, it wouldn't pull in updates. The ostree that gets embedded in the installer would be the one from the bodhi updates run and it would already have the f27-atomic-overrides packages baked into the ostree. Anaconda has no ability to pull newer updates, the ostree is already baked.

The risk of koji-gc has nothing to do with the f27-atomic-overrides tag and everything to do with the signed copies being cleaned up before they are tagged into it. we can work around it by setting up some process to ensure they are signed and have signed copies written out to disk.

pungi builds the installer every time we run it to ensure that we have the correct bits inside of the installer. it has to gather the rpms and make the repos anyway as part of the compose process, you are adding additional overhead

so I talked with dgilmore (log below). The TL;DR is that he does not want to use a dist-repo but he is fine with the other parts of the proposal. I accept not using a dist-repo but building the f27-atomic-overrides repo every time we do an updates compose that includes an ostree creation. We should be able to move forward with this ticket.

09:33:36        +dgilmore | dustymabe: the pungi run will make the ostree, but we also need to make it make the isntaler dvd and the
                          | cloud images
09:33:50        dustymabe | dgilmore: which pungi run? 
09:33:59        +dgilmore | dustymabe: the one taht makes teh ostree
09:34:05        dustymabe | dgilmore: yes, that's bodhi
09:34:11        dustymabe | bodhi->pungi  
09:34:14        +dgilmore | dustymabe: the nightly cronjob that we have today needs to go
09:34:23        dustymabe | dgilmore: sure I don't disagree with that
09:34:41        +dgilmore | dustymabe: and the pungi run for making the ostree will also make the things taht today are done in the
                          | cronjob
09:34:57        dustymabe | dgilmore: ok. but I still don't think that means we shouldn't do a dist-repo
09:35:09        +dgilmore | ideally witha  agate that checks if any of the inputs has actually changed
09:35:10        dustymabe | since the content will rarely change
09:35:15        +dgilmore | so we only make it when things change
09:35:52        dustymabe | so we either create a dist-repo or we make the repo every time the bodhi->pungi run goes
09:35:57        dustymabe | i'm fine with either
09:36:06        dustymabe | was just trying to not complicate the bodhipungi run 
09:38:19              <-- | claudioz (~darklight@unaffiliated/darklight/x-2853543) has quit (Quit: My MacBook has gone to sleep.
                          | ZZZzzz…)
09:38:35        +dgilmore | dustymabe: we are making the repos anyway
09:39:49        dustymabe | dgilmore: so you prefer the 'non dist-repo' case ? 
09:40:12              --> | vedranm (~vedranm@inf2.uniri.hr) has joined #fedora-releng
09:40:21           smooge | Reminder: Fedora Infrastructure will meet at 1800 UTC today. This will be 1 hour earlier on the clock
                          | for many people. Please update gobby with any info as the meeting is rather bare
09:44:06        dustymabe | dgilmore: if so I'll update the ticket with our conversation
09:44:20        dustymabe | dgilmore: and I assume you have no other issues with the proposal
09:44:29        +dgilmore | dustymabe: I do
09:44:42              <-- | cqi (~cqi@125.33.67.165) has quit (Quit: Konversation terminated!)
09:44:48        +dgilmore | that is I do prefer the non dist-repo case
09:44:52        dustymabe | ok
09:45:14        dustymabe | and you are ok with everything else in the proposal?
09:45:23        +dgilmore | otherwise I am fine with it. though I am not afan of special casing the config to pull in older nvrs
09:45:51        dustymabe | dgilmore: indeed, I understand the concern
09:45:55        dustymabe | thanks for the chat I appreciate it 

from the meeting today:

12:14:10          mboddu | #info Except for the dist-repo part, we are okay on the proposal. The tasks are divided as: releng has to
                         | create tags and dustymabe will take care of config changes and infra will handle "atomic" koji permission

so we can move forward with these changes. The only other thing that came up in the meeting is, considering we will now ship updates for Atomic Workstation, should we namespace these tags. I think we should so that would mean we would create a tags like f27-atomichost-installer and f27-atomichost-overrides instead of what was proposed above.

ok discussed in the meeting today we're going to go with f27-atomic-host-installer and f27-atomic-host-overrides. Please add those two new tags and permissions for @dustymabe @cgwalters and @jlebon.

mohan showed me these commands and had me review them. LGTM:

$ koji add-tag f27-atomic-host-installer --parent=f27
$ koji add-tag f27-atomic-host-overrides
$ koji edit-tag f27-atomic-host-installer --perm=atomic
$ koji edit-tag f27-atomic-host-overrides --perm=atomic
$ koji grant-permission atomic dustymabe walters jlebon

I am okay with that being run. We do however need to have a tracker issue in pagure where we keep track of overrides. When something is tagged into the overrides tag it needs to be listed in the tracker issue witha reference to the bug that is being solved. I would like to better understand how often we need to use the permission and why. at the very least to be able to point the CI team at to show them some needed test cases to be covered.

@ausil sure - we can create the tracker issue in the atomic-wg pagure if you're ok with that

@dustymabe Can we just use releng repo? Its more a tracking for us on whats going on in the overrides.

sure: created this ticket: Tracker for packages tagged into f27 atomic tags

$ koji add-tag f27-atomic-host-installer --parent=f27
$ koji add-tag f27-atomic-host-overrides
$ koji grant-permission --new atomic mohanboddu
$ koji edit-tag f27-atomic-host-installer --perm=atomic
$ koji edit-tag f27-atomic-host-overrides --perm=atomic
$ koji grant-permission atomic dustymabe walters jlebon

All the necessary tags and permissions are created.