It turns out that Koji will clean up distrepos regularly, which means that some time after things get tagged into infra tags, the repo will be cleaned up and unavailable. We need to regularly regenerate them with tag2distrepo.
Perhaps a daily cron? Or how often does it nuke them?
So, I spoke to Dennis, and it should not nuke the latest (but it seemingly does). A daily cron would be just fine, as I think it cleans them with the standard koji-gc policy.
Because tag used for dist repos is also source tag of a build target, kojira is expiring and deleting dist repos when newer build repo is generated.
Example timeline: 2018-02-15 09:04:27 bodhi starts generating dist repo for f27-infra 2018-02-15 09:05:28 dist repo is generated 2018-02-15 10:24:40 kojira considers dist-repo as non-current (because tag listing changed) 2018-02-15 10:29:20 kojira starts generating build repo for f27-infra 2018-02-15 10:49:35 f27-infra build repo is generated by kojid 2018-02-15 10:49:35 kojira marks dist repo as expired and deletes it from NFS
Proposed solution 1: set up different tag for dist repos Proposed solution 2: fix kojira not to expire dist repos
This is still an issue and it blocks my during mornings when the only few Koji admins from Europe timezones are away. Can this be prioritized?
I think we could add:
ignore_tags = infra infra-stg
Does that seem like it would be reasonable? If so, can push a FBR with it...
Metadata Update from @kevin: - Issue assigned to kevin - Issue priority set to: Waiting on Reporter
Given the proper command and ACLs, I'd be willing to help regenerating in Europ's mornings if that can help mitigate this.
Alternatively, we can do something like what we are doing for non-infra tags, which might also resolve another question I've gotten from time to time: add $rel-infra-build tags, which inherit from $rel-infra and are used as build tags. The advantages would be that: 1. The $rel-infra tags are no longer build tags, so the repos don't get garbage collected. 2. It becomes possible to build on top of packages even before they make it to production (e.g. when someone wants to build a stack of new updates without pushing anything to prod yet).
$rel-infra-build
$rel-infra
Sounds good to me. Lets try it!
Metadata Update from @kevin: - Issue priority set to: Next Meeting (was: Waiting on Reporter)
This is now done for f27, f28, f29 and epel7.
Thanks @puiterwijk !
:ideograph_advantage:
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.