Learn more about these different git repos.
Other Git URLs
We've some discussions about this but it is not covered anywwhere.
Kojira nowdays regenerate too many repos from which many are never used. We want to redesign it a bit and create priority queue at the hub/db. Now is everything tracked and calculated in kojira which means that e.g. user can't demand faster repo regeneration via queue (only explicit spawning of the task).
So, let's collect requirements here before we start the design part.
1) User should be able to request repo for the build. In such case it should be reprioritized in queue accordingly to his permission level. 2) Some tasks could have also modify its repo regeneration priority (MBS, chainbuilds, ...) 3) kojira should only alter queue, not care about the task execution (it could be handled by new scheduler as recurring "maintenance" type of task) 4) deleting repos - not clear if it should be done in kojira or somewhere else (probably not hub due to IO, maybe some createrepo builder with RW access?) 5) maintenance daemon - that was another idea - Currently GC and kojira are separate tools, while they have parts which are similar (deleting old repos is basically GC task). So, maybe separate "maintenance" builder could look more like regular builder just with elevated privileges and handling all these tasks via events (kojira queue and/or timing - simply scheduling repeated GC task - it could be done also with new scheduler (e.g. new time field earliest_time until which task will not be picked)
Here is a very simple older PoC - I wouldn't do it exactly this way nowadays but is a source of inspiration.
https://pagure.io/fork/tkopecek/koji/c/664455e276ac44b52815f3eb17ae27e4687cf9b1?branch=priority-queue
Metadata Update from @tkopecek: - Custom field Size adjusted to None
PR #4027
In terms of the overall design, the main thing that we need is to move to "on demand" repo generation. Hence the notion of requests in point 1 and a queue for them (separate from the task queue) in point 3.
Apart from that, while this is an opportunity to also reconsider other aspects of kojira behavior, I think we'd do well to limit it to changes that are actually necessitated by the on-demand change.
Currently, kojira does the following things:
TBH, most of this activity will need to remain there, though some of it will need to change:
I think maintenance unification is a longer term goal. I would rather defer that part for the moment, though it certainly would be nice to have.
Login to comment on this ticket.