#2728 Better utilization of arch builders
Opened 3 years ago by msuchy. Modified 2 years ago

While browsing the code because of #2726 I found that for noarch packages Koji pickup first suitable architecture:

https://pagure.io/koji/blob/master/f/plugins/builder/runroot.py#_140

It would be nice if Koji considers the waiting queue and from the list (set) of arches it picks up the architecture with the shortest queue.

This can lead to better utilization of builders.


The code you're linking is doing something different. It just tests if given builder's arch intersects with some arch available in buildroot. If yes, then this builder can pick the runroot task (and only runroot task).

For buildArch tasks for noarch packages it works like you describe. Every builder is put to bin (channel/arch) and from that bin builder with least load is picked. There is some heuristics but basically it does this.

Metadata Update from @tkopecek:
- Custom field Size adjusted to None

3 years ago

It would be nice to be able to stop noarch builds from being picked up by certain builders. We have some slow arm64 machines we would like to dedicate to aarch64 builds exclusively, but right now they're also picking up noarch builds and slowing things down.

FTR, the centrally-scheduled system used by Copr (resalloc) is able to arbitrarily
"tag" builders, and the builders (or pools of builders) not tagged by say
noarch_builder tag would not be picked up. But as @tkopecek informed me
some time ago, Koji buildsystem uses some kind of de-centralized worker
scheduler.

Bringing this usecase to @mikem's eyes. It could be some of the usecases for scheduler redesign.

Metadata Update from @tkopecek:
- Issue tagged with: scheduler

2 years ago

Login to comment on this ticket.

Metadata