#33 handling of builders diasappearing
Opened 6 years ago by ausil. Modified 3 months ago

Sometimes a builder disappears for some reason, when it does the tasks on it just sit there never to complete until the builder gets restarted.

We need a way to detect that a builder has not checked in, then determine if its because it is completing some koji task. or some other reason, (i.e. kojid died, the builder fell over, etc) and free the tasks so that they can be picked up by a active builder and provide some notification that the builder is MIA.


Should this be done by kojira?

Should this be done by kojira?

maybe. not really sure where it would best fit. kojira does seem somewhat logical.

Yeah, I think kojira may grow into a more general koji maintenance service.

After the task manager taking a task, it will fork a task handler process to do the task. If a task handler crashed (so the task is not failed, canceled, or closed), and the task manager will free these tasks. So here the builder disappearing means the task manager disappeared, right?

I do wonder, since the kojira is a daemon that is in charge of admin yum repositories, does the kojira will be too general to check the builder die or not?

I think that the koji system maybe require a more power way to control and insulate processes.

Metadata Update from @mikem:
- Issue tagged with: backlog, feature

3 years ago

Discussed this in Koji grooming call. We would like to keep this open and address it as part of broader task scheduling refactoring in the future.

Metadata Update from @dgregor:
- Issue untagged with: backlog, feature

3 years ago

Metadata Update from @dgregor:
- Issue tagged with: backlog, feature

3 years ago

Metadata Update from @yulwang:
- Issue priority set to: Normal
- Issue tagged with: groomed

3 years ago

Metadata Update from @tkopecek:
- Custom field Size adjusted to None
- Issue tagged with: scheduler

3 months ago

Login to comment on this ticket.

Metadata