#8590 Some builders have no disk space free - again
Closed: Fixed 4 years ago by kevin. Opened 4 years ago by kkeithle.

A few builders have no space free, free up space on them and investigate why this is happening.

e.g. ppc64le, https://kojipkgs.fedoraproject.org//work/tasks/5417/41095417/build.log




at least buildvm-ppc64le-12.ppc.fedoraproject.org

Multiple builders have over a hundred builds on them from the mass rebuild. I am not sure what I can clean without breaking that.

Not sure, but we have had to restart kernel builds several times because 1 arch fails due to lack of space, a couple of hours ago, it was ppc64, now it is arm.

The mass rebuild has not yet started. ;) It should in a few here.

The problem is koschei. It's rebuilding failing packages too quickly. For example, it's done ~600 builds of python-debtcollector in the last day. At ~350MB per buildroot, thats... a lot of space.

So:

I have filed: https://pagure.io/koschei/issue/2

I have disabled koschei for now (for the mass rebuild and this issue)

I have cleaned out space on all builders.

So, we should be ok for the mass rebuild and any packages that had problems before. Very sorry for this issue...

Metadata Update from @kevin:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

4 years ago

Login to comment on this ticket.

Metadata