#2800 unplanned OUTAGE on buildsystem (2011-06-01)
Closed: Fixed None Opened 9 years ago by kevin.

During reboots and updates yesterday, the filesystem for the
buildsystem was written to from two seperate places at the same time.

A snapshot was taken of the filesystem and a fsck started.

Builds were completing normally and the filesystem was functioning,
until about 08:11UTC when the filesystem switched to read-only.
Builds since that time have failed.

We have disabled the builder hosts for now (submitted jobs will queue
up and wait for their return) until the file system check is completed.

You can follow progress of this issue in:

Sorry for the inconvenience.

This outage is over.

A full filesystem check was run on the koji filesystem, and then issues were checked.
New repodata was made for all build targets.

Login to comment on this ticket.