Hello,
There will be an outage starting at:
$ date --date '2025-12-10 07:00 UTC'
The outage will last approximately 3 hours. The copr-backend storage (copr build results) will stay mostly online during this outage but some downtime is expected.
Reason for outage: We're updating copr packages to the new versions which will bring new features and bugfixes.
Affected Services: copr-frontend - https://copr.fedorainfracloud.org copr-backend - https://copr-be.cloud.fedoraproject.org/
Contact persons: @praiskup / @nikromen
Please join Fedora Build System Matrix channel https://matrix.to/#/#buildsys:fedoraproject.org or comment on this ticket.
Metadata Update from @zlopez: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: copr, medium-gain, medium-trouble, ops, outage
Metadata Update from @praiskup: - Issue assigned to nikromen
The update is done. We are facing some weird DDoS issues, though -- the front page is under a heavy load, at least a dozen of various bots are hitting us (it's weird, these things repeatedly happen during/after our planned outages, we start to feel this is a pattern).
Commit 50223282 relates to this ticket
None of my gcc snapshot builds are working, which appears to be timed with this outage. Builders are consistently failing with this error:
bash: line 1: /usr/bin/rpmbuild: No such file or directory
Full logs here for example:
https://download.copr.fedorainfracloud.org/results/siddhesh/gcc-snapshots/fedora-rawhide-aarch64/09899526-gcc/builder-live.log.gz
and this is the build in question:
https://copr.fedorainfracloud.org/coprs/siddhesh/gcc-snapshots/build/9899526/
which is largely working fine in koji:
https://koji.fedoraproject.org/koji/taskinfo?taskID=139913942
although even Koji appears to be having its own difficulties, probably unrelated:
Traceback (most recent call last): File "/usr/lib/python3.14/site-packages/koji/daemon.py", line 1421, in runTask response = (handler.run(),) ~~~~~~~~~~~^^ File "/usr/lib/python3.14/site-packages/koji/tasks.py", line 341, in run self.createWorkdir() ~~~~~~~~~~~~~~~~~~^^ File "/usr/lib/python3.14/site-packages/koji/tasks.py", line 367, in createWorkdir os.makedirs(self.workdir) ~~~~~~~~~~~^^^^^^^^^^^^^^ File "<frozen os>", line 226, in makedirs File "<frozen os>", line 236, in makedirs OSError: [Errno 5] Input/output error: '/var/tmp/koji/tasks/3971'
Sorry, that was probably a self-goal; the partially finished snapshot build got in the way of the new ones.
The i/o error builder was sacked. :)
This outage is over, so lets close this....
Metadata Update from @kevin: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.