Planned Outage - koji upgrade - 2024-10-09 21:00 UTC
There will be an outage starting at 2024-10-09 21:00 UTC, which will last approximately 2 hours.
To convert UTC to your local time, take a look at http://fedoraproject.org/wiki/Infrastructure/UTCHowto or run:
date -d '2024-10-09 21:00UTC'
Reason for outage:
We will be upgrading koji to the latest upstream version, 1.35.0 with various bugfixes and enhancements. During the outage the koji hubs will be down as the database schema is updated, and various builders may restart as their koji version is updated. Additionally, we will be reinstalling some virthosts with rhel9.
Affected Services:
koji.fedoraproject.org
Ticket Link:
https://pagure.io/fedora-infrastructure/issue/1221
Please join #fedora-admin or #fedora-noc on irc.libera.chat or #admin:fedoraproject.org / #noc:fedoraproject.org on matrix. Please add comments to the ticket for this outage above.
Updated status for this outage may be available at https://www.fedorastatus.org/
Metadata Update from @zlopez: - Issue priority set to: Waiting on Assignee (was: Needs Review) - Issue tagged with: high-gain, medium-trouble, outage
I think it would be wise to bring down Bodhi during Koji outage. I don't think Bodhi will be happy failing tagging builds and we might end up with a lot of builds out of synch with update statuses.
I've stopped Koschei scheduler in advance to reduce the load on Koji during the outage. I will resume it once the outage is over and Koji is fully operational. Koschei scheduler can be resumed by running: oc -n koschei rollout resume deploy/scheduler.
oc -n koschei rollout resume deploy/scheduler
Metadata Update from @mizdebsk: - Issue tagged with: koji
Thanks @mizdebsk and @mattia
Outage is complete. Please report any issues as normal.
I resumed koschi and bodhi should be all scaled back up too.
Metadata Update from @kevin: - Issue close_status updated to: Fixed with Explanation - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.