#11504 Retire osbs clusters
Closed: Fixed with Explanation 10 months ago by kevin. Opened a year ago by kevin.

This ticket is mostly for visibility and collecting any blockers from anyone. The actual work should be pretty simple.

Currently we have 4 osbs clusters. Two of them (one staging and one prod) are x86_64 vm's and running rhel7 and openshift 3.something. The other two are aarch64 (one staging and one prod) running fedora 34 and openshift origin 3.something.

In the past they were used for:

  • Building containers. However, most all of those moved to building in quay.io. The only one left was toolbox, but in fedora 39 we have moved this to using imagefactory like the base and base-minimal images. They do not use osbs at all, it's all done in imagefactory on builders.

  • Building flatpaks. However, in fedora 39 we have moved (thanks to all the tireless work of @otaylor and the flatpak sig) to using a koji plugin that just builds them on builders and doesn't use OSBS at all anymore.

I'd like to:

  • Wait until Fedora 39 is released
  • Wait to confirm that flatpaks are all switched and working with the new setup
  • Have toolbox maintainers push last f37/f38 updates to toolbox
  • shutdown the clusters for a few weeks
  • If no need for them, undefine and reclaim resources. This will also involve removing some packages from builders and config also.

If anyone sees a problem with this plan or something I didn't think of, please shout. ;)


Metadata Update from @kevin:
- Issue assigned to kevin

a year ago

Metadata Update from @zlopez:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: OSBS, high-gain, medium-trouble, ops

a year ago

While we really don't want to update F38 flatpak apps, we do still have a few that aren't ported to F39 yet, and more importantly, we still need to have the means to EOL the F37 and F38 runtimes, plus at least one app which was retired in F39. EOLing a flatpak involves creating one last build of it. The F37 runtimes could probably be done soon, but we can't do the same for F38 until all remaining apps are ported or EOL'd themselves. So we may need just a bit more time to clean that up before we can turn out the lights.

/cc @otaylor @kalev

In light of osbs-stg being decommissioned today (albeit unplanned), a status update from our end.

We are continuing to work on migrating the remaining f38 flatpaks to f39 and the new tooling. Three (3, out of ~300) flatpaks remain to be migrated, each requiring additional work to migrate which is underway but not yet complete. Once complete, we will then need a final EOL build of the f38 runtime+sdk, and then we should be done. We want to make this happen in a matter of weeks (rather than months) but not all the expected changes are within our control.

FYI, the toolbox maintainers also wish to push out a last update of toolbox on f37/f38 before we turn out the lights.

Any news here? Any ideas when this might be done? I would really like to retire those clusters.

We finally have fixes in flight for the last two apps. We have been unable to make a new EOL build of the F38 runtime+sdk, but there is a build from December. Therefore, I think we can safely say that the old flatpak tooling can be retired, but @otaylor should have the final word here.

@kevin @otaylor the last two apps have finally been migrated to F39, currently in or on their way to testing, along with the December EOL runtime builds. We should have no further need for the old flatpak tooling.

excellent news.

I will plan on doing the retirement after beta freeze is up.

ok, I have retired things. I did save the disk images off and virt xml. In case of emergency we could revert the ansible commit and bring things back.
But hopefully we will not have to. ;)

Metadata Update from @kevin:
- Issue close_status updated to: Fixed with Explanation
- Issue status updated to: Closed (was: Open)

10 months ago

Log in to comment on this ticket.

Metadata
Boards 1
ops Status: Backlog