#12147 Retire EPEL 8 Next
Opened 2 months ago by carlwgeorge. Modified 11 days ago

  • Describe the issue

The time has come to retire EPEL 8 Next. This is our first time retiring a branch of EPEL Next, so we're figuring it out as we go. I'm creating this ticket to track the work and to mark it's completion. I don't have permission to assign tickets in this repo, but feel free to assign this to me if possible.

  • When do you need this? (YYYY/MM/DD)

The buildroot for EPEL 8 Next is CentOS Stream 8, which itself went EOL on 2024-05-31. We can start the retirement at any time, and it's done when it's done.

  • When is this no longer needed or useful? (YYYY/MM/DD)

It will always be useful to get this retired.

  • If we cannot complete your request, what is the impact?

Users might continue creating EPEL 8 Next builds against a buildroot of unmaintained packages.

Metadata Update from @jnsamyak:
- Issue assigned to carlgeorge
- Issue tagged with: high-gain, high-trouble, ops

2 months ago

Awesome, the standard EOL process is here: https://docs.fedoraproject.org/en-US/infra/release_guide/release_eol/ (Just for the reference)
, I have assigned this ticket to you, it will be nice to have the sop for this documented once this ticket ends!

In case you need me for anything, tag me :D

Additional items should be done in infrastructure side:
1. Stop and remove the cron job on batcave01 which tries to download the centos-stream8 and This is the script /etc/cron.d/sync-centos8
2. Stop and remove the cron job on batcave01 which runs grobisplitter daily on the downloads. This is the script /etc/cron.d/cents8-split.cron
3. Remove these entries from the Fedora Infrastucture ansible so they don't get readded.
4. Remove the /mnt/fedora/app/fi-repo/centos/centos-8 and /mnt/fedora/app/fi-repo/centos/centos-8-stream directories
5. Whatever other items in the ansible or administration I have either forgotten or have changed since I was in fedora infra :jack_o_lantern:

I've got step 3 implemented in this pr.

You should be able to remove the targets anytime (sooner the better so people don't try and do new builds)

If you like we can do the bodhi archiving step, the pdc step, and the move to archive steps?

You should be able to remove the targets anytime (sooner the better so people don't try and do new builds)


If you like we can do the bodhi archiving step, the pdc step, and the move to archive steps?

Yes please. I believe the bodhi archive command will be:

bodhi releases edit --name EPEL-8N --state archived

I tried to run this myself with the --staging flag, but I don't appear to have permission for that.

I don't think we actually need to move anything to the archive, because in theory all the builds that were in EPEL 8 Next should have a newer build upgrade path in EPEL 8.

bodhi archiving step done.

If you view the EPEL 8 Next release page in bodhi it does show the state as archived, however the releases page still lists EPEL 8 Next as a current release. Is this expected, or is something missing to mark the release as archived/inactive?

I'm not sure... @mattia might know? It might be that it never expects a 'pending' release to be 'archived'.

That's just a cache problem on the releases web page which I can't figure out. The method which returns the list of releases with their properties seems to return the right output, but the web page is not updated.

If it's just a cache problem I guess it will eventually go away on its own. It doesn't seem to be hurting anything at the moment.

To move forward with the retirement, I think these are the steps that are left:

  • delete /mnt/fedora/app/fi-repo/centos/centos-8-stream on batcave
  • delete /mnt/fedora/app/fi-repo/centos/centos-8 on batcave (technically unrelated to this retirement, but a might as well clean up at the same time)
  • delete https://dl.fedoraproject.org/pub/epel/next/8/
  • probably do something in PDC
  • mass retire epel8-next branches in dist-git repos
  • probably update epel-release-8 to obsolete epel-next-release-8

Sounds ok on the first 3.

I don't think we need to do anything in pdc... we could adjust the eol on all epel8-next branches, but... no one can build from them anyhow, so I don't think it matters. Also we are retiring pdc really soon.

The mass retire we could do I guess, but again, no one can build from it, we could just leave it?

Since stream 8 is gone already does it make sense to obsolete epel-next-release-8?

Glad to hear PDC is close to going away soon, one less thing to worry about.

I do think it's still worthwhile to retire epel8-next branches just from a cleanliness perspective, and to avoid maintainers even attempting to create builds from those branches. We didn't leave epel6 or f38 branches around after EOL, I don't see why epel8-next should be any different.

In the most recent countme dataset (2024-06-09 to 2024-06-16), there were over 100k non-CentOS-Stream systems requesting epel-next-8 repos, so I definitely think obsoleting epel-next-release will be worthwhile. I'll set up a PR for that. EDIT: Here is the PR.

We didn't leave epel6 or f38 branches around after EOL

We don't? https://src.fedoraproject.org/rpms/apg/tree/f38 for example? we don't retire all packages at eol. We just leave the branch there, but since it's eol currently the pdc hook will not let anyone push to them. (And soon a bodhi query will do the same thing less fine grained).

Huh, I must have imagined that then. I can see on the pagure branches page there are active and inactive branch sections, and I could have sworn that retired released had their branches marked as inactive, but I guess not. Nevermind then on that part, again one less thing to worry about. :grinning:

No, you're not hallucinating. It definitely was part of the SOP to retire the release in PDC because that allows pagure-dist-git to know that the corresponding branch should be marked as inactive and block pushing into it.

That code is here: https://pagure.io/pagure-dist-git/blob/master/f/dist_git_auth.py#_242-275

Thos packages aren't really retired in this case, they are just end of life. We don't for example push a dead.package commit for every package or unassign the maintainer(s) or other things we do if a package is retired in the active collections. Theres also no need to block them, as theres no composes or way that could ever go anywhere...

Log in to comment on this ticket.

Boards 1
Ops Status: Backlog