@zlopez do you have an example of cleaning the post release doc ?
Anyway, I can work on it , but need clarification from someone who worked on it .

@kevin Do we have some docs for post release steps that needs to be done by Fedora Infra or is this part of release engineering guide?

Really the 'cleaning' here should just be 'oc delete project/$i' for all projects (that are not openshift-*/system ones)

Perhaps we could enhance the existing playbook for communishift to remove the projects instead of adding them, then you run that and remove them from the list in git.

We do have https://docs.fedoraproject.org/en-US/infra/release_guide/final_release/ but perhaps we could add a 'post final release' one?

@kevin I agree that adding post final release guide makes sense as this shouldn't be bothering people during release.

After DevConf.CZ and discussion happening there, I would like to add that part of this should be also some announcements sent to contacts for communishift projects that will let owners know the post release cleaning is coming.
This could happen a month before the release and then again 14 days, so owners have time to react.

I saw this get built by someone at my university some years back, so maybe its not too hard to adapt to the communishift cluster and infra: https://github.com/WillNilges/ShelfLife

looks like it was also built for an openshift cluster and i believe it was also set up to notify people via email when their stuff was about to expire.

As far as scheduling when things get deleted, i think it may be nice to reuse the same date as the EOL-ing of F[N-2] (so 2024-11-12 for F41, see https://fedorapeople.org/groups/schedule/f-41/f-41-key-tasks.html) to give users an extra heads up well in advance (i.e. the user docs could document that the deletion date is around the time of the EOLing of the fedora release).

That said, an arbitrary date like that that may change every cycle may be a moving target thats hard to pin down and automate things against.

We will have this as part of Fedora Infra hackfest to discuss when and how to delete the project. The delete with EOL date makes sense, but how much before it we should notify owners of the project. And all of this should be documented as well.

For now we have two requests for not deleting the projects:

• communishift-log-detective
• communishift-fedora-review-service

As Fedora 39 EOL is coming near, we should move this forward.

@dkirwan Did you have time to work on this?

@zlopez when this is needed can you assign this task of clean the cluster? I'll go through what ever is involved manually, add an ansible task and gather all the moving parts into a SOP.

@dkirwan What task do you mean?

The SOP would be great, we also need to let owners of the projects know about that each cycle of cleaning. So they can react in time.

Looking at adding a playbook to retrieve project admin users emails from each project then send them a notification email about incoming project deletions.

Considering the following template:

to: "{{ item.value.comma_separated_admin_email_list}}"
from: "admin@fedoraproject.org"
subject: "Fedora Communishift Notification: {{ item.value.name }}"
body: "Dear Administrator,

This is a reminder that the Communishift project {{ item.value.name }} will be deleted during the Fedora post release process. Please ensure you have a backup of any important configuration or data from your project.

To request an exemption for project deletion please open a ticket here: https://pagure.io/fedora-infrastructure/issues"

Might make it more discoverable when the date is?

I guess we decided to do this when EOLing the old fedora release. So, in this case it's 2024-11-26 that F39 goes eol. We could put in the date, or just try and say something like 'at the same time the oldest fedora is retired (1 month after a new fedora release)" or something?

I also wonder, perhaps we should make a ticket in advance and just tell people to add to that one ticket instead of filing a bunch of them?
("To request an exemption for project deletion please add your request to https://pagure.io/fedora-infrastructure/issue/12345678")

Otherwise looks good.

I like the idea of having one ticket created in advance, we can close it once the cleanup is done.

Updated the communishfit role, and added a new playbook. Can retrieve the communishift projects plus the email addresses of the admins from fasjson. Also have task that can then send the email template in ansible.

Final piece of work to update the To field to point at these admin email addresses.

Awesome. I think at this point we should give them more time (since next week is a holiday in the us and not much notice), but perhaps sometime in mid december?