Issue #9266: prune EOL content from OSTree repos - releng

releng

#9266 prune EOL content from OSTree repos

Closed: Fixed 8 months ago by dustymabe. Opened 4 years ago by dustymabe.

Let's prune our ostree repos so that we can migrate less data to our new location for ostree content (see https://pagure.io/releng/issue/8811).

take a backup of /mnt/koji/compose/ostree/repo and /mnt/koji/ostree/repo
run the following script to make sure there isn't already content that is loose not reachable from any ref:

#!/usr/bin/bash
set -x
cd /mnt/koji/ostree/repo
ostree --repo=./ prune --refs-only --no-prune
cd /mnt/koji/compose/ostree/repo
ostree --repo=./ prune --refs-only --no-prune

run the following script to delete refs:

#!/usr/bin/bash
set -x
cd /mnt/koji/ostree/repo
for ref in fedora/{27,28}/{x86_64,aarch64,ppc64le}/{,updates/,testing/}{atomic-host,workstation,silverblue}; do
    ostree --repo=./ refs --delete="${ref}"
done
cd /mnt/koji/compose/ostree/repo
for ref in fedora/{27,28}/{x86_64,aarch64,ppc64le}/{,updates/,testing/}{atomic-host,workstation,silverblue}; do
    ostree --repo=./ refs --delete="${ref}"
done

run the following script to prune (delete content)

#!/usr/bin/bash
set -x
cd /mnt/koji/ostree/repo
ostree --repo=./ prune --refs-only
cd /mnt/koji/compose/ostree/repo
ostree --repo=./ prune --refs-only

dustymabe commented 4 years ago

I'll work with @mohanboddu @kevin and @jlebon to make sure the steps above are valid and schedule some time to perform this request.

dustymabe commented 4 years ago

Right now we're using just under 2T. I'm hoping the above pruning will cut down on our space usage by quite a bit.

[dustymabe@composer repo][STG]$ pwd
/mnt/fedora_koji_prod/koji/compose/ostree
[dustymabe@composer ostree][STG]$ du -sh repo/
1017G   repo/
[dustymabe@composer ostree][STG]$ cd /mnt/fedora_koji_prod/koji/ostree
[dustymabe@composer ostree][STG]$ du -sh repo/
  810G  repo/

jlebon commented 4 years ago

Looks sane to me. Maybe perform the operations on the compose repo first instead of prod? Also, I think we can delete the workstation/silverblue f29 refs too, right?

dustymabe commented 4 years ago

Looks sane to me. Maybe perform the operations on the compose repo first instead of prod? Also, I think we can delete the workstation/silverblue f29 refs too, right?

yes, but to be conservative I'd like to leave them and put a policy in place where we clean up EOL-1 on EOL day.

jlebon commented 4 years ago

Sure, WFM. One thing we could do too is keep the tip only with --retain-branch-depth. That way users can still upgrade to the latest tree we released before jumping off and rebasing.

Edited 4 years ago by jlebon