fedora-infrastructure

#7467 blender dist-git contains huge binary files
Closed: Duplicate by pingou. Opened by fcami.

Closed: Duplicate
Reopen Issue
  • Describe what you need us to do:

The following large binary objects are contained in blender dist-git:

size packed objectId path
134473 134037 8a7e6d7dc970f5b1fdd66e9a9e57e814db4de9a2 results_blender/2.77a/1.fc26/blender-debuginfo-2.77a-1.fc26.x86_64.rpm
134464 134028 7966c177500299784bcaa963381f14b74eddc108 results_blender/2.77a/1.fc23/blender-debuginfo-2.77a-1.fc24.x86_64.rpm
131699 131252 12907ae1397483d284498820e3fac62ad7e17c07 results_blender/2.78/1.fc26/blender-debuginfo-2.78-1.fc26.x86_64.rpm
43447 43303 a4a9f0f6a24e622799cfefccf1804cae48cdfcae blender-2.78.tar.gz
43343 43335 e17f27a4415c31a747f76e10a68e592bf5d25b47 blender-2.78-1.el7.src.rpm
40838 40829 30bb773c1b4242fdb48b33788e53374a4a900e14 blender-2.77a-1.el7.src.rpm
31751 31610 c89795526e03033ff0255495f94ced42cea86271 results_blender/2.78/1.fc26/blender-2.78-1.fc26.x86_64.rpm
29923 29794 12a5eee6eb79f3ec8c257cc4b02a16e1c0774966 results_blender/2.77a/1.fc26/blender-2.77a-1.fc26.x86_64.rpm
29923 29795 2340a78e377e89c3a3b751a997ce0a563a9714ff results_blender/2.77a/1.fc23/blender-2.77a-1.fc24.x86_64.rpm
20108 496 678061a4cff46f22b389621f7b33a376a654c8ba results_blender/2.77a/1.fc26/build.log

It looks like there are no commits referencing those anymore.
Could you please clean the repository up or ask the maintainers to do so?
I'm not sure who should do it.

  • When do you need this? (YYYY/MM/DD)

No actual urgency.

  • If we cannot complete your request, what is the impact?

It makes cloning this repo far longer than it should, wastes bandwidth and storage space on both sides.

I sometimes get the following error:
fatal: the remote end hung up unexpectedly1 MiB | 2.73 MiB/s
fatal: early EOF
fatal: index-pack failed
while cloning.

For reference the dist-git is 634MiB and could be brought down to less than 1MiB with:

git remote rm origin
rm -rf .git/refs/original/ .git/refs/remotes/ .git/*_HEAD .git/logs/
git for-each-ref --format="%(refname)" refs/original/ | xargs -n1 --no-run-if-empty git update-ref -d
git -c gc.reflogExpire=0 -c gc.reflogExpireUnreachable=0 -c gc.rerereresolved=0 -c gc.rerereunresolved=0 -c gc.pruneExpire=now gc

It looks like this preserves history but I'd rather someone doublechecked.

Confirmed that /srv/git/rpms/blender.git/ takes 635M of disk space. Unfortunately by policy we are not allowed to remove any objects from dist-git. There is an open FESCo ticket about relaxing this policy in some way - you can escalate this issue to FESCo. If big size of this repository poses problems to you then as a workaround you may consider requesting a new repository (with a new name) and pushing cleaned-up code there.

Metadata Update from @mizdebsk:
- Issue assigned to mizdebsk
- Issue close_status updated to: Will Not/Can Not fix
- Issue priority set to: None (was: Needs Review)
- Issue status updated to: Closed (was: Open)
- Issue tagged with: src.fp.o

Just to update anyone else who (like me) might end up here after encountering the same issue, a related releng#7265 ticket is still open, and contains updates on the FESCo-blessed solution, though it remains unimplemented in the blender repo case.

@zbyszek wrote:

FESCo discussed this issue on 2018-03-02 and gave the green light:

AGREED: Move branches with huge commits to refs/archive/ and create new sanitized branches (+6, 0, 0)

The idea is to move the bad heads to refs/archive/, so that they will remain in the repo, will not be garbage collected by git, but will not be cloned by git by default. I.e. the repo will still be large, but only on pagure, and cloning will be fast.

Metadata Update from @fcami:
- Issue status updated to: Open (was: Closed)

Reopening as per the latest comment. The fix should be implemented as working with the blender repository is cumbersome at best.

I'm going to close this again in favour of the releng ticket which has more traction than here.

Metadata Update from @pingou:
- Issue close_status updated to: Duplicate
- Issue status updated to: Closed (was: Open)
Metadata

src.fp.o

None
None
None
Static archive. pagure.io was sunset in 2026; this is a read-only snapshot.
Red Hat Logo

Fedora is sponsored by Red Hat. Learn more about the relationship between Red Hat and Fedora »

© Red Hat, Inc. and others.