#727 Repo is too big
Opened a year ago by misc. Modified 9 months ago

The repository checkout is 1.5G on disk at the moment. This is a problem, since that mean some people with metered connexion can't easily download it (or people with small /home like me).

I would propose that we drop all pngs from the repository, and only keep the svg. Then, we could rebuild the png from the svg with automation without storing that in git.

This would likely make it easier later to move to openshift, since we wouldn't have consume a ton of ressources each time we rebuild the image.

However, this likely requires to rewrite history since the files are in git kinda forever for now, which mean breaking people existing repository among others.

For now, we are missing svgs on 121 badges:

[misc@windgrace fedora-badges]$ (for i in $(find pngs/ -name '*png' ); do [ -f ${i//png/svg} ] || echo $i; done; ) |wc -l

I would like to propose the following policy and steps:
- only accept badges with svg along png (right now)

  • make sure that missing png are converted from svg with automation need likely to wait for the move to openshift)

  • once we are sure that png are generated, convert slowly the existing png to svg, or get the original

  • fix the repo.

Another plan could be to create a new repo for the badges (svg only), migrate the existing ones there if possible, and continue to use the old one during the migration, but combining the 2 of them. This would permit a smooth transition until we are ready.

Converting old badges also provides a reason to upgrade the style, and give newcomers some easy tasks (albeit not that creative, I agree)

Metadata Update from @misc:
- Custom field artwork adjusted to None
- Custom field concept_review_passed adjusted to None

a year ago

I just verified, badgr do support both svg and png. So once we migrate, we could get a svg only repo.

Ok so I did not see that there is also a stls directory, who weight 1.1G at the moment. That's quite big. While I am not a expert in 3D (despites living a 3D world since a long time), I suspect the problem come from the fact that out badges are round, and STL do not support curved surfaces (cf https://en.wikipedia.org/wiki/STL_(file_format) )

So it approximate that with a ton of smaller surfaces, ergo the relatively big files (I suspect).

Png files are small enough to not be a problem. And they take less space than the svg files contrary to my assumption. However, once I compress a svg file, it get much smaller than the equivalent png, so I would still recommend using the svg:

changes-f23.png     41820
changes-f23.png.gz  41703
changes-f23.svg.gz  7122
changes-f23.svg     39414

There is some svg who do embed a png, so I guess that also explain the size difference:

For example, this one:

$ grep -l image/png *svg  |wc -l

Since the STL files are not used for anything in production in the current code base, I would recommend that we split the STLS in a separate git repo using some magic with git filter-branch (I can take care of that), then force push the cleaned repo and rewrite history.

Rewriting history mean that people would have to reclone the repository, might break links (since the commit ID would change), and have some side effects on the forge. I will have to dig a bit more into that before telling "here is the exact consequences" so people can decide.

In the mean time, i also still recommend that we do not duplicate files if possible, so either have a png or a svg (and possibily a svg), but not both, as it seems cleaner to me.

I just came here to give the same suggestion. We should split out the STLs out this repo. We don't even have the STLs these days, if I am right.

Also, we need to give badges the png urls as that is the one that get rendered.

Note ticket #680. The docs were separated out now because of the same issue.

I'm not sure this is an issue we should track in Badges Pagure. I think we should translate this to a software requirement for making SVGs the universal asset, and generating other things (STLs, PNGs, anything else) on the web app.

We could re-open this as an issue / user story on an appropriate Fedora Badges repo. Where should we take this?

In #680, I moved the whole commit history, you can safely full remove the docs folder from history if you have to impact everyone's git history anyway.

Login to comment on this ticket.