#8390 Create custom git refs to archive deprecated branches in dist-git
Opened 4 months ago by bookwar. Modified 3 months ago

The issue of dist-git having "too many branches" regularly appears on mailing lists and in offline conversations.

I'd like to propose a solution to this problem which uses custom git references.

In Git branches and tags are just two special cases of a git ref, stored under .git/refs/heads/
and .git/refs/tags/ respectively.

These refs are treated in a special way by the high level tools, but internally these are the same thing. Because tags and branches have special meaning they are made visible in CLI and GUI.

Let's create additional ref namespace called archive (.git/refs/archive) and move deprecated branch references to it.

  • Since ref will be preserved, git garbage collection will not remove the git commits which they point to.
  • Since it is a custom namespace, these branches will not be exposed to the user by default.
  • Since we change namespace, not the branch name, it will be easy to find the old branch after archiving.

User will still be able to fetch those references, by requesting them through a fully quailified path: Instead of running git checkout old-branch one would need to use git checkout refs/archive/old-branch.


on the technical side this is not a big deal, we could add a pagure-admin archive-branch project branchor an api endpoint or whatever depending on how are we going to apply it, but shouldn't we change the policy first?

I agree, should be doable, both through Pagure API and directly via git.

1) I want to get the confirmation that there are no technical obstacles to this. Maybe some infrastructure dependencies which I am not aware of.

2) On the policy side, for me it doesn't look like we are changing it. We don't remove old branches, we just reorganize them. So it is not a policy change, but we would need to update some docs to explain how it can be used.

What happens if one then creates a new branch with the same name, and then wants to archive the new branch, too?

What happens if one then creates a new branch with the same name, and then wants to archive the new branch, too?

git pre-receive hook denying such branches should help

I guess you'll want a command to unarchive an archived branch then, so that the name is not lost forever if you accidentally archive a branch.

I guess you'll want a command to unarchive an archived branch then, so that the name is not lost forever if you accidentally archive a branch.

Possibly. But this should be done just on request. The question is if you want to recycle branch names for different purposes then originally.

What happens if one then creates a new branch with the same name, and then wants to archive the new branch, too?

Thanks, I haven't thought about it.

One option is a githook as Vit said. The other - to change the proposal slightly and add a timestamp or counter to the archived branch name

git checkout refs/archive/old-branch/20190815

This will mess up with the discoverability of an archived branch a bit, but it provides a possibility to reuse branch names for some other purpose, and archive them several times.

Although, since our branches are not totally random, and usually are restricted by other policies so they don't overlap, the hook seems to be a much cleaner option.

What branches were you picturing as being archived? Only requested by maintainers? All EOL branches as they EOL?

Do you know the git operations to make this archive namespace and move branch refs to it off hand? (ie, we will need some kind of script to do this at EOL or on request or whatever).

Do you know the git operations to make this archive namespace and move branch refs to it off hand? (ie, we will need some kind of script to do this at EOL or on request or whatever).

I was thinking on creating the refs/archive pointing to the same obj that refs/heads points and removing the refs/heads entry after with an pagure-admin subcommand or a pagure-dist-git script

What about visibility? Should we show them someway on src.fp.o ?

This was the trigger for this ticket:

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/T66D6KPT5KOZKDRBMRVKCMECUDW2PBAA/

So obviously, I would like to see removed all the modularity branches, which are not used (but TBH, I was more thinking about something similar to dead.package we have for retired packages). I would not mind if the regular Fnn branches after their EOL were removed as well.

And of course I would like to be able to get rid of any other branch I am able to create, but not allowed to delete.

May be there could also be more namespaces, depending what was the reason for "removal"

Metadata Update from @mizdebsk:
- Issue tagged with: src.fp.o

4 months ago

So, I am still not sure of the scope here. Is it:

a) when branches are retired or eoled, we script something to move those branches to archive/ when they are eol or archived. Users can request other branches be archived via a releng ticket.

or

b) Users are allowed to move any branches to archive they like.

or something else?

If it's a, we just need to scope the work on making the scripts and adjusting eol and retire processes.

If it's b, we may need to see if our hook setup would allow that (ie, only moving a ref from heads to archive, not allowing delete?

Metadata Update from @kevin:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

3 months ago

So, I am still not sure of the scope here.

a) is the scope. I don't think users can move branches or should even be able to move them.

However it needs to be supported by some git hooks to prevent pushes into archived branches, pushing branches with colliding names, etc.

So, I am still not sure of the scope here.

a) is the scope. I don't think users can move branches or should even be able to move them.

:thumbsup:

However it needs to be supported by some git hooks to prevent pushes into archived branches, pushing branches with colliding names, etc.

push to retired branches is denied already[0] but... is it possible to push refs to refs/archive with git? No idea really. If its possible, we'll need small tweaks to deny pushing to refs/archive/* and we should decide wether we make the basename(refname) unique denying reusing of names on refs/archive or if we allow to reuse them and archive with refs/archive/data-name or something similar to avoid collisions.

0: https://pagure.io/pagure-dist-git/blob/master/f/dist_git_auth.py#_82-117

Login to comment on this ticket.

Metadata