#473 Git Forge Evaluation 2024
Opened 6 months ago by amoloney. Modified 2 months ago

Fedora depends on Pagure, a home-grown git forge that lacks important features and runs in an inconsistent and undermaintained capacity. From providing features a growing community needs to responding to security issues or incidents, reliance on Pagure represents a vulnerability for all concerned. Our goal is to eliminate this vulnerability by moving workloads to one or more maintained alternative tools, preferring free and open source software wherever possible.

In order to do this right, we need community involvement when gathering and reviewing the requirements that a forge must be able to provide, and when determining which options suit these needs the most. We must also acknowledge that not every requirement may be met by any one tool. To finally progress this work, we need to begin the conversation anew and agree on core and critical requirements the Fedora Project must have in order to continue to build and release on its current cadence, and to grow for the future. Therefore, I propose the following in order to begin this work in a transparent and open way:

  • Create a space on discussion.fedoraproject.org for talking about potential git forge options
  • Identify the constraints or requirements for the Fedora Project are as a whole and publish these
  • Decide, in consultation with the Fedora community, what git forges have the potential to replace the use of various pagure instances and create a timeline for technical investigations to be done looking at each option the Fedora Project is considering migrating to
  • Create a space to track work openly (maybe discourse suits this too)

I believe these steps are needed to guide both the conversation to work, and the work to delivery. There are nuances to be added and points to be fleshed out, but this is the general flow of work required at a very high level. The request to the Fedora Council is to advise on whether the above steps are the right way to proceed, and give suggestions and feedback on improving the initial plan so that the whole Fedora community can feel confident that these decisions are made in the proper Fedora way.

I've created a topic on Fedora Discussion for this ticket.

Please keep this ticket focused. Discuss there, and record votes and decisions here. Thanks!

Metadata Update from @jflory7:
- Issue priority set to: Coming Up (was: Needs Review)
- Issue tagged with: policies

6 months ago

For the record, other related discussion threads are under this tag:


Metadata Update from @amoloney:
- Issue assigned to amoloney

4 months ago

Discussed in our 2024-04-10 meeting.

The announcement following our 2024 Hackfest is now published on the Community Blog.

For next steps, the Council will review a list of high-level requirements we would like the investigation by Fedora Infrastructure / Red Hat Community Platform Engineering to use. @amoloney will share these soon. We are mindful of including stakeholders from specific key groups relying on our git forges, like QA, release engineering, Infrastructure, Design Team, and others.

To keep feedback in one place, we will continue using the #git-forge-future tag on Fedora Discussion primarily. As a reminder, please share feedback and comments in the Fedora Discussion tag. Thanks!

The set of requirements provided to CPE for the investigation are as follows:

  • Suitability for dist-git and src.fedoraproject.org replacement
  • Suitability for replacement of bugzilla for packaging issues and review process
  • Suitability for replacement of Pagure and bugzilla for release issues (change process, blocker bugs, etc)

  • Suitability for replacement of Pagure for SIG and Team ticket tracking (e.g. FESCo tracker)

  • Cost of hosting and maintenance (hardware + time and resources from CPE and wider infrastructure team)
  • Ease of migration from current Pagure,github and gitlab.com
  • Ease of extension and enhancement — can we improve things ourselves to add missing features / features that are really cool and useful like CI integration?
  • New features like Asciidoc support, Online editor and others to make things easier for the Fedora teams and their workflows.
  • Estimate Future risk for Fedora project and Infrastructure team
    • GitLab: open core squeeze and enshittification of services
    • Forgejo: community health and sustainability as a younger project

These represent a generalised, top-level view of the requirements the project is most likely to need holistically and further refined requirements are expected to fall in under some of these high level requirements per 'department' .

Please ack these requirements, or provide feedback and suggestions on iterating/adding to the requirements set by no later than Friday 12th April.

cc: @mattdm @t0xic0der @ffmancera @rwright @jflory7 @sumantrom @jonatoni @dcantrel @bt0dotninja @smeragoel

Will the packagers as a whole also be surveyed for their requirements? It seems that no part of this effort has been brought to the devel list—i.e., the canonical place where packaging is discussed—which is unfortunate given that we are the primary users of distgit.

Certainly, and thank you for adding where the best place to engage with this group is too. All of the discussions in recent months have been taking place on discussions.fpo, as I'm sure you're aware of the effort to move project discussions from email to here. From a quick read of the packager guidelines page, it mentions that the packaging email list is where the packager committee discuss guidelines, etc. Do you suggest using this email too when requesting input from the packaging group?

In general, development discussions don't happen on Discourse except for Change documents. So when soliciting feedback from the Fedora developer community, discussion.fp.o isn't the place to go.

Re the discussion piece: that's why I said effort and not mandate (or other word to convey this meaning) to move our discussions over to discourse :) This shift is something the project is moving towards slowly and afaik the change proposal process is/was the first in this effort, but there have been other discussions happening in that forum and the git forge evaluation conversations have been happening (quietly) there too, so there are conversations taking place there which is nice to see.

All that aside though, coz it's wildly off-topic and sorry for my part in it, the requirements shared above are not a finite list by any means and as said in the comment, they are to represent the needs of a project in a more generalised way for the council to review and ack on. They are based on a general understanding of what we know is needed for the project as a whole from previous discussions on this topic, but we fully expect the project to have way more requirements and quite specific ones too. The formal requirements gathering phase has not started yet, and there will be input required from all groups on what they would need from a git forge before the investigation begins. And the requirements gathering phase will include posting the call for requirements in as many venues as possible to maximise awareness, including mailing lists and discussions.fpo.

I am working with the CPE teams product owner, @humaton, who will be overseeing the investigation work and we have been fleshing out a message to share out to the community on what we need from the and how they can/need to/should get involved in this work.
We need to make sure all requirements are recorded and complied.
We need to re-share this formal set of requirements again to the community to validate they are captured correctly before the investigation starts.
And then we need to plan the investigation itself - have it be community-driven, establish timelines and reasonable outcomes, information sharing, etc.
Keep an eye out for an email and post from @humaton in the next few days (if not tomorrow, then early next week) asking for the more detailed requirements from the community, and please reply to the message with the needs your group has for git.

I hope this makes sense, and I didnt mean to write an essay but I was probably not clear enough at the start of where we truly are in our evaluation journey!

@amoloney, +1 to the list you've made above.

Add deez more points.

  • Native SSO extension support and group synchronization with Fedora Account System
  • Themeability? Optional - But let our Git Forge show our "Fedora" identity
  • Support for per-repository or per-group Wiki or documentation
  • Support for using custom runners or custom CI services like Zuul etc.
  • Heuristic evaluation for user experience - Assume technical and non-technical personas

Do you suggest using this email too when requesting input from the packaging group?

No, I suggest requesting input from packagers via the devel list. I confess that I am not particularly aware of how the CPE investigation process usually works, but I am surprised that this discussion started here 4 months ago and still hasn't been brought to the packagers—the primary users of distgit. I understand that this is a larger, project-wide issue, as it seems the effort encompasses replacing both pagure.io and distgit, but I would strongly suggest additionally bringing the requirements before the devel list and a FESCo vote like we do for other engineering decisions to ensure that side of the project is accounted for. distgit is a crucial packager tool and the issue is somewhat sensitive, so please make sure that we are included as well.

Suitability for dist-git and src.fedoraproject.org replacement

This would include:

  1. Allow setting the-new-hotness monitoring status
  2. Allow automatic orphan/taking of packages
  3. Allow setting Bugzilla assignee overrides
  4. Integrate with fedpkg

1-3 are implemented via a homegrown Pagure plugin (https://pagure.io/pagure-dist-git) that would need to be reimplemented in Gitlab or Forgejo (if they even allow this type of plugin) or as a separate webapp.

Metadata Update from @amoloney:
- Issue tagged with: In Progress

2 months ago

Metadata Update from @amoloney:
- Issue tagged with: Next Meeting

2 months ago

Log in to comment on this ticket.