#8 Datanomer/Datagrepper V.2
Opened 4 months ago by amoloney. Modified 19 hours ago

New initiative: Datanomer/Datagrepper V.2

What is this initiative about?

The datanommer and datagrepper stacks are currently relying on fedmsg which we want to deprecate.
These two applications need to be ported off fedmsg to fedora-messaging. As these applications are 'old-timers' in the fedora infrastructure, we would also like to look at optimizing the database or potentially redesigning it to better suit the current infrastructure needs.
For a phase one, we would like to want to upgrade these applications from python2 and move them to fedora-messaging.
For a phase two, we would like to focus on a DB overhaul - this can be opened as a separate initiative when required.

Why this initiative?

A lot of Fedora services are tied into these two applications. They are overworked and underperform as they require updates to make them more efficient. This causes delays when working with other applications and makes contributing to fedora infra overall harder.

Definition of success

Acceptance Criteria

  • Both applications are updated with fedora-messaging schemas
  • Applications are updated to the latest technology that they currently run on, eg. py2 -> py3

Nice to Haves

  • Investigate requirements for referencing topics in the DB & compile a list of headers for phase 2 work

Area/community impacted

Fedora infrastructure

Dependencies

A lot of Fedora apps - the full scope of affected services is not known

Possible Skill Sets Needed

  • Infrastructure access
  • Python experience
  • Messaging schemas familiarity
  • Developers to update & review codebases

Pre-Requisites

None known at this time

Deadline

No deadline at this time

Is this initiative under a time-constraint? Should it start or end before a certain date?


ARC Team have investigated this initiative request and developed a proof of concept for progressing this work.
This will be included in Q2 initiative selection.
ARC documentation can be read here https://fedora-arc.readthedocs.io/en/latest/datanommer_datagrepper/index.html

Metadata Update from @amoloney:
- Issue untagged with: To be scoped
- Issue tagged with: Accepted, Ready for QP

2 months ago

Issue tagged with: 2021Q2

a month ago

Metadata Update from @amoloney:
- Issue untagged with: 2021Q2

20 hours ago

Issue tagged with: To be scoped

19 hours ago

Metadata Update from @amoloney:
- Issue untagged with: To be scoped

19 hours ago

Metadata Update from @amoloney:
- Issue tagged with: POC Complete

19 hours ago

Metadata Update from @amoloney:
- Issue set to the milestone: Fedora Messaging Overhaul

19 hours ago

Login to comment on this ticket.

Metadata
Boards 1
2021 Backlog Status: Ready for QP