#51 Add rewrite proposal details for Fedora Badges
Merged a year ago by zlopez. Opened a year ago by t0xic0der.
Unknown source badges-proposed  into  main

empty or binary file added
empty or binary file added
empty or binary file added
empty or binary file added
empty or binary file added
empty or binary file added
empty or binary file added
file modified
+1
@@ -49,6 +49,7 @@

      current_implementation

      exploring_the_development_environment

      expectations_and_wishes

+     proposal_rewrite

  

  Conclusions

  ----

@@ -0,0 +1,146 @@

+ .. _prop_rewrite_entities:

+ 

+ Entities involved

+ ====

+ 

+ Let us dive deep into the entities that would come together to make the Fedora 

+ Badges project functional. We will call them "entities" for the context of this

+ discussion. Also, the names used for the entities here are temporary in nature 

+ and are, for the most parts, only representative of the functions that these 

+ entities perform. 

+ 

+ Here is a diagram exhibiting the proposed architecture.

+ 

+ .. image:: ../_static/badges-proposed-architecture.png

+     :target: ../_images/badges-proposed-architecture.png

+ 

+ 

+ Internal entities

+ ----

+ 

+ **Accolades API** - Backend service

+    1. Succeeds the existing 

+       `Tahrir <https://github.com/fedora-infra/tahrir>`_'s backend API and/or 

+       is planned to be a reimplementation of the same.

+    2. This is planned to act as the backend service for the entire project and 

+       has direct interactions with its neighbouring internal entities (i.e. 

+       Collection, Database, Liberation, Messages Consumer, Accolades CLI).

+ 

+ **Accolades CLI** - Client application

+    1. Succeeds the existing 

+       `Tahrir API <https://github.com/fedora-infra/tahrir-api>`_ client and/or

+       is planned to be a reimplementation of the same.

+    2. This is planned to act as a client application and/or library, usable for

+       automating purposes like pushing of badges (and their related awarding 

+       conditions) to the Collection via the Accolades API, and other such high 

+       level interactions offered by the Accolades API.

+    3. This is planned to be a standalone entity with just one possible 

+       interaction with the neighbouring internal entity (i.e. Accolades API) 

+       and three possible interactions with the neighbouring external entity 

+       (i.e. community users, badge owners and service admins).

+ 

+ **Liberation** - Web interface

+    1. Succeeds the existing 

+       `Tahrir <https://github.com/fedora-infra/tahrir>`_'s frontend part and/or

+       is planned to be a reimplementation of the same.

+    2. This is planned to act as an interactive client application, accessible 

+       via modern web browsers, usable by community members and service admins 

+       for purposes like looking up the Fedora Badges leaderboard, checking 

+       badges awarded to self or someone else etc. and adding new badges (and 

+       their related awarding conditions) etc. respectively by interacting with 

+       the Accolades API.

+    3. This is planned to be a standalone entity with just one possible 

+       interaction with the neighbouring internal entity (i.e. Accolades API) 

+       and three possible interactions with the neighbouring external entity 

+       (i.e. community users, badge owners and service admins).

+ 

+ **Collection** - Artworks and rules

+    1. Succeeds the existing `Fedora Badges <https://pagure.io/fedora-badges>`_

+       repository and/or is planned to be a reimplementation of the same.

+    2. This is planned to be a repository of supporting assets (artworks, 

+       awarding conditions, consumer rules etc.) for the badges available for 

+       being awarded to the community members, which can be read from and 

+       updated by the interactions with the Accolades API entity and the 

+       Liberation entity as well as by admins having relevant accesses to the

+       repository.

+    3. This is planned to be a standalone entity with just one possible

+       interaction with the neighbouring internal entity (i.e. Accolades API) 

+       and one possible interaction with the neighbouring external entity (i.e. 

+       repository members).

+ 

+ **Database** - Achievements storage

+    1. Succeeds the existing Fedora Badges database storage for storing the 

+       badges awarded to community members, date of felicitation and other 

+       details and/or is planned to be a reimplementation of the same.

+    2. A huge chunk of the existing database is planned to be reimported into 

+       the newer schema and the columns specific to the Open Badges (or Badgr) 

+       compatibility will not be taken into consideration. This can be read 

+       into and updated by the interactions with the Accolades API only.

+    3. This is planned to be a standalone entity with just one possible 

+       interaction with the neighbouring internal entity (i.e. Accolades API)

+       and one possible interaction with the neighbouring external entity (i.e. 

+       service admins).

+ 

+ **Messages consumer** - Conditions listener

+    1. Succeeds the existing 

+       `fedbadges <https://github.com/fedora-infra/fedbadges>`_ fedmsg listener 

+       and/or is planned to be a reimplementation of the same.

+    2. This is planned to act as a messages consumer, listening into the Fedora 

+       Messaging bus matching the schemas that it is configured for and

+       comparing the messages against the awarding conditions available on the 

+       Collection, and on a successful match, making a request to the Accolades

+       API for awarding the said badge to a certain contributor who have met the

+       conditions specified in the related badge rule.

+    3. This is planned to be a standalone entity with two possible interactions

+       with the neighbouring internal entities (i.e. Accolades API and 

+       Collection) and no possible interaction with the neighbouring external 

+       entities.

+ 

+ External entities

+ ----

+ 

+ **Service admins** - Access Level IV

+    1. Consists of people belonging to the 

+       `Badges Administrators <https://accounts.fedoraproject.org/group/sysadmin-badges/>`_ 

+       group.

+    2. People have access to the deployment environments (staging, development, 

+       production etc.) and can control most of the aspects of all the internal 

+       entities.

+ 

+ **Badge owners** - Access Level III

+    1. Consists of people who either lead or represent certain subcommunity, SIG

+       or subproject within the Fedora Project community, who want to make new 

+       badges available and set rules for awarding.

+    2. People have an elevated access to the Liberation and Accolades CLI, which

+       enables them to access, create, change, award and delete badges that 

+       they "own" (not those that they "have been awarded").

+    3. If there does not exist a designated IPA group for marking out these 

+       entities, a new one can be created and they can be added to the same for 

+       easing access control processes.

+ 

+ **Repository members** - Access Level II

+    1. Consists of people who either design, create and/or facilitate new badges

+       and want to make them available and set rules for awarding.

+    2. These are the people who can fall under the "Badge owners" category too 

+       but that might not necessarily be the case always (for eg. badge 

+       designers who do not lead or represent another subcommunity, SIG or 

+       subproject but are tasked to create and upload badge assets).

+    3. There need not be a designated IPA group for marking out these entities

+       as their access is restricted to accessing, creating, changing and 

+       deleting badges assets available on the Collection entity, which is 

+       simply a version control system mediated repository.

+ 

+ **Community Users** - Access Level I

+    1. Consists of generally everyone from the Fedora Project community who have

+       a valid and activated, `IPA account <https://accounts.fedoraproject.org/>`_ 

+       and have signed the 

+       `Fedora Project Contributor Agreement <https://accounts.fedoraproject.org/user/t0xic0der/settings/agreements/>`_.

+    2. These are the people who are eligible to only receive badges for their 

+       recordable actions (i.e. those which either trigger a message on the 

+       Fedora Messaging bus or are noticed by a Badge Owner entity) within the 

+       Fedora Project community.

+ 

+ **NOTE** - These access levels are not exclusive in nature. A person who has an

+ Access Level IV, can (but not necessarily will) have those beneath it too and 

+ by default, every FPCA signed member of the Fedora Project community, has at 

+ least, Access Level I.

@@ -0,0 +1,241 @@

+ .. _prop_rewrite_interactions:

+ 

+ Interactions possible

+ ====

+ 

+ The interactions between the previously mentioned entities would dictate the

+ workflow of the project and the direction of the flow of information among

+ multiple entities involved in an interaction. They are denoted by the line 

+ connecting the entities and can be distinguished using the labels written on 

+ the top of the aforementioned joining lines.

+ 

+ Internal-only interactions

+ ----

+ 

+ Interactions involving only internal entities are termed, in this context, as

+ internal-only interactions. 

+ 

+ .. image:: ../_static/badges-proposed-architecture.png

+     :target: ../_images/badges-proposed-architecture.png

+ 

+ AA

+ ^^^^

+ 

+ 1. Defines the interaction between entities, Accolades CLI and Accolades API.

+ 2. Accolades CLI can help abstract reading, creating, changing and deleting of 

+    badge related assets or information with the help of the endpoints that the 

+    Accolades API provides.

+ 3. This interaction causes DA and CA interactions to take place, as the data

+    requested is read from and data modified is changed into the Database and

+    Collection entities. For eg. fetching information about the badges awarded 

+    to a certain user from the Database entity (DA) and automating adding new 

+    badges with the awarding conditions to the Collection entity (CA).

+ 4. This interaction affects LA and MA interactions as changes made to the 

+    Accolades API would affect how it reacts to the actions made by the 

+    Liberation and Message Consumer entities. For eg. new badges added into the

+    Collection entity might show up on the catalog of the Liberation entity (LA)

+    and the awarding condition added to the Collection entity would change the 

+    way the Message Consumer behaves in awarding badges (MA).

+ 

+ LA

+ ^^^^

+ 

+ 1. Defines the interaction between entities, Liberation and Accolades API.

+ 2. Liberation provides an interactive web interface for abstracting reading,

+    creating, changing, awarding and deleting of badge related assets or 

+    information with the help of the endpoints that the Accolades API provides.

+ 3. This interaction causes DA and CA interactions to take place, as the data

+    requested is read from and data modified is changed into the Database and

+    Collection entities. For eg. displaying the leaderboard of the top 100 

+    community members by fetching it from the Database entity (DA) and adding 

+    new badges with the awarding conditions to the Collection entity (CA).

+ 4. This interaction affects AA and MA interactions as changes made to the

+    Accolades API would affect how it reacts to the actions made by the 

+    Accolades CLI and Message Consumer entities. For eg. newly awarded badge 

+    from Liberation would show up on the Accolades CLI for the user querying 

+    about that information (AA) and the awarding condition added to the 

+    Collection entity would change the way the Message Consumer behaves in 

+    awarding badges (MA).

+ 

+ DA

+ ^^^^

+ 

+ 1. Defines the interaction between entities, Database and Accolades API.

+ 2. Database stores the information of the achievements made by the community 

+    members by their recordable actions within the Fedora Project community, 

+    along with the metadata of when the badges were awarded etc.

+ 3. This interaction cannot be directly triggered and can only happen as a cause

+    of LA, AA and MA interactions. As a result, this interaction does not become

+    a causative agent for any other interaction. For eg. awarding a user with a 

+    badge from the Liberation entity would lead to changes made in the Database 

+    entity (LA), querying about the badges achieved by a user in the Accolades 

+    CLI entity requires reading information from the Database entity (AA) and 

+    conditioned awards made from the Message Consumer entity would be reflected 

+    on the Database entity (MA). 

+ 4. This interaction affects only those interactions, that cause it to happen in

+    the first place. This can be explained as the consequences of the 

+    aforementioned example interactions - for eg. Liberation entity would be 

+    able to see the badge under the username that it was awarded to recently 

+    (LA) and Accolades CLI entity would be able to retrieve the information it

+    requested for (AA) but the MA interaction would stay unaffected.

+ 

+ CA

+ ^^^^

+ 

+ 1. Defines the interaction between entities, Collection and Accolades API.

+ 2. Collection entity stores the badge assets (artworks, badge rules, awarding 

+    conditions etc.) that the Accolades API can read content from, add new 

+    badges, change/update and delete/remove existing ones. 

+ 3. This interaction cannot be directly triggered and can only happen as a

+    cause of internal interactions like LA, AA, DA and MA as well as external

+    interactions like changes made into the Collection entity by external 

+    entities like Repository Members etc. For eg. adding new badges for 

+    distribution using the Liberation entity (LA) and doing so using the 

+    Accolades CLI (AA) would lead to changes in the Collection entity, changes

+    in the database due to the awarding of badges would require reading 

+    information from the Collection entity (DA) and rules on how the Message

+    Consumer entity would award badges would be described by the conditions

+    mentioned in the Collection entity (MA).

+ 4. This interaction affects only those interactions, that cause it to happen

+    in the first place. This can be explained as the consequences of the 

+    aforementioned examples - for eg. addition of new badge for distribution 

+    would cause it to be accessible on Liberation entity (LA) and on Accolades

+    API (AA), Message Consumer entity would now have new rules as a result to

+    changes in the Collection entity (MA) but the DA interaction would stay 

+    unaffected.

+ 

+ MA

+ ^^^^

+ 

+ 1. Defines the interaction between entities, Messages Consumer and Accolades 

+    API.

+ 2. Message Consumer listens in to the Fedora Messaging bus and compares the

+    messages against the awarding conditions, and when the said condition is 

+    satisfied, it makes a request to the Accolades API to award an external 

+    entity, a certain Community User with the relevant badge.

+ 3. This interaction cannot be directly triggered and can only happen as a

+    cause of internal interactions like CM and CA. For eg. any new additions 

+    made to the Collection entity would change the way the Message Consumer 

+    entity checks for conditions for badges (CA) and the Message Consumer would 

+    read the content regarding the badges from the Collection entity (CM).

+ 4. This interaction affects DA, LA and AA interactions as changes made to the

+    Accolades API entity by the Messages Consumer entity would affect how it 

+    reacts to the actions made into the Database entity, by the Liberation 

+    entity and Message Consumer entities. For eg. badges awarded automatically

+    by the Messsage Consumer entity's interaction would cause a change to be 

+    made on the Database entity (DA) and lead them to be visible on the 

+    Liberation entity (LA) and accessible on the Accolades CLI entity (AA).

+ 

+ CM

+ ^^^^

+ 

+ 1. Defines the interaction between entities, Collection and Message Consumer.

+ 2. In order for the Message Consumer to obtain badge assets (artworks, rules, 

+    awarding conditions etc.), it has to interact with the Collection entity.

+ 3. This interaction cannot be directly triggered and can only happen as a cause

+    of internal interactions like CA and external interactions like Repository 

+    Members making changes to the Collection entity. For eg. internal changes 

+    made to the Collection entity and external changes made to the Collection 

+    entity (for eg. by the Repository Members) would change the way the Messages

+    Consumer entity would check for awarding conditions for the badges (CA).

+ 4. The interaction does not directly affect any other internal interaction but

+    can indirectly affect interactions like MA. For eg. changes in the 

+    Collection entity can change the frequency and cases when the Messages 

+    Consumer would interact with the Accolades API (MA).

+ 

+ 

+ External-Internal interactions

+ ----

+ 

+ Interactions involving both external and internal entities are termed, in this

+ context, as external-internal interactions.

+ 

+ Accolades API interacting with external entities

+ ^^^^

+ 

+ .. image:: ../_static/badges-proposed-api-ext-interactions.png

+     :target: ../_images/badges-proposed-api-ext-interactions.png

+ 

+ 1. Service Admins [Access Level IV] would have a direct control on the 

+    deployment environments (staging, development, production etc.).

+ 

+ Accolades CLI interacting with external entities

+ ^^^^

+ 

+ .. image:: ../_static/badges-proposed-cli-ext-interactions.png

+     :target: ../_images/badges-proposed-cli-ext-interactions.png

+ 

+ 1. Service Admins [Access Level IV] would have a direct control on the 

+    deployment environments (staging, development, production etc.).

+ 2. Badge Owners [Access Level III] would have an elevated access to the 

+    Accolades API entity using the Accolades CLI entity where they can 

+    automate addition of new badges for distribution within the community, 

+    create, access, modify and remove the badges that they "own" (not the ones

+    that they "have been awarded").

+ 3. Community Users [Access Level I] would have a limited access to the

+    Accolades API entity using the Accolades CLI entity where they can view 

+    the badges received by them (and by others), access the catalog of 

+    available badges, view leaderboards and other such similar tasks.

+ 

+ Liberation interacting with external entities

+ ^^^^

+ 

+ .. image:: ../_static/badges-proposed-web-ext-interactions.png

+     :target: ../_images/badges-proposed-web-ext-interactions.png

+ 

+ 1. Service Admins [Access Level IV] would have a direct control on the 

+    deployment environments (staging, development, production etc.).

+ 2. Badge Owners [Access Level III] would have an elevated access to the

+    Accolades API entity using the Liberation entity where they can manually 

+    add new badges for distribution within the community, create, access, 

+    modify and remove the badges that they "own" (not the ones that they "have

+    been awarded").

+ 3. Community Users [Access Level I] would have a limited access to the

+    Accolades API entity using the Liberation entity where they can view the

+    badges received by them (and by others), access the catalog of available

+    badges, view leaderboards and other such similar tasks.

+ 

+ Collection interacting with external entities

+ ^^^^

+ 

+ .. image:: ../_static/badges-proposed-col-ext-interactions.png

+     :target: ../_images/badges-proposed-col-ext-interactions.png

+ 

+ 1. Repository Members [Access Level II] would have complete access to the 

+    Collection entity where they can access, create, modify and remove badge

+    assets (i.e. artworks, badge rules, awarding conditions etc.). As the 

+    access is unrestricted, they have access to the assets created by someone

+    else too, which belong to the entity.

+ 

+ Database interacting with external entities

+ ^^^^

+ 

+ .. image:: ../_static/badges-proposed-dtb-ext-interactions.png

+     :target: ../_images/badges-proposed-dtb-ext-interactions.png

+ 

+ 1. Service Admins [Access Level IV] would have a direct control on the 

+    deployment environments (staging, development, production etc.).

+ 

+ Messages Consumer interacting with external entities

+ ^^^^

+ 

+ .. image:: ../_static/badges-proposed-msg-ext-interactions.png

+     :target: ../_images/badges-proposed-msg-ext-interactions.png

+ 

+ 1. Service Admins [Access Level IV] would have a direct control on the 

+    deployment environments (staging, development, production etc.).

+ 

+ External-only interactions

+ ----

+ 

+ Interactions involving only external entities are termed, in this context, as

+ external-only interactions.

+ 

+ 1. Badge Owners [Access Level III] might need to interact with Repository 

+    Members [Access Level II] for assistance with changing badge assets in

+    the Collection entity.

+ 

+ 2. Community User [Access Level I] might need to interact with the Badge 

+    Owners [Access Level III] to obtain badges for the action that they have 

+    done within the community.

+ 

@@ -0,0 +1,52 @@

+ .. _prop_rewrite_technologies:

+ 

+ Technologies suggested

+ ====

+ 

+ Database

+ ----

+ 

+ *Suggested using:*

+     | Database: Postgres

+ 

+ Accolades API

+ ----

+ 

+ *Suggested using:*

+     | Web framework: `FastAPI <https://fastapi.tiangolo.com/>`_ or `Flask <https://flask.palletsprojects.com/>`_

+     | CLI compositor: `Click <https://click.palletsprojects.com/>`_

+     | Database abstraction: `Psycopg2 <https://www.psycopg.org/docs/>`_

+     | ORM and SQL library: `SQLAlchemy <https://sqlalchemy.org/>`_

+     | Configuration parsing: `PyYAML <https://pyyaml.org/wiki/PyYAML>`_ or `Tomllib <https://docs.python.org/3/library/tomllib.html>`_

+     | Caching and indexing: `Redis <https://redis.io/>`_

+ 

+ Accolades CLI

+ ----

+ 

+ *Suggested using:*

+     | CLI compositor: `Click <https://click.palletsprojects.com/>`_

+     | HTTP library: `Requests <https://requests.readthedocs.io/>`_

+     | Configuration parsing: `PyYAML <https://pyyaml.org/wiki/PyYAML>`_ or `Tomllib <https://docs.python.org/3/library/tomllib.html>`_

+ 

+ Liberation

+ ----

+ 

+ *Suggested using:*

+     | Frontend library: `VueJS <https://vuejs.org/>`_

+     | Frontend framework: `Vuetify <https://vuetifyjs.com/>`_

+     | Server: `Vue server <https://vuejs.org/>`_

+ 

+ Collection

+ ----

+ 

+ *Suggested using:*

+     | Version controlled forge: `GitLab <https://gitlab.com/>`_

+ 

+ Messages Consumer

+ -----

+ 

+ *Suggested using:*

+     | Listener: `Fedora Messaging <https://fedora-messaging.readthedocs.io/en/stable/>`_

+     | CLI compositor: `Click <https://click.palletsprojects.com/>`_

+     | HTTP library: `Requests <https://requests.readthedocs.io/>`_

+     | Configuration parsing: `PyYAML <https://pyyaml.org/wiki/PyYAML>`_ or `Tomllib <https://docs.python.org/3/library/tomllib.html>`_

@@ -0,0 +1,64 @@

+ .. _proposal_rewrite:

+ 

+ Rewrite Fedora Badges From The Ground Up

+ ====

+ 

+ | *Something which is state-of-the-art right now,* 

+ | *will eventually and inevitably become a tech debt tomorrow.*

+ | -- **Pierre-Yves Chibbon**, Sometime in 2021

+ 

+ We are proposing a rewrite for the project from the ground up.

+ 

+ Justification

+ ----

+ 1. The web interface and the API service for Fedora Badges is written in 

+    Python 2 which has gone `EOL <https://www.python.org/doc/sunset-python-2/>`_ 

+    on January 1st, 2020. We could have proposed a 1:1 rewrite of the same

+    functionality in Python 3 but reimplementing the features in ways that are

+    more relevant now, makes more sense.

+ 2. The `messages consumer <https://github.com/fedora-infra/fedbadges>`_ 

+    is bound to the `fedmsg <https://github.com/fedora-infra/fedmsg>`_ and needs

+    to be reimplemented to be compatible with 

+    `Fedora Messaging <https://github.com/fedora-infra/fedora-messaging>`_.

+    As these two projects are inherently different from each other in their 

+    function, not a lot of code can be salvaged from the previous implementation

+    of the messsages consumer, even though the strategy/approach can be.

+ 3. The web interface and the API service for Fedora Badges is written using 

+    the `Pyramid web framework <https://trypyramid.com/>`_ and makes use of 

+    `Mako templating system <https://www.makotemplates.org/>`_ which are not as

+    `popular <https://gustavwillig.medium.com/python-web-development-in-2021-which-web-frameworks-are-the-most-popular-by-github-stars-e07b1d7ef6f7>`_ 

+    as their alternatives, `Flask <https://flask.palletsprojects.com/>`_ or 

+    `FastAPI <https://fastapi.tiangolo.com/>`_ and 

+    `Jinja <https://jinja.palletsprojects.com/>`_ respectively are.

+       * Maintaining the codebase in its current state can be a challenge in 

+         the absence of comprehensive documentation, popular community support 

+         and frequent question/answer activity on forums regarding the topics 

+         related to the Pyramid web framework and Mako templating system. 

+       * Finding interested community members (or getting people interested to

+         contribute to the project) can be difficult if a set of comparatively 

+         obscure or unpopular technologies are used, which might deincentivize 

+         participation here due to a relatively steep learning curve.

+ 4. The frontend of the web interface is written in a form of a meta-application

+    (i.e. the frontend is supposed to behave like an application and yet is 

+    implemented like a website) using (now, dated) templating systems, plain 

+    HTML, CSS3 frameworks and Javascript libraries. These could use a rewrite 

+    for improved application-like functionality and interactivity.

+ 5. The current codebase has residual and/or partial implementation to 

+    accommodate Open Badges (or Badgr) standards for storing artworks data, 

+    awarding conditions, awardees etc. which have potentially never been 

+    completed and/or deployed. Cleaning up the said codebase or completing it

+    before maintaining the project in its current state would require lots of 

+    effort.

+ 

+ Implementation

+ ----

+ 

+ Here is how we are planning to have the project redeveloped.

+ 

+ .. toctree::

+     :maxdepth: 1

+     

+     prop_rewrite_entities

+     prop_rewrite_interactions

+     prop_rewrite_technologies

+ 

Signed-off-by: Akashdeep Dhar akashdeep.dhar@gmail.com

This is a work in progress. Please do not merge yet.

EDIT - This PR is now complete and can be reviewed, as of 1138IST, 14 Dec 2022.

1 new commit added

  • Finish making rewrite proposal for Fedora Badges
a year ago

Aight, this PR is no longer a WIP.

Please feel free to review it.

rebased onto e60223481cbcdfc34dc5e32fd9e8b9f01126e96e

a year ago

I would like the idea to do these interactions through web interface as well.

We can use the same approach as the new FMN, just consume messages that we have message schema for.

Do we really need to use a repository, isn't storage and API to interact with it enough? It will make interaction easier, no need to use git for every change.

rebased onto a1fdc5772eb38f0212c01d7551c1ea126fc894b9

a year ago

rebased onto 0e41bb0

a year ago

Pull-Request has been merged by zlopez

a year ago