#6497 [RFR] Census
Closed: Insufficient data 6 years ago by kevin. Opened 8 years ago by npmccallum.

https://lists.fedoraproject.org/archives/list/kernel@lists.fedoraproject.org/thread/IE4Z4UWEAWD66US5MH7UGMV54P3BI2BN/

We used to have a hardware database called Smolt. It was largely unmaintained and had various problems. But it provided valuable data. The biggest problem was that changes were very hard. It was fairly easy to collect data. But all access to the data required the development of full web pages. There was no data API. So interest in the project waned.

In response to this, I created a project called Census. Census is a data collection and query API service. It makes it easy to collect data (plugin directory with executables that return JSON) and store it (server-side JSON parsing plugins). But most importantly, once the data is in Census it can be queried by anyone. You can submit a query and the result is returned to you in JSON. There is also a minimal web UI which serves as a development environment for crafting queries.

The goal is that domain experts can write plugins for the data they wish to collect and write queries to analyze that data. To that end, Census itself intends to be a minimal service that others can build upon.

Census currently requires MongoDB. But we'd like to change to PostgreSQL. Our requirements are as follows:

  1. PostgreSQL master server
  2. 0..n PostgreSQL read-only mirror query servers (to scale)
  3. 1..n Web Front Ends (to scale); preferably OpenShift S2I

Load balancing would not be initially needed in the development environment. But as we scale it up, we may want to load balance across the web front ends and pgsql instances.

We do not require additional packages. Though we would require Python3 in the web front ends.


Phase I

  • Software: census
  • Advantage for Fedora: query API for Fedora installation statistics and hardware
  • Sponsor: @puiterwijk

Phase II

Phase III

  • SOP link:
  • Audit request:
  • Audit timeline:

Phase IV

  • Ansible playbooks:
  • Fully rebuilt from ansible:
  • Production goal:
  • Approved audit:

Metadata Update from @mizdebsk:
- Issue tagged with: request-for-resources

7 years ago

Metadata Update from @kevin:
- Issue priority set to: Waiting on Asignee

7 years ago

Sorry this languished.

Can we instead set you up with a communishift account and you can build your app/tool there and once it's all done and stable if you wish to hand it off to be supported by us we can look at it then?

https://fedoraproject.org/wiki/Infrastructure/Communishift?rd=Infrastructue/Communishift

Please feel free to re-open this or talk to us more about using communishift for this.

:construction:

Metadata Update from @kevin:
- Issue close_status updated to: Insufficient data
- Issue status updated to: Closed (was: Open)

6 years ago

Log in to comment on this ticket.

Metadata