#9441 RFR: fedora-packages-static
Opened 6 months ago by mymindstorm. Modified 2 days ago

Description

fedora-packages-static is a simpler, statically generated version of the old Fedora Packages app.

How it works

packages-static is a collection of Python scripts that pulls repository metadata and compiles that into statically generated HTML files. It has a Docker container that runs packages-static every week and serves files via nginx. Because of this approach, packages search is handled by an external service such as Apache Solr or elasticsearch.

Example

Currently, I have a prototype container with searching handled by Solr and search results returned by a small uwsgi script.

Is it possible to have one pod secure a write mount to a volume and have multiple read-only mounts for load-balancing? I've looked at the openshift docs and it seems impossible.

Benefit to Fedora

  • Provides current and prospective users easy access to package information
  • Some users have noticed that old packages went down
  • Having Solr provides the framework for in-house docs.fp.o search, which is wanted by #6750

RFR Template

Phase I:

  • Software: fedora-packages-static and Apache Solr
  • Advantage for Fedora: See above
  • Sponsor:

Phase II:

Phase III:

  • SOP link:
  • Application Security Policy self-evaluation:
  • Audit request:
  • Audit timeline:

Phase IV:

  • Ansible playbooks:
  • Fully rebuilt from ansible:
  • Production goal:
  • Approved audit:

Metadata Update from @smooge:
- Issue priority set to: Waiting on Assignee (was: Needs Review)
- Issue tagged with: medium-gain, medium-trouble, ops

6 months ago

I am kind of curious on how hard/easy is it to deploy solr in a container/openshift, do you know?

I do not think that it will be hard given that there is an official container and a distributed cloud mode that is well documented.

The only complicated part is the admin UI which looks like it supports authentication with kerberos and openid. It is strongly recommended to put that (and the api) behind a non-public network.

Is it possible to have one pod secure a write mount to a volume and have multiple read-only mounts for load-balancing? I've looked at the openshift docs and it seems impossible.

I not sure if thats possible, RWX (ReadWriteMany) access mode is likely needed, which we can provide with NFS backed storage.

Looking at examples roles and playbook at:

and

Do you think you could open a PR against the ansible repo with what is needed for this project?

I not sure if thats possible, RWX (ReadWriteMany) access mode is likely needed, which we can provide with NFS backed storage.

Thanks, I mostly just want this to have one pod control writing and have the rest just serve from it instead of having different copies per-instance. If this is a bad idea/too complicated, I can just keep it how it is now.

Do you think you could open a PR against the ansible repo with what is needed for this project?

Yes, I'll make a PR as soon as I have the time.

Commit a4c1b1448f421c489db42035479626fa58b3b661 (Add fedora-packages-static and solr to openshift) is in fedora-infra/ansible since Mon Apr 5 18:25:10 2021 +0000. What is the state of the deployment? I couldn't find it on packages{{ env_suffix }}.fedoraproject.org with .stg or .test as suffix, but there are many moving parts here and I might have missed something (or it might not be online yet?).

@fmux it was merged by me, but I didn't run the playbook on it... we have been sorting out permissions so @mymindstorm can run the playbook and do both the initial deployment and any iterations after.

So, it should be up soon I hope!

It is not online yet. As Kevin has mentioned there were some access issues and I made a mistake in the solr playbook (which packages-static playbook won't run without). PR soon.

@mymindstorm is there anything still blocking on getting this one done?

There is some sort of networking issue with my openshift builds: https://pagure.io/fedora-infra/ansible/pull-request/574#comment-149131

Other than that, I should have a testing version up soon. I will send out an email to the infrastructure list when it is ready.

Login to comment on this ticket.

Metadata
Boards 1
ops Status: Backlog