PR#203: SOPs for deploying the fas2discourse operator on Fedora Infra - infra-docs-fpo

infra-docs-fpo

#203 SOPs for deploying the fas2discourse operator on Fedora Infra

Merged 2 years ago by dkirwan. Opened 2 years ago by dkirwan.

dkirwan/infra-docs-fpo master into master

fas2discourse: SOPs

David Kirwan • 2 years ago

244bf31

modules/ocp4/pages/sop_fas2discourse_operator.adoc

file added

+13

		`@@ -0,0 +1,13 @@`
		`+ = fas2discourse Operator`
		`+ The following SOPs are related to the administration of the fas2discourse operator.`
		`+`
		`+ == Resources`
		`+ - https://pagure.io/cpe/fas2discourse/[Code]`
		`+ - https://quay.io/repository/fedora/fas2discourse-operator[Image]`
		`+ - https://pagure.io/fedora-infrastructure/issue/10952[Initial ticket]`
		`+ - xref:sop_fas2discourse_operator_installation.adoc[Install the fas2discourse operator]`
		`+ - xref:sop_fas2idscourse_operator_testing.adoc[Testing the fas2discourse operator]`
		`+ - xref:sop_fas2discourse_operator_build.adoc[Building/releasing the fas2discourse operator]`
		`+ - xref:sop_fas2discourse_operator_interacting.adoc[Interacting with the the fas2discourse operator]`
		`+ - xref:sop_fas2discourse_operator_debugging.adoc[Debugging issues with the the fas2discourse operator]`
		`+`

modules/ocp4/pages/sop_fas2discourse_operator_build.adoc

file added

+23

		`@@ -0,0 +1,23 @@`
		`+ = Build/release the fas2discourse Operator`
		`+`
		`+ == Resources`
		`+ - [1] Code: https://pagure.io/cpe/fas2discourse`
		`+ - [2] Quay: https://quay.io/repository/fedora/fas2discourse-operator`
		`+`
		`+ == Installation`
		+ To build the operator and tag it with version `0.0.63` as an example:
		`+`
		`+ - First ensure that you are logged into quay.io and have access to the repository at [2].`
		`+ - Check out the code at [1].`
		+ - Make the change to the version of the operator being built by editing the `Makefile` and change the variable at the top `VERSION ?= 0.0.63`
		`+`
		`+ ----`
		`+ make build`
		`+ ----`
		`+`
		`+ Push the operator to the quay.io catalog then with the following:`
		`+`
		`+ ----`
		`+ podman push quay.io/repository/fedora/fas2discourse-operator:0.0.63`
		`+ ----`
		`+`

modules/ocp4/pages/sop_fas2discourse_operator_debugging.adoc

file added

+37

		`@@ -0,0 +1,37 @@`
		`+ = Debugging issues with the fas2discourse Operator`
		`+`
		`+ == Resources`
		`+ - [1] Code: https://pagure.io/cpe/fas2discourse/`
		`+ - [2] Playbook: https://pagure.io/fedora-infra/ansible/blob/main/f/playbooks/manual/fas2discourse.yml`
		`+ - [3] Role: https://pagure.io/fedora-infra/ansible/blob/main/f/roles/fas2discourse`
		`+`
		`+ == Workload`
		+ The operator runs in the namespace: `fas2discourse-operator` on both the staging and production openshift clusters.
		`+`
		`+ There is a single pod running. First port of call should be to examine the logs of this pod.`
		`+`
		+ By default, the verbocity of logs are set low. To increase them to debug level add the following annotation to the `Fas2DiscourseConfig` object in the `fas2discourse-operator` namespace:
		`+`
		`+ ----`
		`+ apiVersion: fas2discourse.apps.fedoraproject.org/v1alpha1`
		`+ kind: Fas2discourseConfig`
		`+ metadata:`
		`+ annotations:`
		`+ ansible.sdk.operatorframework.io/verbosity: '5'`
		`+ ----`
		`+`
		`+ This will enable full output from logging, which may aid in debugging.`
		`+`
		+ The following task list is contained inside the operator. This list is repeated in the reconcile loop which is currently set to run every `20 minutes`.
		`+`
		`+ ----`
		`+ # tasks file for Fas2discourseConfig`
		`+`
		`+ - include_tasks: retrieve_openshift_secrets.yml # Retrieves the secrets such as discourse api key etc and populates variable which feeds into the later tasks`
		`+ - include_tasks: kerberos_auth.yml # Authenticate to fasjson via keytab`
		`+ - include_tasks: retrieve_discourse_groups.yml # Contact Discourse API, retrieve the list of groups, and retrieve the list of users in each group`
		`+ - include_tasks: retrieve_ipa_groups.yml # Contact fasjson, using the Discourse group list, retrieve the membership of each group in IPA`
		`+ - include_tasks: sync_group_membership.yml # Using set functions, discover who is not in Discourse group but is in IPA group: add them. Who is in Discourse group but not in IPA group: remove them.`
		`+ ----`
		`+`
		`+ The results of each call in the workflow is outputted in the log. If any task fails the entire loop stops and retries.`

modules/ocp4/pages/sop_fas2discourse_operator_installation.adoc

file added

+34

		`@@ -0,0 +1,34 @@`
		`+ = Installation of the fas2discourse Operator`
		`+`
		`+ == Resources`
		`+ - [1] Code: https://pagure.io/cpe/fas2discourse/`
		`+ - [2] Playbook: https://pagure.io/fedora-infra/ansible/blob/main/f/playbooks/manual/fas2discourse.yml`
		`+ - [3] Role: https://pagure.io/fedora-infra/ansible/blob/main/f/roles/fas2discourse`
		`+`
		`+ == Installation on Fedora Infra`
		`+`
		+ There is a playbook [2] and role [3]. To install the operator in staging and production, run the playbook [2]. Users in the `sysadmin-openshift` group have permissions to run this playbook.
		`+`
		`+`
		`+ == Installation on a CRC cluster`
		`+ There is a Makefile bundled with the code [1] of this operator.`
		`+`
		`+ To install the operator the basic steps are followed:`
		`+`
		`+ - From a terminal, be logged into the cluster with cluster-admin privileges.`
		+ - Run `make deploy`
		`+`
		+ To activate the operator we need to create a `fas2discourseconfig` custom resource. An example of one exists in `config/samples/_v1alpha1_fas2discourseconfig.yaml`
		`+`
		`+ Create it with the following:`
		`+`
		`+ ----`
		`+ oc apply -f config/samples/_v1alpha1_fas2discourseconfig.yaml`
		`+ ----`
		`+`
		`+`
		`+`
		`+`
		`+ == Configuration`
		`+`
		`+ - No other configuration is required for this operator.`

modules/ocp4/pages/sop_fas2discourse_operator_interacting.adoc

file added

+48

		`@@ -0,0 +1,48 @@`
		`+ = Interacting with the fas2discourse Operator`
		`+`
		`+ == Resources`
		`+ - [1] Code: https://pagure.io/cpe/fas2discourse/`
		`+ - [2] Playbook: https://pagure.io/fedora-infra/ansible/blob/main/f/playbooks/manual/fas2discourse.yml`
		`+ - [3] Role: https://pagure.io/fedora-infra/ansible/blob/main/f/roles/fas2discourse`
		`+`
		`+ == Overview of the fas2discourse Operator`
		`+ The role of this operator is to synchronise group membership between IPA and Discourse. It does not synchronise all groups and all members, but only groups which exist in Discourse.`
		`+`
		`+ To start the synchronisation of a group, you must first request that a Discourse admin create it in Discourse. The fas2discourse operator will then begin to synchronise users to that group based on their membership in this group in IPA.`
		`+`
		`+ == Configuration of the fas2discourse operator`
		`+ All configuration for the fas2idscourse operator is contained in the Fedora Infra private ansible repo.`
		`+`
		`+ Default vars contains the list which are used in the playbook which deploys the operator:`
		`+`
		`+ ----`
		`+ fas2discourse_hostname: "fas2discourse.hostna.me"`
		`+ fas2discourse_namespace: "fas2discourse-operator"`
		`+ fas2discourse_project_description: "The fas2discourse-operator is responsible for synchronising group membership for users between Discourse and IPA."`
		`+ fas2discourse_keytab_file: "OVERRIDEME WITH A FILE LOOKUP"`
		`+ fas2discourse_discourse_apikey: "OVERRIDEME WITH A DISCOURSE APIKEY"`
		`+ ----`
		`+`
		`+ The Operator has the following vars which it uses internally. These vars are populated by querying secrets in Openshift:`
		`+`
		`+ ----`
		`+ # defaults file for Fas2discourseConfig`
		`+ fas2discourse_keytab_path: "/etc/fas2discourse"`
		`+ fas2discourse_principal: "fas2discourse/fas2discourse.hostna.me@FEDORAPROJECT.ORG"`
		`+ f2d_namespace: "fas2discourse-operator"`
		`+ f2d_secret: "fas2discourse-operator-k8s-secret"`
		`+ f2d_discourse_secret: "fas2discourse-operator-discourse-apikey-secret"`
		`+ fasjson_host: "OVERRIDEME"`
		`+ discourse_host: "OVERRIDEME"`
		`+ discourse_api: "OVERRIDEME"`
		`+ discourse_ignored_groups:`
		`+ - "admins"`
		`+ - "staff"`
		`+ - "moderators"`
		`+ - "trust_level_0"`
		`+ - "trust_level_1"`
		`+ - "trust_level_2"`
		`+ - "trust_level_3"`
		`+ - "trust_level_4"`
		`+ ----`
		`+`

modules/ocp4/pages/sop_fas2discourse_operator_testing.adoc

file added

+14

		`@@ -0,0 +1,14 @@`
		`+ = Test the fas2discourse Operator`
		`+`
		`+ == Resources`
		`+ - [1] Code: https://pagure.io/cpe/fas2discourse/`
		`+ - [2] Molecule: https://molecule.readthedocs.io/en/latest/`
		`+`
		`+ == Installation`
		`+ There is a molecule directory bundled with the code [1] of this operator. They currently are designed to only run against a code ready container cluster.`
		`+`
		`+ To run the operator molecule tests:`
		`+`
		+ - Ensure that the molecule utility is installed `dnf install python3-molecule`
		`+ - From a terminal, be logged into the crc cluster with cluster-admin privileges.`
		+ - Run `molecule test`

modules/ocp4/pages/sops.adoc

file modified

		`@@ -21,3 +21,4 @@`
		`- xref:sop_velero.adoc[SOP Velero]`
		`- xref:sop_aws_efs_operator.adoc[SOP AWS EFS Operator]`
		`- xref:sop_communishift.adoc[SOP Communishift Cluster Administration]`
		`+ - xref:sop_fas2discourse_operator.adoc[SOP fas2discourse operator]`

dkirwan commented 2 years ago

SOPs for deploying the fas2discourse operator on Fedora Infra

kevin commented on line 22 of modules/ocp4/pages/sop_fas2discourse_operator_build.adoc 2 years ago

typo here?

kevin commented 2 years ago

So is the config baked into the image? if we want to sync a new group, how can we do that? (or remove one)

Is there any way to debug what its doing?

Should we monitor anything here?

Thanks for the doc...

mattdm commented 2 years ago

So is the config baked into the image? if we want to sync a new group, how can we do that? (or remove one)

As I understand it, the current implementation does not create Discourse groups -- it syncs those that exist. So Discourse is effectively the configuration for that. (This can be done by anyone with Discourse admin. The admin also can optionally identify some group members as owners -- syncing that from sponsors is a potential future feature.)

Not sure about config for things like server name, api key, etc.

dkirwan commented 2 years ago

So configs are in the private ansible repo, search for fas2discourse to get a list of things such as discourse api key, few things like keytab principal etc.

Yep the way this works, it retrieves discourse groups, filters out a predetermined list of system groups which should be ignored eg the trust levels 0-4. Then queries fasjson, gets list of membership for each of these groups, then updates the discourse group membership to match what is in IPA via fasjson.

You add new groups, by creating a group in Discourse side, and as users login and create accounts on Discourse, they'll then be added to the groups which they are members of.

1 new commit added