fedora-infra / ansible

Clone
Source Code
GIT

#1295 Bodhi: try and do critical path data generation in openshift
Merged a year ago by kevin. Opened a year ago by adamwill.
fedora-infra/ adamwill/ansible bodhi-critpathcron  into  main

file modified
-3 
@@ -43,9 +43,6 @@ 
 

  

    - bodhi2/backend
 

  

-   - role: bodhi2/critpath
 

-     when: env == 'staging'
 

- 

    - role: nfs/client
 

      mnt_dir: '/mnt/koji/ostree'
 

      nfs_src_dir: 'fedora_ostree_content/ostree'

file modified
+16 
@@ -67,6 +67,10 @@ 
 

      tags:
 

        - apply-appowners
 

      when: env == "staging"
 

+   - role: openshift/object
 

+     app: bodhi
 

+     file: pvc.yml
 

+     objectname: pvc.yml
 

    - role: openshift/keytab
 

      app: bodhi
 

      key: koji-keytab
 
@@ -132,6 +136,13 @@ 
 

      app: bodhi
 

      template: deploymentconfig.yml
 

      objectname: deploymentconfig.yml
 

+   - role: openshift/object
 

+     app: bodhi
 

+     template: critpathcron.yml
 

+     objectname: critpathcron.yml
 

+   - role: openshift/imagestream
 

+     app: bodhi
 

+     imagename: bodhi-critpathcron
 

    - role: openshift/rollout
 

      app: bodhi
 

      dcname: bodhi-web
 
@@ -165,3 +176,8 @@ 
 

      command: "oc -n bodhi annotate route bodhi-web --overwrite haproxy.router.openshift.io/timeout=180s"
 

      tags:
 

      - route-timeout
 

+   - name: trigger critpath cronjob
 

+     command: "oc -n bodhi create job critpathcron-manual-{{ lookup('pipe','date +%s') }} --from=cronjob/fedora-bodhi-critpathcron"
 

+     tags:
 

+     - never
 

+     - critpathcron

file removed
-6 
@@ -1,6 +0,0 @@ 
 

- #!/bin/sh
 

- 

- # generate/update critical path data for Bodhi
 

- 

- cd /etc/bodhi/critpath
 

- /root/releng-git/scripts/critpath.py all

file removed
-31 
@@ -1,31 +0,0 @@ 
 

- - name: Install required packages
 

-   package:
 

-     name:
 

-       - git # for checking out critpath generation script
 

-       - python3-dnf # script uses this
 

-     state: present
 

-   tags:
 

-   - packages
 

- 

- - name: Create the bodhi critical path configuration directory
 

-   file:
 

-     path: /etc/bodhi/critpath
 

-     state: directory
 

-     owner: root
 

-     group: root
 

- 

- - name: Check out releng repo with the script in it
 

-   git:
 

-     repo: https://pagure.io/releng.git
 

-     dest: /root/releng-git
 

-   register: gitreleng
 

- 

- - name: Create a cron job to run the script daily
 

-   copy: src=critpath.cron dest=/etc/cron.daily/critpath owner=root group=root mode=0755
 

-   tags:
 

-   - config
 

- 

- - name: Do a first run of the script if it has never run
 

-   command:
 

-     cmd: /etc/cron.daily/critpath
 

-     creates: /etc/bodhi/critpath/rawhide.json

file added
+11 
@@ -0,0 +1,11 @@ 
 

+ apiVersion: v1
 

+ kind: PersistentVolumeClaim
 

+ metadata:
 

+   name: bodhi-critpath-storage
 

+ spec:
 

+   accessModes:
 

+     - ReadWriteMany
 

+   resources:
 

+     requests:
 

+       storage: 5Mi
 

+   storageClassName: ocs-storagecluster-cephfs

file modified
+24 
@@ -25,5 +25,29 @@ 
 

        to:
 

          kind: ImageStreamTag
 

          name: bodhi-base:latest
 

+ - apiVersion: v1
 

+   kind: BuildConfig
 

+   metadata:
 

+     labels:
 

+       build: bodhi-critpathcron
 

+     name: bodhi-critpathcron
 

+   spec:
 

+     runPolicy: Serial
 

+     source:
 

+       dockerfile: |-
 

+         {{ load_file('dockerfile-critpathcron') | indent(8) }}
 

+       type: Dockerfile
 

+     strategy:
 

+       type: Docker
 

+       dockerStrategy:
 

+ {% if env == "staging" %}
 

+         noCache: True
 

+ {% else %}
 

+         noCache: False
 

+ {% endif %}
 

+     output:
 

+       to:
 

+         kind: ImageStreamTag
 

+         name: bodhi-critpathcron:latest
 

  kind: List
 

  metadata: {}

file added
+42 
@@ -0,0 +1,42 @@ 
 

+ apiVersion: batch/v1
 

+ kind: CronJob
 

+ metadata:
 

+   name: fedora-bodhi-critpathcron
 

+ spec:
 

+   concurrencyPolicy: Forbid
 

+   schedule: "10 3 * * *"
 

+   successfulJobsHistoryLimit: 1
 

+   failedJobsHistoryLimit: 1
 

+   activeDeadlineSeconds: 900
 

+   startingDeadlineSeconds: 300
 

+   jobTemplate:
 

+     spec:
 

+       backoffLimit: 0
 

+       template:
 

+         metadata:
 

+           labels:
 

+             parent: "cronjobbuild"
 

+         spec:
 

+           containers:
 

+           - name: critpath
 

+             image: image-registry.openshift-image-registry.svc:5000/bodhi/bodhi-critpathcron:latest
 

+             command: ["/bin/bash", "-c"]
 

+             args:
 

+             - |
 

+               cd /tmp
 

+               git clone --depth 1 --single-branch https://pagure.io/releng.git
 

+               cd /bodhi-critpath
 

+               /tmp/releng/scripts/critpath.py all
 

+             env:
 

+             - name: BUILD_ENV
 

+               value: "{{ env_short }}"
 

+             volumeMounts:
 

+             - name: bodhi-critpath
 

+               mountPath: /bodhi-critpath
 

+               readOnly: false
 

+           restartPolicy: Never
 

+           startingDeadlineSeconds: 600
 

+           volumes:
 

+           - name: bodhi-critpath
 

+             persistentVolumeClaim:
 

+               claimName: bodhi-critpath-storage

file modified
+24 
@@ -53,6 +53,9 @@ 
 

            - name: config-volume
 

              mountPath: /etc/bodhi
 

              readOnly: true
 

+           - name: bodhi-critpath-storage
 

+             mountPath: /etc/bodhi/critpath
 

+             readOnly: true
 

            - name: keytab-volume
 

              mountPath: /etc/keytabs
 

              readOnly: true
 
@@ -91,6 +94,9 @@ 
 

          - name: config-volume
 

            configMap:
 

              name: bodhi-configmap
 

+         - name: bodhi-critpath-storage
 

+           persistentVolumeClaim:
 

+             claimName: "bodhi-critpath-storage"
 

          - name: keytab-volume
 

            secret:
 

              secretName: bodhi-keytab
 
@@ -164,6 +170,9 @@ 
 

            - name: config-volume
 

              mountPath: /etc/bodhi
 

              readOnly: true
 

+           - name: bodhi-critpath-storage
 

+             mountPath: /etc/bodhi/critpath
 

+             readOnly: true
 

            - name: keytab-volume
 

              mountPath: /etc/keytabs
 

              readOnly: true
 
@@ -186,6 +195,9 @@ 
 

          - name: config-volume
 

            configMap:
 

              name: bodhi-configmap
 

+         - name: bodhi-critpath-storage
 

+           persistentVolumeClaim:
 

+             claimName: "bodhi-critpath-storage"
 

          - name: keytab-volume
 

            secret:
 

              secretName: bodhi-keytab
 
@@ -257,6 +269,9 @@ 
 

            - name: config-volume
 

              mountPath: /etc/bodhi
 

              readOnly: true
 

+           - name: bodhi-critpath-storage
 

+             mountPath: /etc/bodhi/critpath
 

+             readOnly: true
 

            - name: keytab-volume
 

              mountPath: /etc/keytabs
 

              readOnly: true
 
@@ -279,6 +294,9 @@ 
 

          - name: config-volume
 

            configMap:
 

              name: bodhi-configmap
 

+         - name: bodhi-critpath-storage
 

+           persistentVolumeClaim:
 

+             claimName: "bodhi-critpath-storage"
 

          - name: keytab-volume
 

            secret:
 

              secretName: bodhi-keytab
 
@@ -352,6 +370,9 @@ 
 

            - name: config-volume
 

              mountPath: /etc/bodhi
 

              readOnly: true
 

+           - name: bodhi-critpath-storage
 

+             mountPath: /etc/bodhi/critpath
 

+             readOnly: true
 

            - name: keytab-volume
 

              mountPath: /etc/keytabs
 

              readOnly: true
 
@@ -374,6 +395,9 @@ 
 

          - name: config-volume
 

            configMap:
 

              name: bodhi-configmap
 

+         - name: bodhi-critpath-storage
 

+           persistentVolumeClaim:
 

+             claimName: "bodhi-critpath-storage"
 

          - name: keytab-volume
 

            secret:
 

              secretName: bodhi-keytab

file added
+16 
@@ -0,0 +1,16 @@ 
 

+ {% if env == "staging" %}
 

+ FROM fedora:37
 

+ {% else %}
 

+ FROM fedora:37
 

+ {% endif %}
 

+ LABEL \
 

+     name="bodhi-critpathcron" \
 

+     vendor="Fedora Infrastructure" \
 

+     license="MIT"
 

+ {% if env == "staging" %}
 

+ RUN curl -o /etc/yum.repos.d/infra-tags.repo https://infrastructure.fedoraproject.org/infra/ansible/files/common/fedora-infra-tags.repo
 

+ RUN curl -o /etc/yum.repos.d/infra-tags-stg.repo https://infrastructure.fedoraproject.org/infra/ansible/files/common/fedora-infra-tags-stg.repo
 

+ {% else %}
 

+ RUN curl -o /etc/yum.repos.d/infra-tags.repo https://infrastructure.fedoraproject.org/infra/ansible/files/common/fedora-infra-tags.repo
 

+ {% endif %}
 

+ RUN dnf install -y python3-dnf git fedora-repos

The previous attempt to have staging Bodhi use grouped JSON
critical path data didn't work because we did it in the backend
(i.e. the composer), not the openshift pods where the web UI
and the consumers run.

We need at least the web UI and consumer pods to have the
critpath data, as both those pods may create updates (Bodhi
decides if the update is critical path at creation time). This
attempts to handle that by having a daily openshift cron job
that runs a simple container with the necessary packages in it,
checks out the script, and runs it. It's run on a persistent
storage volume which is also mounted by all the Bodhi pods in
the place where Bodhi will look for the data.

The cron job frequency is temporarily set to once an hour; this
is so it will run soon after initial deployment. Once it has
run once we can cut it back to once a day.

This also includes a commit to revert most of the previous attempt (keeping only the config file change, which we still need).

rebased onto 91c0302df61d1a383ea539b7e4444fef858959da
a year ago

Build succeeded.

This seems ok to me from a quick glance... but it's 6pm on a friday. :)

Perhaps @darknao and/or @dkirwan could see if we missed anything here?

yeah, another pair of eyes would be good. Bear in mind I've never done much with openshift before: I may have missed/screwed up something really obvious that no first-day intern would get wrong, so look at everything :D thanks!

to be clear, what we're trying to achieve here is to get the results of a simple script from the releng git repo placed in /etc/bodhi/critpath so far as each of the bodhi pods is concerned.

namespace is not right here, should be:
image-registry.openshift-image-registry.svc:5000/bodhi/bodhi-critpathcron:latest
You also need to create this ImageStream first in the bodhi playbook:

  - role: openshift/imagestream
    app: bodhi
    imagename: bodhi-critpathcron

* 10 * * * is every minute of the 10th hour. Is that what you meant?

You specified an empty storageClass here. It's ok if you want a manually provisioned NFS share and need access to it from outside the cluster.
It this storage is only used by pods inside the cluster, I suggest you use a CephFS volume instead (storageClassName: ocs-storagecluster-cephfs).

The cron job frequency is temporarily set to once an hour; this
is so it will run soon after initial deployment. Once it has
run once we can cut it back to once a day.

You can trigger the cronjob on-demand if you want, with something like this:

  post_tasks:
  - name: trigger critpath cronjob
    command: "oc -n bodhi create job critpathcron-manual-{{ lookup('pipe','date +%s') }} --from=cronjob/fedora-bodhi-critpathcron"
    tags:
    - never
    - critpathcron

Then run with sudo rbac-playbook -l os_control_stg[0] -t critpathcron openshift-apps/bodhi.yml

Thanks a lot for the review and corrections! I'll fix this up shortly.

rebased onto f0f2f685e650dee82a9281c84461cc6405f199d1
a year ago

OK, rebased this and incorporated all of @darknao 's fixes. Thanks again.

Build succeeded.

rebased onto 95e4480
a year ago

Build succeeded.

Pull-Request has been merged by kevin
a year ago
Metadata
None
No Tags
Changes Summary 9
+0 -3
file changed
playbooks/groups/bodhi-backend.yml
+16 -0
file changed
playbooks/openshift-apps/bodhi.yml
-6
file removed
roles/bodhi2/critpath/files/critpath.cron
-31
file removed
roles/bodhi2/critpath/tasks/main.yml
+11
file added
roles/openshift-apps/bodhi/files/pvc.yml
+24 -0
file changed
roles/openshift-apps/bodhi/templates/buildconfig.yml
+42
file added
roles/openshift-apps/bodhi/templates/critpathcron.yml
+24 -0
file changed
roles/openshift-apps/bodhi/templates/deploymentconfig.yml
+16
file added
roles/openshift-apps/bodhi/templates/dockerfile-critpathcron
Powered by Pagure 5.13.3
Documentation • File an Issue • About • SSH Hostkey/Fingerprint
© Red Hat, Inc. and others.