PR#1930: Add a role and playbook for the cloud-image-uploader - fedora-infra/ansible

fedora-infra / ansible

#1930 Add a role and playbook for the cloud-image-uploader

Merged 21 days ago by kevin. Opened a month ago by jcline.

fedora-infra/ jcline/ansible cloud-uploader into main

cloud-image-uploader: swap staging and prod blocks in config

Jeremy Cline • 21 days ago

cf9b606

Add a role and playbook for the cloud-image-uploader

Jeremy Cline • 21 days ago

e564d0c

playbooks/openshift-apps/cloud-image-uploader.yml

file added

+77

		`@@ -0,0 +1,77 @@`
		`+ - name: make the app be real`
		`+ hosts: os_control_stg`
		`+ user: root`
		`+ gather_facts: False`
		`+`
		`+ vars_files:`
		`+ - /srv/web/infra/ansible/vars/global.yml`
		`+ - "/srv/private/ansible/vars.yml"`
		`+ - /srv/web/infra/ansible/vars/{{ ansible_distribution }}.yml`
		`+`
		`+ roles:`
		`+ - role: rabbit/queue`
		`+ username: "cloud-image-uploader"`
		`+ queue_name: "cloud-image-uploader"`
		`+ routing_keys:`
		`+ - "org.fedoraproject.{{ env }}.buildsys.build.state.change"`
		`+ thresholds:`
		`+ warning: 10`
		`+ critical: 50`
		`+`
		`+ - role: openshift/project`
		`+ app: cloud-image-uploader`
		`+ description: AMQP consumer that uploads Cloud images to cloud providers`
		`+ appowners:`
		`+ - jcline`
		`+`
		`+ - role: openshift/object`
		`+ app: cloud-image-uploader`
		`+ file: imagestream.yml`
		`+ objectname: imagestream.yml`
		`+`
		`+ - role: openshift/object`
		`+ app: cloud-image-uploader`
		`+ template: buildconfig.yml`
		`+ objectname: buildconfig.yml`
		`+`
		`+ - role: openshift/object`
		`+ app: cloud-image-uploader`
		`+ template: configmap.yml`
		`+ objectname: configmap.yml`
		`+`
		`+ - role: openshift/secret-file`
		`+ app: cloud-image-uploader`
		`+ secret_name: cloud-image-uploader-fedora-messaging-key`
		`+ key: cloud-image-uploader.key`
		`+ privatefile: "rabbitmq/{{env}}/pki/private/cloud-image-uploader{{env_suffix}}.key"`
		`+`
		`+ - role: openshift/secret-file`
		`+ app: cloud-image-uploader`
		`+ secret_name: cloud-image-uploader-fedora-messaging-crt`
		`+ key: cloud-image-uploader.crt`
		`+ privatefile: "rabbitmq/{{env}}/pki/issued/cloud-image-uploader{{env_suffix}}.crt"`
		`+`
		`+ - role: openshift/secret-file`
		`+ app: cloud-image-uploader`
		`+ secret_name: cloud-image-uploader-fedora-messaging-ca`
		`+ key: cloud-image-uploader.ca`
		`+ privatefile: "rabbitmq/{{env}}/pki/ca.crt"`
		`+`
		`+ - role: openshift/object`
		`+ app: cloud-image-uploader`
		`+ template: secret.yml`
		`+ objectname: secret.yml`
		`+`
		`+ - role: openshift/start-build`
		`+ app: cloud-image-uploader`
		`+ buildname: cloud-image-uploader-build`
		`+ objectname: cloud-image-uploader-build`
		`+`
		`+ - role: openshift/object`
		`+ app: cloud-image-uploader`
		`+ file: deployment.yml`
		`+ objectname: deployment.yml`
		`+`
		`+ - role: openshift/rollout`
		`+ app: cloud-image-uploader`
		`+ dcname: cloud-image-uploader`

roles/openshift-apps/cloud-image-uploader/files/imagestream.yml

file added

+11

		`@@ -0,0 +1,11 @@`
		`+ ---`
		`+ apiVersion: image.openshift.io/v1`
		`+ items:`
		`+ - apiVersion: image.openshift.io/v1`
		`+ kind: ImageStream`
		`+ metadata:`
		`+ name: cloud-image-uploader`
		`+ labels:`
		`+ build: cloud-image-uploader`
		`+ kind: List`
		`+ metadata: {}`

roles/openshift-apps/cloud-image-uploader/templates/buildconfig.yml

file added

+31

		`@@ -0,0 +1,31 @@`
		`+ apiVersion: build.openshift.io/v1`
		`+ items:`
		`+ - apiVersion: build.openshift.io/v1`
		`+ kind: BuildConfig`
		`+ metadata:`
		`+ labels:`
		`+ build: cloud-image-uploader-build`
		`+ name: cloud-image-uploader-build`
		`+ spec:`
		`+ runPolicy: Serial`
		`+ source:`
		`+ type: git`
		`+ git:`
		`+ uri: https://pagure.io/cloud-image-uploader.git`
		`+ {% if env == 'staging' %}`
		`+ ref: main`
		`+ {% else %}`
		`+ ref: prod`
		`+ {% endif %}`
		`+ strategy:`
		`+ type: Docker`
		`+ dockerStrategy:`
		`+ dockerfilePath: Containerfile`
		`+ triggers:`
		`+ - type: ImageChange`
		`+ output:`
		`+ to:`
		`+ kind: ImageStreamTag`
		`+ name: cloud-image-uploader:latest`
		`+ kind: List`
		`+ metadata: {}`

roles/openshift-apps/cloud-image-uploader/templates/config.toml

file added

+101

		`@@ -0,0 +1,101 @@`
		`+ # This file is in the TOML format.`
		`+ # For complete details on all configuration options, see the documentation`
		`+ # https://fedora-messaging.readthedocs.io/en/latest/configuration.html.`
		`+`
		`+ amqp_url = "amqps://cloud-image-uploader:@rabbitmq{{ env_suffix }}.fedoraproject.org/%2Fpubsub"`
		`+ callback = "fedora_cloud_image_uploader:Uploader"`
		`+ passive_declares = true`
		`+`
		`+ [tls]`
		`+ ca_cert = "/etc/pki/rabbitmq/ca/fedora-messaging-cloud-image-uploader-ca.crt"`
		`+ keyfile = "/etc/pki/rabbitmq/key/fedora-messaging-cloud-image-uploader.key"`
		`+ certfile = "/etc/pki/rabbitmq/cert/fedora-messaging-cloud-image-uploader.crt"`
		`+`
		`+ [client_properties]`
		`+ app = "Fedora Cloud Image Uploader"`
		`+ app_url = "https://pagure.io/cloud-image-uploader"`
		`+ app_contacts_email = "cloud@lists.fedoraproject.org"`
		`+`
		`+ [[bindings]]`
		`+ queue = "fedora-image-uploader"`
		`+ exchange = "amq.topic"`
		`+ routing_keys = ["org.fedoraproject.{{ env }}.buildsys.build.state.change"]`
		`+`
		`+ [queues.fedora-image-uploader]`
		`+ durable = true`
		`+ auto_delete = false`
		`+ exclusive = false`
		`+ arguments = {}`
		`+`
		`+ [consumer_config.azure]`
		`+ location = "eastus"`
		`+ {% if env == "staging" %}`
		`+ resource_group_name = "fedora-cloud-staging"`
		`+ storage_account_name = "fedoraimagesstaging"`
		`+ {% else %}`
		`+ resource_group_name = "fedora-cloud"`
		`+ storage_account_name = "fedoraimages"`
		`+ {% endif %}`
		`+ gallery_name = "Fedora"`
		`+ gallery_description = "The Fedora compute gallery."`
		`+ storage_container_name = "vhds"`
		`+`
		`+ # A list of regions and replication settings for uploaded images.`
		`+ #`
		`+ # Images need to be replicated to the region to be usable.`
		`+ {% if env == "staging" %}`
		`+`
		`+ [[consumer_config.azure.target_regions]]`
		`+ name = "eastus"`
		`+ regional_replica_count = 1`
		`+ storage_account_type = "Standard_LRS"`
		`+`
		`+ {% else %}`
		`+`
		`+ [[consumer_config.azure.target_regions]]`
		`+ name = "eastus"`
		`+ regional_replica_count = 3`
		`+ storage_account_type = "Standard_ZRS"`
		`+`
		`+ {% endif %}`
		`+`
		`+`
		`+ [qos]`
		`+ prefetch_size = 0`
		`+ prefetch_count = 25`
		`+`
		`+ [log_config]`
		`+ version = 1`
		`+ disable_existing_loggers = true`
		`+`
		`+ [log_config.formatters.simple]`
		`+ format = "[%(asctime)s %(name)s %(levelname)s] %(message)s"`
		`+`
		`+ [log_config.handlers.console]`
		`+ class = "logging.StreamHandler"`
		`+ formatter = "simple"`
		`+ stream = "ext://sys.stdout"`
		`+`
		`+ [log_config.loggers.fedora_messaging]`
		`+ level = "INFO"`
		`+ propagate = false`
		`+ handlers = ["console"]`
		`+`
		`+ [log_config.loggers.fedora_cloud_image_uploader]`
		`+ level = "INFO"`
		`+ propagate = false`
		`+ handlers = ["console"]`
		`+`
		`+ [log_config.loggers.twisted]`
		`+ level = "INFO"`
		`+ propagate = false`
		`+ handlers = ["console"]`
		`+`
		`+ [log_config.loggers.pika]`
		`+ level = "WARNING"`
		`+ propagate = false`
		`+ handlers = ["console"]`
		`+`
		`+ [log_config.root]`
		`+ level = "ERROR"`
		`+ handlers = ["console"]`

roles/openshift-apps/cloud-image-uploader/templates/configmap.yml

file added

+15

		`@@ -0,0 +1,15 @@`
		`+ {%- macro load_file(filename) %}{% include filename %}{%- endmacro -%}`
		`+ ---`
		`+ apiVersion: v1`
		`+ kind: List`
		`+ metadata: {}`
		`+ items:`
		`+ - apiVersion: v1`
		`+ kind: ConfigMap`
		`+ metadata:`
		`+ name: cloud-image-uploader-configmap`
		`+ labels:`
		`+ app: cloud-image-uploader`
		`+ data:`
		`+ config.toml: \|-`
		`+ {{ lookup('template', 'config.toml') \| indent(6) }}`

roles/openshift-apps/cloud-image-uploader/templates/deployment.yml

file added

+79

		`@@ -0,0 +1,79 @@`
		`+ apiVersion: apps/v1`
		`+ kind: Deployment`
		`+ metadata:`
		`+ name: cloud-image-uploader`
		`+ annotations:`
		`+ image.openshift.io/triggers: >-`
		`+ [`
		`+ {`
		`+ "from": {`
		`+ "kind":"ImageStreamTag",`
		`+ "namespace":"cloud-image-uploader"`
		`+ "name":"cloud-image-uploader:latest",`
		`+ },`
		`+ "fieldPath":"spec.template.spec.containers[?(@.name==\"cloud-image-uploader\")].image",`
		`+ "pause":"false"`
		`+ }`
		`+ ]`
		`+ spec:`
		`+ replicas: 1`
		`+ selector:`
		`+ app: cloud-image-uploader`
		`+ strategy:`
		`+ type: Recreate`
		`+ template:`
		`+ metadata:`
		`+ labels:`
		`+ app: cloud-image-uploader`
		`+ spec:`
		`+ volumes:`
		`+ - name: config-volume`
		`+ configMap:`
		`+ name: cloud-image-uploader-configmap`
		`+ - name: fedora-messaging-ca-volume`
		`+ secret:`
		`+ secretName: cloud-image-uploader-fedora-messaging-ca`
		`+ - name: fedora-messaging-key-volume`
		`+ secret:`
		`+ secretName: cloud-image-uploader-fedora-messaging-key`
		`+ - name: fedora-messaging-cert-volume`
		`+ secret:`
		`+ secretName: cloud-image-uploader-fedora-messaging-crt`
		`+ containers:`
		`+ - name: cloud-image-uploader`
		`+ image: cloud-image-uploader/cloud-image-uploader:latest`
		`+ imagePullPolicy: Always`
		`+ env:`
		`+ - name: AZURE_SECRET`
		`+ valueFrom:`
		`+ secretKeyRef:`
		`+ name: azure-credentials`
		`+ key: secret`
		`+ - name: AZURE_CLIENT_ID`
		`+ valueFrom:`
		`+ secretKeyRef:`
		`+ name: azure-credentials`
		`+ key: client_id`
		`+ - name: AZURE_TENANT`
		`+ valueFrom:`
		`+ secretKeyRef:`
		`+ name: azure-credentials`
		`+ key: tenant_id`
		`+ - name: AZURE_SUBSCRIPTION_ID`
		`+ valueFrom:`
		`+ secretKeyRef:`
		`+ name: azure-credentials`
		`+ key: subscription_id`
		`+ volumeMounts:`
		`+ - name: config-volume`
		`+ mountPath: /etc/fedora-messaging`
		`+ readOnly: true`
		`+ - name: fedora-messaging-ca-volume`
		`+ mountPath: /etc/pki/rabbitmq/ca`
		`+ readOnly: true`
		`+ - name: fedora-messaging-key-volume`
		`+ mountPath: /etc/pki/rabbitmq/key`
		`+ readOnly: true`
		`+ - name: fedora-messaging-cert-volume`
		`+ mountPath: /etc/pki/rabbitmq/cert`
		`+ readOnly: true`

roles/openshift-apps/cloud-image-uploader/templates/secret.yml

file added

+17

		`@@ -0,0 +1,17 @@`
		`+ apiVersion: v1`
		`+ kind: Secret`
		`+ metadata:`
		`+ name: "azure-credentials"`
		`+ labels:`
		`+ app: "cloud-image-uploader"`
		`+ data:`
		`+ {% if env == 'staging' %}`
		`+ secret: "{{stg_azure_secret}}"`
		`+ client_id: "{{stg_azure_client_id}}"`
		`+ tenant_id: "{{stg_azure_tenant_id}}"`
		`+ {% else %}`
		`+ secret: "{{prod_azure_secret}}"`
		`+ client_id: "{{prod_azure_client_id}}"`
		`+ tenant_id: "{{prod_azure_tenant_id}}"`
		`+ {% endif %}`
		`+ subscription_id: "{{azure_subscription_id}}"`

jcline commented a month ago

This app is an AQMP client that uploads VM images to public clouds. It
currently supports Azure images.

Ref: https://pagure.io/fedora-infrastructure/issue/11860

This assumes that a client cert has been set up for the fedora-messaging consumer, as well as the presence of the following ansible secrets:

stg_azure_secret
stg_azure_client_id
stg_azure_tenant_id
prod_azure_secret
prod_azure_client_id
prod_azure_tenant_id
azure_subscription_id

I need to do a little research to figure out the minimal permission set required before I can generate the Azure secrets, but I figured I'd go ahead and open this up.

kevin commented a month ago

Looks pretty good. It's like you've seen this entire setup before? :)

I'd suggest setting 'hosts: os_control_stg' for now and we can then add prod when everything is working in stg.

Also, deploymentconfig is actually going away (we need to convert all our existing apps, @darknao was gonna look at doing that). Not a stopper, but if you want to convert to deployment now, it won't need converting...

jcline commented a month ago

Looks pretty good. It's like you've seen this entire setup before? :)

Heh, perhaps, but I definitely did a lot of copy/pasting.

I'd suggest setting 'hosts: os_control_stg' for now and we can then add prod when everything is working in stg.

Will do.

Also, deploymentconfig is actually going away (we need to convert all our existing apps, @darknao was gonna look at doing that). Not a stopper, but if you want to convert to deployment now, it won't need converting...

Sure, I'm happy to move to whatever replaces deploymentconfig, any pointers to what to do instead?

kevin commented a month ago

Yeah, deploymentconfig was a openshift add on, they are dropping it now and wanting everyone to move to the k8s 'deployment' object instead.

There's a few apps already moved, you can look for 'deployment.yml' in the ansible repo. It's pretty similar at this point...

rebased onto 315389afd434a56b11de2ff9af3cfaae8225827b

a month ago

rebased onto da8a95d

a month ago

jcline commented a month ago

Yeah, deploymentconfig was a openshift add on, they are dropping it now and wanting everyone to move to the k8s 'deployment' object instead.

There's a few apps already moved, you can look for 'deployment.yml' in the ansible repo. It's pretty similar at this point...

Perfect, I moved it to a vanilla deployment and renamed the file for clarity. I also dropped the os_control from the hosts.

kevin commented 24 days ago

Looks good to me.

Perhaps we can merge and deploy early next week sometime? If you let me know when you are available I can do so and we can work through any problems.

I can also setup things so you can run the playbook, but since we are in freeze, that would need a freeze break. :(

jcline commented 22 days ago

Super. Maybe Tuesday? I'll be available any time except between 11:30-12:30 EDT. How should I get the Azure app credentials to you? It probably makes sense to add you to the Azure group, which @dustymabe did for me, in which case you can just create the secret yourself.

I think we can wait until after freeze to get permissions for me to run it set up since I can run the service locally to manually upload images.

Edited 22 days ago by jcline

1 new commit added

cloud-image-uploader: swap staging and prod blocks in config

22 days ago

zuul commented 22 days ago

Build succeeded.
https://fedora.softwarefactory-project.io/zuul/buildset/194434588ef643078892e8766b3e65ab

fi-ansible-lint-diff : SUCCESS in 3m 56s
fi-yamllint-diff : SUCCESS in 2m 35s

kevin commented 22 days ago

I might have access there already, not sure. But if not, you can add me, or just create the credentials and put them in a mode 600 file on batcave01? or gpg them to me, I can send pubkey...

jcline commented 21 days ago

I'm happy with any of those options. I don't think I have access to batcave01 anymore (or, at least, the keys I have in accounts.fedoraproject.org don't work). I forget how that works, probably a FAS group?