#301 regular releases: uploading to clouds
Opened 7 months ago by dustymabe. Modified a month ago

In the 04/14/2020 community meeting we discussed what it would take to ship cloud images more regularly.

One work item that is on the Cloud working group is to control the uploads to clouds. We currently do this with fedimg but we want to replace it with ore like what is done to upload images for Fedora CoreOS.

We'd essentially make a new listener app that watches the fedmsg bus for when a new compose is done and then grabs the cloud images and uploads them to each respective cloud.

We discussed this in the community meeting yesterday.

@michaelgugino volunteered to give a look at this with guidance from @dustymabe!

Thanks @michaelgugino!

I think the way we achieve this is to write an openshift app that listens to fedmsgs, grabs the cloud image file, then kicks off upload/create to respective clouds.

It might be worth considering using SUSE's Mash tool to do this. It looks like it covers all the bases we need, is generic, supports emitting messages on AMQP already, which means it'd be fedora-messaging compatible, and it supports all the clouds we care about (AWS, GCP, and Azure).

The only thing is the unfortunate name (it collides with the existing mash tool previously used in Fedora infrastructure)... We might want to talk to them about renaming it.

A few of us in the group met to discuss this on Monday.

Currently @michaelgugino is going to look at setting up an application to listen to fedmsg events and do some task. The current proposal is to use ore to upload cloud images, but @ngompa is looking to see if there are other healthily maintained and commonly used projects that we could possibly use instead.

Sorry to be late on this.. Finally got around to showing the example of how to upload to GCP using ore:

$ ore gcloud --project fedora-coreos-devel \
             --json-key /srv/dustymabe-service-account-fedora-coreos-devel.json \
             --log-level INFO upload --basename fedora-cloud                    \
             --bucket gs://fedora-coreos-devel-image-uploads/image-import       \
             --name fedora-cloud-base-gcp-33-1-2-x86-64                         \
             --file /srv/Fedora-Cloud-Base-GCP-33-1.2.x86_64.tar.gz             \
             --description "Fedora, Fedora Cloud Base, 33, x86_64 published on 2020-10-29" \
             --license https://compute.googleapis.com/compute/v1/projects/vm-options/global/licenses/enable-vmx

2020-10-29T20:24:30Z cli: Started logging at level INFO
Writing /srv/Fedora-Cloud-Base-GCP-33-1.2.x86_64.tar.gz to gs://fedora-coreos-devel-image-uploads/image-import/fedora-cloud-base-gcp-33-1-2-x86-64.tar.gz ...
(Sometimes this takes a few minutes)
Upload successful!
Creating image in GCE: fedora-cloud-base-gcp-33-1-2-x86-64...

Now I can launch instances with fedora-cloud-base-gcp-33-1-2-x86-64.

Login to comment on this ticket.