#4436 Requesting resources for Fedimg
Closed: Fixed None Opened 9 years ago by oddshocks.

= phenomenon =

Fedora images that are currently built in Koji must be uploaded to cloud providers such as EC2 and our own internal cloud services by hand. Fedora wants to remove the step where someone has to "press the button". [https://github.com/fedora-infra/fedimg Fedimg] is an application being developed this summer which resolves that problem. It listens to the Fedmsg hub for successful Koji image builds and makes them available on internal and external cloud providers.

= reason =

This would resolve [https://fedorahosted.org/cloud/ticket/35 ticket #35].

Thanks to Fedmsg, manually triggering image uploads is a step that can be eliminated. Currently, Ubuntu far surpasses us in cloud image uploading and availability because they automatically upload their image builds to cloud providers. Not only will Fedimg improve Fedora's cloud presence, it will also make Fedora images far more available to everyone. This could be a major win for many Fedora teams, including infrastructure, releng, and the cloud working group.

= recommendation =

Fedimg emits its own fedmsgs and will use Datanommer as a database, leveraging our awesome, already-existing infrastructure. Additionally, image builds are something that happen [http://koji.fedoraproject.org/koji/tasks?method=createImage&state=closed&view=tree&order=-id relatively infrequently] (0-8 or so times a day), so there is not a huge workload. Therefore, its resource requirements shouldn't be complicated.

First, we need an isolated, secure machine that will run Fedimg. We don't need a lot of disk space because image file downloading and handling is done on "utility instances" we spin up in places like EC2. The heavy processing occurs in the cloud -- Fedimg just waits for completed builds and kicks off the image uploading process in cloud-land. Of course, this means that the second requirement is that we have access to cloud resources. Currently, Fedora is already set up with sufficient EC2 resources and the costs are covered. Fedimg doesn't perform any tasks which require extensive cloud costs, nor does it keep cloud resources around for long, so the additional costs to our EC2 bill will be negligible. However, we don't yet have accounts with services like Rackspace, GCE, or HP, but we're working on it (blocking on legal right now, who are trying to establish agreements between these providers and Fedora). Once we're hooked up with these other providers, Fedimg can work with them, too.

I suggest we stand up a staging and production machine for Fedimg, and agree that it will have access to official Fedora account credentials for cloud services like EC2.


OK.

I've setup fedimg01.stg and fedimg01. They don't have any of the fedimg-specific config yet, but they do have all the basics in place.

I've also added the fedimg playbook to 'rbac-playbook' such that members of the groups 'sysadmin-releng' and 'sysadmin-datanommer' now have permissions to run it.

The AWS credentials will still need to be placed in the private git repo by a member of sysadmin-main. contact one of us privately to manage that, please.

Login to comment on this ticket.

Metadata