#8281 Create fedora-cloud and fedora-testing Google Cloud projects
Closed: Fixed 3 years ago by dustymabe. Opened 4 years ago by bgilbert.

One of the target platforms for the Fedora CoreOS stable release is Google Cloud Platform. We plan to create GCP images for every Fedora CoreOS release (on the three production streams, testing, stable, and next) so users can launch Fedora CoreOS in GCP without having to upload their own disk images. We'll also maintain a GCP image family for each production stream, which will point to the recommended OS image for that stream.

To support this, we'd like Infra to create two GCP projects:

  • fedora-cloud, which will contain the published disk images (in GCS), OS images (in GCE), and image families (also in GCE). The name fedora-cloud would allow Fedora to use the same project for other Fedora editions in the future. Alternatively, if Infra would prefer a dedicated project specifically for Fedora CoreOS, it could be called fedora-coreos-cloud instead.

    The name of the project, as well as its contents (such as the names of image families), will be long-term user-facing ABI. Once this project is created, we'll have Google mark the project public, which will cause all images in the project to be publicly accessible. By convention, public projects are named with the suffix -cloud.

  • fedora-testing (or other suitable name), for private use by Fedora CoreOS CI, and also for testing our image publication pipeline.

Both projects should be accessible by several folks in the Fedora CoreOS working group (at least @dustymabe, @jlebon, @slowrie, and myself). We'll also want a machine account for the publication pipeline.

I'd be happy to answer any questions y'all might have, and to work with Infra folks to get this set up.

Thanks in advance!


My questions are more to make sure I know where said project is being made, if these are in Google: who is paying for it, do we have a legal agreement, etc. [We don't have any GCP accounts currently that I am aware of so usually have to start a chain of legal queries etc.]

It's my understanding that the CoreOS team has worked through the legal situation. Benjamin can you confirm? Also, I'm working on figuring out the financial aspect.

So, let's focus on the technical / practical here.

What are the expectation in terms of date for this to be done ? Since we have already quite a few different things in progress.

Having a rough idea of when this is needed would help us plan that.

Metadata Update from @cverna:
- Issue tagged with: backlog

4 years ago

Some questions to answer:

  • Does infra have a GCP account already? I'm betting not, because the amorphous legal stuff has been in limbo so long.
  • Is setting up these projects a pretty quick effort (i.e. if it's anything like setting up an API or other project in their spaces, it may not be a big deal)?

If I read this right, the request here is not related to code at all, and just about creating namespaces in a way the team can live with long term. However, I'll wait for @bgilbert to confirm that too.

Some questions to answer:

Does infra have a GCP account already? I'm betting not, because the amorphous legal stuff has been in limbo so long.

No. None that I am aware of.

Is setting up these projects a pretty quick effort (i.e. if it's anything like setting up an API or other project in their spaces, it may not be a big deal)?

Completely unknown since we have never dealt with GCP before.

If I read this right, the request here is not related to code at all, and just about creating namespaces in a way the team can live with long term. However, I'll wait for @bgilbert to confirm that too.

Some questions to answer:
Does infra have a GCP account already? I'm betting not, because the amorphous legal stuff has been in limbo so long.

We have a Google account (used for e.g. mailman Google sign-in), but there's no GCP projects under it yet (in GCP, you create projects under an account).

Is setting up these projects a pretty quick effort (i.e. if it's anything like setting up an API or other project in their spaces, it may not be a big deal)?

It takes about 5 minutes when you have finance sorted (e.g. got a credit card number to put in) to create the actual GCP projects.

If I read this right, the request here is not related to code at all, and just about creating namespaces in a way the team can live with long term. However, I'll wait for @bgilbert to confirm that too.

The hard part here will not be the individual projects, but the overall login flow.
They have documented this pretty well, but it'll take a few steps, and maybe some code writing (since we'll need a way to get users synced).

If, however, we decide to forego the standard auth systems we use for everything else (I'd personally not recommend it, but someone will bring this up), you can be done in about 10 minutes, and you can just grant people's Google accounts access, and it's possible to sync identities and set up the SAML sign-on part afterward.

Thanks all! To answer your questions:

  • Our Google contacts say that publishing to GCP does not require accepting a legal agreement. If anyone discovers otherwise while setting this up, we should stop and investigate.
  • We'd like to start pursuing GCP support sooner rather than later. We expect Fedora CoreOS to go stable in January.
  • Our intent here is not to request code changes; we're just looking for Infra to create and own the namespaces. If y'all would prefer to integrate GCP with the usual login mechanisms, that works for me. In that case, it'd be great if we could get access via temporary credentials, and then switch over to the login integration once it's ready.

Correction to my initial description: we'll need GCS access to upload images to fedora-cloud, but only as a temporary staging area. The images themselves will live entirely in GCE.

On the financial side, our Google contact confirmed that the custom image storage cost is the only relevant piece here; we won't incur bandwidth charges when users access the image. Assuming an inexpensive region like us-central1, 8 GiB/image, and 9 releases/month (6 scheduled + 3 unscheduled), the cost acceleration will be $6.12/month/month, at least until we start garbage-collecting old images. We haven't defined a specific garbage-collection policy yet but some sort of GC seems likely.

However: our contact thinks that Google can supply GCP credits for the storage costs. They're looking into it and will get back to us.

Metadata Update from @kevin:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

4 years ago

Are there any news on this?

Google is still investigating supplying GCP credits for us.

Friendly bump to see if there has been any movement. :smile:

Still waiting for GCP.

Our GCP contacts have asked us to set up a billing account that they can apply credits to. I think doing that might require entering payment information. How difficult would it be to get an account set up?

Thanks @riecatnor! Can you ping me when you've set it up?

@dustymabe I set up an account, pinged on IRC. Let me know what next steps should be

Is there anything form the infra side still needed on this ticket ? or can we close it ?

We are all good now. I have all the access needed to create images in GCP now. The only remaining bit is to set up some sort of "organiziation" within GCP where multiple people (not just me) can manage things. Unfortunately that requires us to agree to another set of "Terms of Service" so it's going to take some time.

For now we are unblocked and there is nothing left for infra to do.

Metadata Update from @dustymabe:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata