#138 Produce updated cloud base images monthly
Opened 4 years ago by dustymabe. Modified 2 years ago

An extension of https://fedorahosted.org/cloud/ticket/94

We have elected to release cloud base images monthly for the "current" release of Fedora. The releng infra is in place to do this already, we just need appropriate testing and process around it to release new images and verify they aren't broken.

This ticket will track progress/status of this.

You have rel-eng and QA buy in for this?

I'm especially interested in the QA side of things given the failures in F-23 that I've seen no follow up from in the retrospective ticket

I've talked with dgilmore about it. The images are being built already. As far as QA buy-in the only contact from there that I really interface with is roshi, who has been a part of the discussions in the past. Should we formally reach out to adamw or someone else to have a conversation around this?

I have several concerns here that I suspect will be echoed by the rest of QA:

  • Who's signing up to make sure that enough testing gets done? This can't be just roshi

  • Assuming that the testing follows a process similar to the releases, what format are the test matrices goign to take on the wiki? How should they be named?

Bringing the topic up at the next QA meeting would be a good way to approach getting QA buy-in.

I'll be at the meeting on Monday at 16:00 UTC. If we can't get QA buy in then we won't do this.


The QA meeting was canceled today.

Last time when we discussed about this we came to a decision about testing the updated images by Cloud WG itself as QA is already overloaded. I will have to find that IRC log (must be somewhere).

In the Fedora 22 cycle I had created one ticket [1] for the update, but may be it was lost.

[1] https://fedorahosted.org/rel-eng/ticket/6219

As decided in the meeting on 2015-01-06 we will release an update cloud base image this month (hopefully by next week). I will discuss with rel-eng and update the list with the details.

Some information has come to light about the release process around the 2 week atomic image releases. Adam has decided to scrap the existing release process and write it from scratch.

Considering the existing release process is "duct tape and band-aids" we are going to scrap the idea of releasing updated cloud images until the F24 release.

On a side note: Does the recent uncovering of the glibc cve affect our decision that we made not to do a one-off release of Fedora cloud (half way between F23 and F24)?

"''On a side note: Does the recent uncovering of the glibc cve affect our decision that we made not to do a one-off release of Fedora cloud (half way between F23 and F24)?''"

I'd really prefer we stick to the one-off release, and perhaps accelerate it.

I will note that the process that makes the nightly atomic composes also makes all the cloud images. They are there, we just need to release them.

Replying to [comment:10 ausil]:

I will note that the process that makes the nightly atomic composes also makes all the cloud images. They are there, we just need to release them.

Yep. You are right. I brought this up but from what I understand the actual process to get the images into the right download locations and updated on the website was more of a pain than it is worth. I believe Adam is the person who has the most information about this (at least when we were having the discussion).

The discussion we had took place before the glibc vulnerability so it may now be worth the extra work.

We are not doing this till F24 release. Mean while we have to find out the exact steps required to get the updated image out as a release.

A few users have been requesting this as of late: [mail thread] (https://lists.fedoraproject.org/archives/list/cloud@lists.fedoraproject.org/message/IADTY4BL3M3YBL7QKOTYNRR54DOP6XHB/)

@mattdm had some questions in this mail copying them here:

So, we are already making these; you can see them at
https://apps.fedoraproject.org/autocloud/compose — they're currently
under a compose ID with "Atomic" in the name, but AIUI that's going to
be split out. (I was worried that these might not include updates, but
that's fixed.)

Let's make a list of what needs to happen... here's what I can think

  1. Decide if we are okay with the level of automated testing these are
    getting, or if we need human testing, or if we need more automated

  2. If we need more automated tests, someone needs to sign up for that

  3. If we need human testing, someone needs to sign up to write the
    release criteria

  4. And, someone would need to commit to doing the validation every

  5. Work with release engineering and infrastructure to adapt the Atomic
    Host gating/release process for Cloud Base

  6. Work with release engineering to get updated images to mirrors and

  7. Someone needs to sign up to update the Vagrant Atlas index

  8. Work with websites to update cloud.fedoraproject.org to also offer
    updated images. (I'm thinking the page should default to the latest,
    but there should be some way to "scroll back" all the way to the GA

  9. Decide if we want to switch Cloud Base entirely to this
    automatic process and eschew release milestones

I'm sure I'm missing something -- what else should be added?

Howto proceed here?

I'm fine with the automated testing. Manual testing maybe needed when kickstarts or release changes.

Metadata Update from @dustymabe:
- Issue assigned to kushal
- Issue tagged with: none

2 years ago

@jdoss @deuscapturus - I finally migrated that ticket you were asking about - here it is :)

Awesome! OK so were do we start? Do we have a process documented that we can follow to update images and get them in the right spot?

@jdoss - the bad news is that it's unfortunately quite complicated. the good news is that releng has quite a few tools for automating things and we just need to put magic pixie dust in a few places and a lot of stuff will happen for us.

I think we should schedule some time to talk about this soon so we can cover some of the bases, it might not happen before f26 gets out the door, though.

an update, we have seperate composes for cloud images now, I updated the script that does them to keep only 3 weeks worth of composes. I would like to figure out a way to only build the images only when something in them changes. we just need to determine at this point in time how we decide what to ship, and where we will put the images.

I like the idea of building only when something changes, although as Dusty pointed out to me, that's practically a moot point for monthly updated images, as there's always going to be something that's changed in that time.

For "where", is /pub/fedora/linux/updates/##/CloudImages in the main mirror out of the question? Failing that, /pub/alt/CloudImages?

For "decide to ship", it looks like the consensus above is that automated testing is considered sufficient. Can someone define specifically what that is to be?

@mattdm today we blindly build new cloud images every day. if there was no updates in the last day we should not build them. but over a month, I would be shocked if we did not have changes.

I think /pub/fedora/linux/updates/##/CloudImages/ should be fine. but it may be that /pub/alt/updates/##/CloudImages/ is more appropriate due to less usage

What is the status here? The docker images at https://store.docker.com/images/fedora/plans/6c3702c4-7bd1-45c2-8a91-2df208e58517?tab=tags seem to be older than a month.

Docker images are handled separately. You should ask @mohanboddu and @cverna about them.

A change request has been made for this issue.

Login to comment on this ticket.