#1427 List of release blocking deliverables
Closed None Opened 4 years ago by jreznik.

At yesterday's Go/No-Go meeting we hit an issue with unclear status of some release blocking deliverables - Atomic in this case - Cloud WG decided it's not a release blocking but it wasn't properly communicated to release engineering.

Proposal: for each Fedora release, the list of release blocking deliverables has to be created and accepted prior Alpha Freeze by FESCo, Fedora release engineering and Fedora QA (in cooperation with WGs and SIGs).

  • QA knows what's release blocking and they know where testing priority is
  • release engineering knows what's going to be produced for upcoming release ahead of time (we added this point to the Change process as releng requested it)
  • we can avoid last minute juggling with release at Go/No-Go meeting

Fedora Program Manager can be responsible for this process as an addition to the Change process.


A few minor points of procedure:
* Blocking deliverables must be by unanimous vote by a representative each of FESCo, QA and Release Engineering. (How each of these groups decides to appoint this representative is up to them; I'd suggest that FESCo will engage the related WGs and use that to come up with their vote). If any of these three groups votes nay, the matter is closed until the next Fedora release.
* This list may be reduced but never added to after Alpha Freeze. This is so we reserve the right to remove a medium from release-blocking if it looks like it will be unfeasible to deliver it without significant slippage. Once removed from the blocking list, it cannot be re-added until the next Fedora release (even if it was dropped for Alpha but looks like it could be done for Beta; at that point it can still be delivered if it's ready, but it cannot affect the schedule if it's not)

IMHO, Working groups should decide what exact deliverables they feel are blocking for their area, and FESCo can manage everything thats not directly under a Working Group (spins, etc).

A wiki page for each release (or the like) should be used showing a list of exactly these deliverables.

I'm not sure I like the idea of removing release blocking deliverables during a release. That seems like it defeats the purpose.

Replying to [comment:3 kevin]:

IMHO, Working groups should decide what exact deliverables they feel are blocking for their area, and FESCo can manage everything thats not directly under a Working Group (spins, etc).

Well, as long as it still requires rel-eng and QA to agree with them, I'm fine with that. (The important thing here is that the WGs or FESCo can't create more work for those teams without their input).

I'm not sure I like the idea of removing release blocking deliverables during a release. That seems like it defeats the purpose.

Well, there will be times when this is necessary, but I agree it should be considered extremely rare. The more important aspect of this was that if such a thing happened, it can't be for just one milestone. If we make the decision to drop it, it can't come back until the next release.

well, since thats happened so far... never (That I can recall). I'd say leave it out of policy. If we get to that kind of state sometime, we can just have FESCo consider what to do then.

Replying to [comment:1 sgallagh]:

A few minor points of procedure:
* Blocking deliverables must be by unanimous vote by a representative each of FESCo, QA and Release Engineering. (How each of these groups decides to appoint this representative is up to them; I'd suggest that FESCo will engage the related WGs and use that to come up with their vote). If any of these three groups votes nay, the matter is closed until the next Fedora release.

By "closed" you mean "it will not be blocking for the release but can be delivered if ready on time"?

Replying to [comment:5 kevin]:

well, since thats happened so far... never (That I can recall). I'd say leave it out of policy. If we get to that kind of state sometime, we can just have FESCo consider what to do then.

Didn't we do exactly that during Fedora 21 to get the Alpha out? I seem to recall that at least some of the media was skipped and reintroduced in Beta. It made sense during the re-tool, but in the future we should probably avoid it. But I suppose we can leave it out and address it if it comes up.

Replying to [comment:6 thozza]:

By "closed" you mean "it will not be blocking for the release but can be delivered if ready on time"?

Yes, sorry. That was ambiguous. It can be delivered if the groups responsible manage it themselves, but if they miss the date, too bad.

+1 to the original proposal.

As for minor points of procedure, not sure we need to be stuck making them perfect… But FWIW I wouldn‘t bother with setting up specialized representatives: just let FESCo vote in a standard ticket / during the usual meeting, and presumably something similar for the other groups. When any “nay” is definitive, the three groups don’t really need to coordinate/negotiate, so delegating representatives doesn’t seem so necessary.

Replying to [comment:8 mitr]:

+1 to the original proposal.

As for minor points of procedure, not sure we need to be stuck making them perfect… But FWIW I wouldn‘t bother with setting up specialized representatives: just let FESCo vote in a standard ticket / during the usual meeting, and presumably something similar for the other groups. When any “nay” is definitive, the three groups don’t really need to coordinate/negotiate, so delegating representatives doesn’t seem so necessary.

Well, a nay from a group is not necessarily the same thing as a nay from any member of that group. It's reasonable that a FESCo majority vote could be equivalent to "FESCo's position on the matter".

Replying to [comment:3 kevin]:

IMHO, Working groups should decide what exact deliverables they feel are blocking for their area, and FESCo can manage everything thats not directly under a Working Group (spins, etc).

FESCo/releng/QA stamps the final list, do not create it. In case of objections raised, it has to be communicated with related WG and talk to them, why they think it should be on list, if it's doable within schedule etc. I definitely don't want overrule WGs!

Thinking more about it, as you mentioned spins, maybe it should be list of all approved deliverables for the release with release blocking flag (yes/no). Aka neverending story with spins readiness. And we have so many deliverables these days... Maybe even wiki is not a good tool for it but for the beginning.

Btw. I do like Stephen's second paragraph about removal.

We agreed to have a list of release deliverables for F23 at least two weeks before the Alpha Freeze with release blocking deliverables marked.

jreznik volunteered to start writing things up and nirik will help him.

Hi.

Do we have any update on this?

jreznik: were you going to do this?

Or was jkurik going to take over this as program manager?

This is unfortunate :-([[BR]]
I was not aware of this task and as I am leaving for holidays tomorrow for upcoming next two weeks I will not be able to deliver this for Alpha release.
I am proposing to defer this till Beta freeze unless nirik&jreznik can work on this earlier.

Nirik, Jaroslav ?

It can not wait until Beta Release.

I'll make sure we will have the list for Alpha.

I don't want to press this too hard, but just as a note, my 'fedfind' thing by necessity has to define some concepts for thinking about the properties of deliverables. It might be good for the list to work with reference to that, or I guess what I want is for it not to conceive of things in such a radically different way that we wind up with conceptual conflicts. We also have the image naming policy that I more or less wrote and releng accepted (but isn't entirely implemented yet). See:

In a really ideal world, it'd be cool if the deliverables list existed in some sort of machine-readable form (json?) so things like fedfind could consume it for purposes like 'list all release-blocking deliverables for release X', and we could build things to check for missing deliverables on top of that.

Replying to [comment:20 adamwill]:

In a really ideal world, it'd be cool if the deliverables list existed in some sort of machine-readable form (json?) so things like fedfind could consume it for purposes like 'list all release-blocking deliverables for release X', and we could build things to check for missing deliverables on top of that.

Yes, I very much agree! See https://lists.fedoraproject.org/pipermail/devel/2015-May/210705.html for discussion. Candidates include:

  • "Simplestreams" format as used by Ubuntu
  • virt-builder's index.asc format
  • appc image discovery scheme

I guess also "Invent our own new thing!" is always a possibility too. :)

That's a nice idea to have it in machine readable format - first, let's take a look on the structure and what we would like to cover. I started with https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora23 - initial copy of our release directories structure (Workstation/Server for now, I'm a bit lost in Cloud space but I'll try ;-). If you can please take a look on the structure, add more/less details, reorganize - this is the first draft, it would be great. We can try to fill some date in and then let WGs/SIGs/individuals to confirm correctness and get it stamped by FESCo.

Other option is https://github.com/release-engineering/product-definition-center - it's developed internally by Red Hat but open sourced to allow Fedora community adoption. And it might be possible to get RFEs fixed etc. - as far as I understand, it goes beyond just deliverables but... I don't have more information but I can reach folks behind it. It would be nice to share same infra and even some definitions with our downstreams.

Replying to [comment:22 jreznik]:

Other option is https://github.com/release-engineering/product-definition-center - it's developed internally by Red Hat but open sourced to allow Fedora community adoption. And it might be possible to get RFEs fixed etc. - as far as I understand, it goes beyond just deliverables but... I don't have more information but I can reach folks behind it. It would be nice to share same infra and even some definitions with our downstreams.

Wow, that sure could benefit from some documentation :)

adding meeting keyword

18:11:50 <nirik> jreznik_pp: lets leave the ticket open and you can update it when ready and we can review. :)

Let's wait here for updates and accordingly we can discuss in the next meeting.

Update:
[https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora23 ReleaseBlocking-Fedora23]
Server WG deliverables - Agreed & complete
Cloud WG deliverables - to be agreed [https://fedorahosted.org/cloud/ticket/118 Cloud #118]
Spins deliverables - updated but not sure whether it is final
Workstation WG deliverables - we have probably all deliverables listed, [https://lists.fedoraproject.org/pipermail/desktop/2015-September/012786.html waiting for a confirmation]
* Ambasadors deliverables - no response; need to chase it

jkurik is going to send the list to rel-eng, QA, and devel-announce.

Next steps are:

  • Machine readable format
  • Collect the F24 list
  • Figure out how atomic plays into this

FYI, I found the current page confusing, so I made:

https://fedoraproject.org/wiki/User:Kevin/Fedora23_release_deliverables

That I think is somewhat better... feedback welcome.

Thanks Kevin, for me it is definitely better and more useful format.
Let's open this on FESCo meeting. If FESCo approves, I will then drop the old page and redirect to this one.

I don't think this really needs FESCo approval, but even if it does then lets try and just get it done in the ticket. There's no need to wait for a meeting.

I'm +1 for the changes Kevin did and Jan supports.

(And I'll note that the filled in F24 page would drop the i686 images.)

+1 to Kevin's changes.

That said, I'd also like to recommend that we make this page restricted-access to FESCo+FPM and that any changes made to it have to be validated by FESCo (so we can avoid the mess we had at Go/No-Go where there was confusion about whether 32-bit Cloud Images were blocking.

I am agree with jwboyer and sgallagh what they said in above comments.
Therefore, let's not wait more here and approve Kevin's changes, +1.

Replying to [comment:35 sgallagh]:

+1 to Kevin's changes.

That said, I'd also like to recommend that we make this page restricted-access to FESCo+FPM and that any changes made to it have to be validated by FESCo (so we can avoid the mess we had at Go/No-Go where there was confusion about whether 32-bit Cloud Images were blocking.

How is restricting access to FESCo+FPM going to avoid the confusion? That seems like something that stems from a WG wanting changes to it's image set and FESCo/FPM not understanding what those changes are. If the WG can't edit the page, you're still going to have confusion if communication isn't clear before hand.

I'm not really opposed to restricted access, but I don't see it solving the problem you highlight.

The problem we had was that at Go/No-Go, we were confused about what images were blocking because the Cloud WG had just gone and added 32-bit Cloud images as blocking without anyone else knowing.

I'm proposing we restrict the access as a simple technical mechanism for ensuring that changes to that page have passed through the people who need to know about them.

While it's technically feasable to lock down permissions on a page, it means we would need to create a "FESCo" namespace and maintain a list in the wiki of fesco members.

I don't think it's worth it. We have history to see who edited what when.

Correction: ARM minimal disk image is release blocking. Per https://meetbot-raw.fedoraproject.org/teams/fedora-qa/fedora-qa.2013-08-12-14.31.log.txt :

"14:49:21 <adamw> #agreed release blocking image set for ARM will consist of minimal and KDE images for F20"

we have since switched from KDE to Xfce, but the agreement that minimal is release blocking has never been changed.

  • AGREED: : we will use nirik's wiki page until we can egt something
    in a better format that can be used to visualise and by tools to
    verify a compose (dgilmore, 18:21:48)

I've folded my changes into:

https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora23

I've also made a prelim:
https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora24

(I set the i686 stuff to not be release blocking there, but it wasn't clear to me if we were even going to make the i686 images anymore or not. Also, we should ask the kde sig about their i686 blocking media)

Login to comment on this ticket.

Metadata