Going through the F16 QA retrospective, there's this series of linked points from Andre Robatino:
"robatino - Alpha and Beta torrents often have unsigned checksum files - this just happened with 16 Alpha. It also happened during F13 and F14 - see http://lists.fedoraproject.org/pipermail/test/2010-April/090106.html for F13 Beta, and https://bugzilla.redhat.com/show_activity.cgi?id=638738 for F14 Alpha and Beta. When it's pointed out, we're told that it's not feasible to change them once people have started downloading, but if this is true, there needs to be better checking (at least as much as for mirror content, which can be changed) to make sure the torrents are correct before being posted. Filed https://fedorahosted.org/fedora-qa/ticket/237 and https://fedorahosted.org/rel-eng/ticket/4906 . No response from releng as of yet. Unless QA is given access prior to public posting, it is powerless to prevent this.
robatino - Also, it's possible that the checksum files can be signed more than once. There are two versions of the signed checksum files for F15, one on the torrents (the original ones) and a different one on the mirrors. See http://robatino.fedorapeople.org/checksums/15-Final/ for the two versions - for example, Fedora-15-i386-CHECKSUM.orig and Fedora-15-i386-CHECKSUM. While not nearly as serious as the above issue (since the checksums are the same and both signatures are good), it could be confusing. "
A quick mail discussion prompted the suggestion from Andre that the best way to cover all these issues would be for there to be a Deliverables wiki page / SOP which clearly lists all the 'bits' that should be delivered for each image drop - Alpha, Beta, Final, TC and RC. This would provide a simple reference for whoever's doing the build to make sure everything has been provided.
It would make most sense for this to live in releng's wiki space, I think, so filing against releng trac, but a QA person could certainly take care of creating the page in theory, I think.
We should have this in place prior to F17 Alpha TC1.
Apologies - the suggestion was from Kevin Fenzi, not Andre.
this does not seem to be a koji-related ticket
PGM has begun publishing release deliverables  but the list that's maintained is not kept up to date and does not cover the concerns outlined here. Because the list is not well maintained it's not really used by many stakeholders. We need to identify an owner for this list and enable that owner to keep it updated in a sane way. The list should be systematically queryable by release and QE tooling at least so we can validate expectations. A likely candidate for storing this data is PDC.
Jan, as noted above, we do have a doc but it's not updated and not enforced as part of the release process which means we have no canonical source of what we're shipping for a release. We'd like for this to be a gating list with a defined process around it. Ideally this would be systematically queryable in a tool like PDC eventually.
I brought this topic to the FESCo  for their last meeting . There was no decision made and the topic has been re-scheduled for the meeting on 2016-Apr-22 due to pending details from rel-eng team.
 https://fedorahosted.org/fesco/ticket/1566 [[br]]
hah. this ticket amuses me greatly, since I've been going round for the last week saying what a dumb idea it was to keep this list in a wiki page. guess which idiot's dumb idea it was. ;)
There was quite extensive discussion about this release deliverables list on FESCo meetings during F23 live cycle. The current format presented on  is the format FESCo finally agreed on. There was also agreement on the way how this list can be changed, which is via Change Process, so FESCo has full control on it. As far as I know the list  is currently up to date.
Using PDC instead of this wiki page will be IMO welcomed by many parties. I am going to point to this ticket during the FESCo meeting on 2016-Apr-22, so we can decide how we would like to track this.
We do have some discussion ongoing around this in pungi tickets, ref. https://pagure.io/pungi/issue/249 and https://pagure.io/pungi/pull-request/269 .
I understand from DGilmore that there are in fact things that we're shipping that aren't on that list (see today's flurry around Live USB amongst others). Jan, please just keep us updated on the discussion and if you want to propose how we should handle this in the future, I think we'd all be on board to make it happen :)
On PDC, two things:
Another idea: since we keep our pungi config public here https://pagure.io/pungi-fedora/raw/master/f/fedora.conf and since it actually defines what we will try to produce, we could build separate tooling around it that reads it in and parses it to produce a list of expected outputs. (And we could go back in the git history to figure out what we intended to produce at some date in the past.)
pungi does not actually have everything. it does not define where the content ships, or what is to be shipped as a torrent etc. we need to have both the what we build and the where and how we ship it.
The list of Release blocking deliverables  has been updated to my best knowledge what deliverables are requested for F24. Tickets  were created or emails sent to WGs requesting review of the list.
I would appreciate if anyone from Release Engineering can do review of the page as well and let me know in case there will be any discrepancies.
 https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora24 [[BR]]
 [https://fedorahosted.org/cloud/ticket/157 Cloud WG][[BR]]
 [https://fedorahosted.org/workstation/ticket/9 Workstation WG][[BR]]
 [https://email@example.com/thread/OEBK2YL6SYVDIWO54TWC66LICNKZHPTY/ Server WG][[BR]]
 [https://firstname.lastname@example.org/thread/JI2HQT7JVV5MYNFVMK3T3L7BHOK6WSVH/ Spins SIG][[BR]]
 [https://email@example.com/thread/VMBQT3HO2R2NO6MMK4SZ4XVGH2F2JGQ4/ Base WG][[BR]]
we do not have a list of everything for fedora 26 afaik. The only reasonable solution here is to define the product in pdc and have poungi talk to it to make the compose configs. @ralph who can we talk to in PDC land about fully defining things in PDC?
Metadata Update from @ausil:
- Issue close_status updated to: None
We have List of Fedora 26 release deliverables for the F26 release. However, I am missing feedbacks from some teams, so I would not 100% bet on reliability of this source :(
I will try to find out who can help us to fill PDC with data and let you know.
Needs to be in PDC to help maintainability.
Metadata Update from @katec:
- Issue assigned to kellin
to comment on this ticket.
Copyright © 2014-2017 Red Hat
2.14.2 — Documentation