You can view the error at https://apps.fedoraproject.org/autocloud/jobs/606/output#227
Related bugzilla bus
We want to mark these issues as non-blocking.
Can the WG members please vote in? (Aye, Nay, Abstain).
First I should vote: Aye (to make them non-blocking).
What is the point of having the testcase if we want to release images even though they have bugs?
This testcase is also not a non-gating test AFAIK. https://github.com/kushaldas/tunirtests/blob/master/cloudtests.py#L61-L98
Replying to [comment:2 trishnag]:
So that we can find and fix bugs faster when they do happen. If we choose to release with the bug anyway then we know it exists and can guide users accordingly until it is fixed.
Does that link prove it is a non-gating test? Where is the list of gating vs non-gating?
I vote aye (+1) to release with this known bug.
Replying to [comment:3 dustymabe]:
Replying to [comment:2 trishnag]: What is the point of having the testcase if we want to release images even though they have bugs? So that we can find and fix bugs faster when they do happen. If we choose to release with the bug anyway then we know it exists and can guide users accordingly until it is fixed. Okay, if we choose to release with the bug anyway does it mean that the bug is not going to affect users much? This testcase is also not a non-gating test AFAIK. https://github.com/kushaldas/tunirtests/blob/master/cloudtests.py#L61-L98 Does that link prove it is a non-gating test? Where is the list of gating vs non-gating? We put non-gating tests here https://github.com/kushaldas/tunirtests/blob/master/nongatingtests.py Rest of the files have gating tests. Journal logging test is a gating test. So we have it in a separate file.
Okay, if we choose to release with the bug anyway does it mean that the bug is not going to affect users much?
Does that link prove it is a non-gating test? Where is the list of gating vs non-gating? We put non-gating tests here https://github.com/kushaldas/tunirtests/blob/master/nongatingtests.py Rest of the files have gating tests. Journal logging test is a gating test. So we have it in a separate file.
Replying to [comment:5 trishnag]:
Replying to [comment:3 dustymabe]: Replying to [comment:2 trishnag]: What is the point of having the testcase if we want to release images even though they have bugs? So that we can find and fix bugs faster when they do happen. If we choose to release with the bug anyway then we know it exists and can guide users accordingly until it is fixed. Okay, if we choose to release with the bug anyway does it mean that the bug is not going to affect users much?
Replying to [comment:2 trishnag]: What is the point of having the testcase if we want to release images even though they have bugs? So that we can find and fix bugs faster when they do happen. If we choose to release with the bug anyway then we know it exists and can guide users accordingly until it is fixed. Okay, if we choose to release with the bug anyway does it mean that the bug is not going to affect users much?
Yeah. The idea is that we put items to a vote and determine the severity. This bug is not that severe as it is just a logging issue. While important, it shouldn't block release in this case IMHO. My opinion could change in the future, but since this has been going out in our images for some time now we aren't making the currently released image any worse by releasing the next image.
This testcase is also not a non-gating test AFAIK. https://github.com/kushaldas/tunirtests/blob/master/cloudtests.py#L61-L98 Does that link prove it is a non-gating test? Where is the list of gating vs non-gating? We put non-gating tests here https://github.com/kushaldas/tunirtests/blob/master/nongatingtests.py Rest of the files have gating tests. Journal logging test is a gating test. So we have it in a separate file.
Thanks
So:
My reason for unblocking these tests is simple:
The version we are currently distributing also fails these tests.
If we had a version on getfedora which passed the tests, and it was only the pending version which was failing, I'd keep them blocking. But I see no point in continuing to block on tests which the "stable" version also doesn't pass. Clearly we've decided it's acceptable to ship a version of Fedora Atomic which fails the log tests.
Replying to [comment:6 dustymabe]:
Replying to [comment:5 trishnag]: Replying to [comment:3 dustymabe]: Replying to [comment:2 trishnag]: What is the point of having the testcase if we want to release images even though they have bugs? So that we can find and fix bugs faster when they do happen. If we choose to release with the bug anyway then we know it exists and can guide users accordingly until it is fixed. Okay, if we choose to release with the bug anyway does it mean that the bug is not going to affect users much? Yeah. The idea is that we put items to a vote and determine the severity. This bug is not that severe as it is just a logging issue. While important, it shouldn't block release in this case IMHO. My opinion could change in the future, but since this has been going out in our images for some time now we aren't making the currently released image any worse by releasing the next image. Got you. Thanks dustymabe. This testcase is also not a non-gating test AFAIK. https://github.com/kushaldas/tunirtests/blob/master/cloudtests.py#L61-L98 Does that link prove it is a non-gating test? Where is the list of gating vs non-gating? We put non-gating tests here https://github.com/kushaldas/tunirtests/blob/master/nongatingtests.py Rest of the files have gating tests. Journal logging test is a gating test. So we have it in a separate file. Thanks
Yeah. The idea is that we put items to a vote and determine the severity. This bug is not that severe as it is just a logging issue. While important, it shouldn't block release in this case IMHO. My opinion could change in the future, but since this has been going out in our images for some time now we aren't making the currently released image any worse by releasing the next image. Got you. Thanks dustymabe.
Replying to [comment:7 jberkus]:
This implies a second decision to be made: once there ''is'' a working version, should the non-blocking test become a blocking one?
My inclination is to say "non-blocking" because I do a lot of testing on Atomic Host and never noticed the bug, personally. I generally figure that gating bugs should be ones which either (a) carry significant security risk or (b) make the system unusable, or significantly less usable, than the prior release.
However, surely the Fedora project has some standards around what's considered a gating bug?
Replying to [comment:10 jberkus]:
We do for the traditional 6-month releases:
https://fedoraproject.org/wiki/Fedora_Release_Criteria
For example, functional system logging is considered to block an alpha release:
https://fedoraproject.org/wiki/Fedora_25_Alpha_Release_Criteria#System_logging
The Atomic release doesn't have to follow the exact same rules, of course, but I'd highly suggest keeping in the ballpark and coordinating with QA for expertise.
Yeah, so then the recommendation would be to unblock until this bug is fixed, but then make it a gating issue thereafter.
Replying to [comment:12 jberkus]:
Agree. I wonder if we can have a concrete release criteria as emphasis on a issue can change quickly.
I don't know if my vote counts but I am +1 (Aye) to mark these as non-blocking.
Yeah, so then the recommendation would be to unblock until this bug is fixed, but then make it a gating issue thereafter. +1 to this.
Done in commit https://infrastructure.fedoraproject.org/cgit/ansible.git/commit/?id=5e5354b86b7be04094ffcd9dfe5b2c98b03d31ea
Worked as a non gating test.
https://apps.fedoraproject.org/autocloud/jobs/634/output#235
@kushal changed the status to Open
Open
Reopeing the ticket to mark the same tests back as gating tests for F26 cycle. Please vote: (Aye, Nay, Abstain)
we should close this ticket - we should do what makes sense and stop bogging ourselves down with "process"
Metadata Update from @dustymabe: - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.