#167 Decide a process about failed tests for Autocloud
Closed None Opened 2 years ago by kushal.

If we have a test failing the compose (right test, catching a real bug), should we stop the release till the time the bug gets resolved?

My personal opinion is yes, first fix, and then only go ahead.

For example [https://apps.fedoraproject.org/autocloud/jobs/353/output#283 this test] fails time to time on the vagrant virtualbox image.


I think for this ticket it would be nice to stop the release for a failing test but we must make sure we have resources to fix issues when they arise otherwise we'll just block release forever.

Right now we don't have enough resources to guarantee an issue will gain enough attention to get fixed in a timely manner. Depending on the bug we should be able to decide if it is bad enough to block the release or not?

Replying to [comment:1 dustymabe]:

I think for this ticket it would be nice to stop the release for a failing test but we must make sure we have resources to fix issues when they arise otherwise we'll just block release forever.

+1 to the above. My only concern is to get things fixed in time, say within a week of the first reporting the issue (for our 2 week release cycle).

Right now we don't have enough resources to guarantee an issue will gain enough attention to get fixed in a timely manner. Depending on the bug we should be able to decide if it is bad enough to block the release or not?

This also sounds good, should we discuss this over the issue bugyou creates for the failed test runs? Or over mailing list etc.

So one of the major problems here is that when a blocker is raised, nobody responds to it until the weekly meeting. Adam already has plans to send a notification to the fedora-cloud mailing list. I think it would also be very helpful to post a notification on IRC somehow, as the fedora-cloud team is very IRC-centric. How hard is it to pull the list of blocker bugs via an API?

Replying to [comment:3 jberkus]:

So one of the major problems here is that when a blocker is raised, nobody responds to it until the weekly meeting. Adam already has plans to send a notification to the fedora-cloud mailing list. I think it would also be very helpful to post a notification on IRC somehow, as the fedora-cloud team is very IRC-centric. How hard is it to pull the list of blocker bugs via an API?

AutoCloud (via Bugyou) puts issues at https://pagure.io/atomic-images/issues. And nirik tells me it is possible to have a bot direct those to a channel.

It was decided that we will be marking any such bug as non-blocker after discussing it over mailing list, or in a ticket. As an example, we have marked the journal tests as non-blocking for now. We will bring it back into blocking after it is fixed.

I am also closing this ticket with this. We can reopen it as required.

Login to comment on this ticket.

Metadata