#1223 SELinux update issue
Closed None Opened 5 years ago by sundaram.

An SELinux policy bug recently caused major trouble for a lot of users

https://lists.fedoraproject.org/pipermail/announce/2014-January/003196.html

It appears the update was pushed to updates repository with very less time on updates-testing repository.

https://lists.fedoraproject.org/pipermail/devel/2014-January/194103.html

FESCo should mandate that critical path updates must spent a minimum amount of time in updates-testing repository (regardless of karma value) to give time for testers to provide feedback which would have caught this problem. FESCo should also look into any other QA procedures that will avoid similar problems in the future. Thanks!


I am wholeheartedly -1 to changing the process of approving updates to final status for all critical-path updates. This is too broad criteria and in case of security updates it would do more harm than good. The current process is already quite slow for some critical path packages which do not obtain wide testing.

I'm against minimum amount of time in updates-testing. Sometimes we have nasty bug in stable release and pushing fix as fast as possible is more than welcome. We can't fix all problems by policies.

-1 to your proposal

I have to agree with tmraz, so -1 for this proposal.

Nevertheless, it should be mentioned here, that karma threshold can be adjusted (increased) or autokarma disabled. Both can be done by maintainer. Hopefully they consider some of these options next time.

Replying to [comment:4 vondruch]:

I have to agree with tmraz, so -1 for this proposal.
(Please don't use +1/-1 votes on the FESCo tracker if you are not a FESCo member, to make the counting simpler.)

I'm also -1 to this; 3 people saying a package is fine is already a high threshold for ''manual'' testing, and already a significant imposition on our contributors. The suggestion that waiting for a day or two in updates-testing would have probably led to discovery of the problem, while true, is something fairly unique to this case (the policy could just as well have broken something else that isn't regularly exercised by installing updates-testing package and voting on them), and AFAICT fairly unreliable/random (updates-testing pushes are manual, so the next one might have been delayed for whatever reason, and then we still wouldn't have noticed).


That said, while I don't think we should (or, practically, can) impose more ''manual'' testing requirements and expect to get significantly more quality from ''manual'' testing, we should be able to do ''automated'' testing.

We ''know'' that some updates we push will be buggy; hopefully not this buggy, but significantly buggy nevertheless. So, we ''know'' that a 2nd update of the package to fix the 1st update will from time to time be required. If only we could have an automated test that:
Takes the SRPM of a proposed update
Adds one new file, and one new scriptlet (line?), bumps %version, and builds this.
* Tests that update from "original" to "proposed update" and then to "proposed update + 1" works as expected.

Paging Adam for consideration of something like this...

Sure, that is yet another thing we can build when taskotron is done. It was already on our radar, in fact, in the form of testing for bad %preun scripts.

It has been remarked before that, along with work, one thing QA is never short of is ideas for building automated tests =)

We sure hope that once taskotron is ready, folks will help out in contributing them. But right now the work is getting the framework built.

On the topic of the ticket - I'd agree that a mass change is probably inappropriate in this case, though I doubt it'd have much of a downside, the case of a security update we want to get out tooty da sweety is probably the most important. It wouldn't make updates that already go out slow go out any slower, as they'd hit the waiting time just fine. :)

I have suggested to the SELinux devs that they set a higher karma threshold in future, given the popularity of the package, but it's worth noting this was a pretty unusual occurrence. They ship updates very regularly and this is the first time I can recall one breaking this badly. It was the result of a mistake, apparently - some code intended for Rawhide landing on the F20 branch. Their policy for updates is that they should never introduce anything that makes the stock (targeted) policy stricter as an update, only things that loosen it.

For the record, we have recorded the automated test case suggestion in the Phabricator instance which is being used for taskotron project management:

https://phab.qadevel.cloud.fedoraproject.org/T56

I don't have anything to add to this discussion that hasn't already been said, so just to reiterate:
1. A knee-jerk policy change is unwarranted after an isolated incident
1. The selinux team has a strict policy that generally avoids this issue
1. Taskotron improvements should mitigate this in the future.

So I'm firmly -1 to changing policy here.

To complete the vote, I'm -1 to the proposal for the same reasons, and +1 to MOAR TASKOTRON.

I'm -1 as well to changing the policy at this time, although I would note to please be more careful in the future.

I'm -1 to this specific proposal — as others have stated, actual pushing of updates is a manual process that operates on a fairly flexible time frame. So it's unlikely that a time limit will have the desired result. I'm certainly open to other proposals, though, to help limit this sort of problem.

Looks like this proposal didn't garner enough votes to pass.

Login to comment on this ticket.

Metadata