#5 [test idea] Upgrade from package version under test to a future version (scriptlet errors)
Opened 10 years ago by adamwill. Modified 6 years ago

Initially filed as https://fedorahosted.org/autoqa/ticket/447 . Suggested by mitr at https://fedorahosted.org/fesco/ticket/1223#comment:5 .


If only we could have an automated test that:

  • Takes the SRPM of a proposed update
  • Adds one new file, and one new scriptlet (line?), bumps %version, and builds this.
  • Tests that update from "original" to "proposed update" and then to "proposed update + 1" works as expected.

The primary intent of this test would be to catch scriptlet errors: in particular the well-known category of bugs where an error in a %preun script results in a package version that cannot be cleanly uninstalled, which requires manual clean-up by the user. It would also catch other bugs in this general category - //bugs that only show up when updating from the package under test// - like the specific SELinux bug that prompted FESCo's discussion: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=1054350

IMHO this should rank among the first tests we should implement after the work on building the taskotron framework is considered sufficiently 'complete', for reasons I explained at https://lists.fedoraproject.org/pipermail/devel/2014-January/194155.html :


It catches bugs which:

1) cause serious problems that are difficult to recover from
2) we are currently very bad at catching manually
3) would be difficult and onerous to reliably catch manually even with improved manual testing procedures


Metadata Update from @adamwill:
- Issue tagged with: task-idea

6 years ago

Login to comment on this ticket.

Metadata