#261 Revise upgrade test case set
Closed: Fixed None Opened 12 years ago by adamwill.

Our present upgrade test cases are probably a sub-optimal set. We exercise various rarely-used choices - bootloader action, and text upgrade path - but don't exercise more common variations, like installed package sets, install media, and disk layouts. While we can't commit to supporting absolutely any upgrade, we should cover at least some more cases in order to catch major fails.

Based on the experience in F16 and the recommendations in the retrospective, I'd suggest we could consider dropping or downgrading the importance of some tests, perhaps:

  • QA:Testcase_Anaconda_Upgrade_Skip_Bootloader
  • QA:Testcase_Anaconda_Upgrade_Skip_Bootloader_Text_Mode
  • QA:Testcase_Anaconda_Upgrade_Update_Bootloader_Text_Mode

and add test cases for all or some of the following scenarios (please add any other suggestions as comments):

  • upgrade from image written to USB stick (livecd-iso-to-disk)
  • upgrade with KDE (and possibly Xfce and LXDE as 'optional' test cases)
  • upgrade an EFI install (see also https://fedorahosted.org/fedora-qa/ticket/256 for organizational issues here)
  • upgrade with bootloader on first partition (not MBR)
  • upgrade with separate /usr , /home and /var partitions
  • upgrade a RAID install

This should be completed ideally by Fedora 17 Alpha phase, and definitely by Beta phase (when upgrade issues become blockers).


We should discuss the issue of upgrading over more than 1 release (e.g. F14 -> F16 directly). We don't have it in the release criteria, but we have a test case for it. That's not optimal.

The installation guide says a direct upgrade over multiple releases is possible:

http://docs.fedoraproject.org/en-US/Fedora/16/html/Installation_Guide/ch-upgrade-x86.html

But it does not say over how many releases. The best way would probably be to ask FESCo how many releases do we want officialy support to upgrade over. And then QA should reflect this approach in its test cases. Anaconda should also warn when user tries to upgrade over more than the officially supported number of releases.

I would love to see only one officially supported release to upgrade from (only F15->F16), but that's probably not gonna happen. Anyway, some clear decision would be highly appreciated.

The current situation is how it's meant to be - though obviously, since it's possible to interpret it as a problem like you did, we need to clarify things.

It's actually okay to have tests in the matrix which are associated with no release phase. I call these 'advisory tests'. The idea is that there are things we want to test so we catch and file any bugs in them and so we know if they're broken or buggy and can document the problems, but bugs in them are not release blockers. That's fine.

That's what the 'upgrade two releases' test is: it's not a release blocking issue if it fails, but we do want to know whether there are major problems we should document and at least try to fix.

Obviously, we need to make this more clear somehow, so people don't have the same questions about it that you did!

The current 'official' support is exactly what's in the criteria: upgrade from one release ago is release-blocking, upgrade from more than one release ago is not.

I would remove the bit about GDM being the display manager in case someone has KDM.

Replying to [comment:6 jdulaney]:

I would remove the bit about GDM being the display manager in case someone has KDM.

Fixed, however it seems that GDM is pulled when XFCE group is selected. see [1]

[1] http://git.fedorahosted.org/git/?p=comps.git;a=blob_plain;f=comps-f16.xml.in;hb=HEAD

athmane: that's correct. Fedora's Xfce uses GDM as its default login manager, at least presently.

For the record - by default, in Fedora, GNOME uses GDM, KDE uses KDM, Xfce uses GDM, and LXDE uses LXDM.

Adamw, apologies. I was thinking of LXDE by mistake.

Need to get these pesky brain farts under control.

We haven't completed this yet (though thanks to Athmane for the added test cases). We still should. Bumping out to F18.

This more or less got done - we have a rather different set of test cases now. We probably should consider re-re-doing it for the Fedora.next flavor split, so we have tests for upgrading each flavor, but we can have a new ticket for that.

Login to comment on this ticket.

Metadata