It became clear in the Fedora 16 cycle that use of Fedora images (both live and non-live) written to USB sticks is now extensive, and our current installation validation tests don't really test this method very comprehensively. We should extend the installation validation matrix to cover at least boot and installation in the following situations:
We may want to consider different l-i-t-d parameters also (especially the use or non-use of --format could be significant). We could also cover use of unetbootin as an informational (non-blocking) test, but I'm not super sure about that.
Part of this ticket is deciding on the best way to represent this - whether it's by simply creating new test cases and adding them into the matrix, or somehow re-rendering the matrix to cover these cases as 'variants' of existing test cases, or some other method.
The changes from this ticket and https://fedorahosted.org/fedora-qa/ticket/256 should be co-ordinated (possibly handled by the same person or group) as they both involve similar questions of how to revise the matrix to cover more situations.
This should be completed by Fedora 17 Alpha TC1.
Let's consider https://fedorahosted.org/fedora-qa/ticket/236 to be merged into this - that's the "Non-live image written to USB stick using dd" case, specifically, ensuring that the non-live images are actually hybridized.
I'm strongly considering to take this ticket, but I don't know anything about EFI, so who of you, gentlemen. will be so kind to cooperate with me on this?
I can help out, but you don't need to know much about EFI to understand the problem space, really: so far as representing the results is concerned, all you need to know is that for some test cases, we'll need to somehow extend the matrices to cover results from EFI installs as well as BIOS installs. I can provide a rough list of the test cases in question. It's pretty much like we have results for both i686 and x86-64 right now: we could simply add a third column to the table labelled 'EFI', but there may be better ways to do it than that. I'm not sure how far we can go just adding columns to the table until we want to look at a different way of representing the data.
Ad USB written images: it would be for the best, if we added a special category for the USB written images, since this seems to be most in line with the current style (as we have categories for DVD/PXE/Live Images etc). I'm still not really sure, about handling EFI - could you provide the list of tests which are EFI-relevant?
Okay, this is my take on the cases we need to consider:
QA:Testcase_Boot_Methods_Boot_Iso QA:Testcase_Boot_Methods_Dvd QA:Testcase_Live_Image_Boot QA:Testcase_install_repository_Mirrorlist_default QA:Testcase_install_repository_DVD_default QA:Testcase_install_repository_Live_Image
These are obviously critical: we really kinda need all possible combinations there. We need to test whether the boot.iso, DVD ISO and live ISOs boot when written to CD/DVD or USB, and in each case, we need to test both BIOS and EFI. In each case, we need to be sure the image boots and a straight-through installation works.
QA:Testcase_Anaconda_autopart_install
For EFI installs we need to make sure anaconda handles the EFI system partition differently (this differs from a non-EFI install). For USB installs we need to make sure the USB stick you're installing from is correctly filtered out: it's not used as an install target or a potential bootloader location. So this test is significant to both.
QA:Testcase_Anaconda_Upgrade_New_Bootloader QA:Testcase_Anaconda_Upgrade_Skip_Bootloader QA:Testcase_Anaconda_Upgrade_Update_Bootloader
Similar to above - upgrades can behave differently with USB and EFI installs, particularly in the area of bootloader management. So we need to test these.
The simplest thing we can do is just add EFI and USB columns to the table, with boxes in those columns only for the tests in question, I guess. Other organization scheme suggestions welcome!
Even though I'm still not sure about the EFI, I believe, that the USB-installs should be special testcases in each category - i.e. testcases "USB stick using livecd-iso-to-disk" and "USB stick using dd" for Boot.iso/Netinst.iso installation & DVD.iso installation; and "USB stick using livecd-iso-to-disk", "USB stick using dd", and "USB stick using liveusb-creator" for the Live.iso installation.
Especially since we need to provide the "how to" part of the testcase for the usb stick creation, and based on the comment#5 we also have different Expected results (e.g. USB stick is not used as install target, etc).
Just one more thing: If we dd dvd.iso to USB stick, should the 'expected results' be that Anaconda uses the repository on the USB stick during installation (as if it was an actual DVD)? Or is this required just for the livecd-iso-to-disk?
AIUI, no, we expect it to work like a boot.iso if you dd it, though there's a parameter you can add to make it use the stick as a repo. If you use livecd-iso-to-disk, though, it should add the parameter automatically so the stick should be used as a source.
So the ticket stalled, but the work got done: Josef wrote up several tests and we added them to the F17 matrix with positive results, F17 is clearly the best-behaving release yet so far as USB writing is concerned. For the record, the tests created are:
Thanks Josef!
Login to comment on this ticket.