I'd like to propose [https://fedoraproject.org/wiki/Category:Obsolete_Test_Cases obsoleting] the following tests from the current [https://fedoraproject.org/wiki/QA:Fedora_13_Install_Test_Plan Fedora 13 install test plan] (and [https://fedoraproject.org/wiki/QA:Fedora_13_Install_Results_Template test matrix]).
I think we can safely remove these tests from the Fedora 13 install matrix without reducing test coverage.
Thoughts/concerns?
Agree.
Do you think if [https://fedoraproject.org/wiki/QA/TestCases/BootMethodsCdrom QA/TestCases/BootMethodsCdrom] is already covered by [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceCdrom QA/TestCases/InstallSourceCdrom], and [https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd QA/TestCases/BootMethodsDvd] covered by [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd QA/TestCases/InstallSourceDvd]?
Replying to [comment:2 rhe]:
Agree. Do you think if [https://fedoraproject.org/wiki/QA/TestCases/BootMethodsCdrom QA/TestCases/BootMethodsCdrom] is already covered by [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceCdrom QA/TestCases/InstallSourceCdrom], and [https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd QA/TestCases/BootMethodsDvd] covered by [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd QA/TestCases/InstallSourceDvd]?
Good question. I'm not fully convinced these tests could be consolidated since I they do target different areas of the installer. For example, the Boot tests are designed to validate that the media is properly constructed, can boot into stage#1 loader. The InstallSource tests focus more on whether the installer is able to load the stage#2 install.img from the specified location. They are similar in that if you boot and install from a DVD, you've tested both scenarios. But it's also possible to boot from {CD,DVD,PXE,USB,grub} and install from {network(http,nfs,ftp,nfsiso),CD,DVD,hd,hdiso}.
The general test strategy for installation test outlines testing the inputs/outputs at each phase of the installer (see https://fedoraproject.org/wiki/QA:Fedora_13_Install_Test_Plan#Test_Strategy). It's definitely a bit odd at first, but given the complexity of the installer, it's proven to be a useful approach. That said, I'm 100% open to devising a new test strategy if there are ideas on improving the workflow/procedure. I'd be hesitant on removing tests that cover different areas until there is a larger workflow/procedure defined to account for the change.
Replying to [comment:3 jlaska]:
Replying to [comment:2 rhe]: Agree. Do you think if [https://fedoraproject.org/wiki/QA/TestCases/BootMethodsCdrom QA/TestCases/BootMethodsCdrom] is already covered by [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceCdrom QA/TestCases/InstallSourceCdrom], and [https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd QA/TestCases/BootMethodsDvd] covered by [https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd QA/TestCases/InstallSourceDvd]? Good question. I'm not fully convinced these tests could be consolidated since I they do target different areas of the installer. For example, the Boot tests are designed to validate that the media is properly constructed, can boot into stage#1 loader. The InstallSource tests focus more on whether the installer is able to load the stage#2 install.img from the specified location. They are similar in that if you boot and install from a DVD, you've tested both scenarios. But it's also possible to boot from {CD,DVD,PXE,USB,grub} and install from {network(http,nfs,ftp,nfsiso),CD,DVD,hd,hdiso}. The general test strategy for installation test outlines testing the inputs/outputs at each phase of the installer (see https://fedoraproject.org/wiki/QA:Fedora_13_Install_Test_Plan#Test_Strategy). It's definitely a bit odd at first, but given the complexity of the installer, it's proven to be a useful approach. That said, I'm 100% open to devising a new test strategy if there are ideas on improving the workflow/procedure. I'd be hesitant on removing tests that cover different areas until there is a larger workflow/procedure defined to account for the change.
I see, the Test Strategy does make sense though a bit odd. :) Thanks.
If no other concerns or thoughts, I'll update the [https://fedoraproject.org/wiki/QA:Fedora_13_Install_Test_Plan Fedora 13 install test plan] (and [https://fedoraproject.org/wiki/QA:Fedora_13_Install_Results_Template test matrix]) to remove the duplicate tests later this week.
Thanks!
please update, no more concerns. :>
After further review, I made a mistake. [https://fedoraproject.org/wiki/QA/TestCases/PartitioningExt4OnNativeDevice QA/TestCases/PartitioningExt4OnNativeDevice] is a unique test in that it specifies installation without LVM. So I've removed the following tests from [https://fedoraproject.org/wiki/QA:Fedora_13_Install_Results_Template test matrix] and moved them into [https://fedoraproject.org/wiki/Category:Obsolete_Test_Cases Category:Obsolete_Test_Cases].
Metadata Update from @adamwill: - Issue untagged with: test review - Issue tagged with: test cases
Login to comment on this ticket.