https://fedoraproject.org/wiki/QA:Testcase_Preupgrade
This test should include having 3 kernels installed in /boot to more accurately reflect what real-world users will be starting from. The test now will not sufficiently model how much free space is available on the average users /boot partition. The test should be updated to reflect this.
Hurry ... do you think you might be able to assist with proposing some updates to the existing preupgrade tests?
Hi James, I tested the latest package, if the space of /boot is not enough, a window would pop up saying 'we recommend you free up 1.1MB by deleting old kernel..'
So, I wanted to include these in test case: 1. The total free space of /boot needed for preupgrade(but not essential). (Is it possible to estimate?) 2. Tell that the window would pop up if the space is not enough, and methods to find enough space: a.the removal of all kernels that are older than the currently running kernel. b.tune2fs -r 0
I also don't wanna make a test case tooo long.
Thanks. What do you think?
For Fedora 12, I think the default /boot size is 250MB (needs to be verified). So that can in updated in the test case.
Also the test case can mention that two kernels are easy to install from repositories (stable and updates repo). The third (and others) can be downloaded and installed from Koji:[[BR]] http://koji.fedoraproject.org/koji/packageinfo?packageID=8
Replying to [comment:2 rhe]:
Is this window from preupgrade or from the anaconda installer? I think what we want is for preupgrade to correctly detect anaconda's /boot needs. The situation to avoid is when preupgrade thinks everything is okay ... but anaconda fails as there isn't sufficient disk space available.
I wouldn't worry about documenting the workarounds for getting more space in ''/boot'' in the test case. I think it's safe to leave that documented at http://fedoraproject.org/wiki/How_to_use_PreUpgrade#Troubleshooting.
I also don't wanna make a test case tooo long. Thanks. What do you think?
The big thing I wanted to update was to make sure that the test case was written to properly capture the real-world upgrade case. Specifically, for users of F-11, they're likely to have multiple kernels installed on the system since they installed. The default maximum, as specified by yum.conf is 3 (''installonly_limit=3''). As Kamil points out, I think we just need to make sure the test has 3 total kernels installed, before testing.
Replying to [comment:3 kparal]:
For Fedora 12, I think the default /boot size is 250MB (needs to be verified). So that can in updated in the test case. <skip>
RC4 is still 200MB for default /boot.
Thank James and Kparal, I've updated the test case, feel free for modification. https://fedoraproject.org/wiki/QA:Testcase_Preupgrade
Hurry is doing all the work on this one, so assigning to rhe
Replying to [comment:6 rhe]:
Looks good Hurry. I've added some instructions on how to use the ''koji'' command to locate and download applicable kernels. If you think that's too much, feel free to remove.
I've made a few other minor wording changes, again feel free to correct/revert.
I like step#8. The only think I don't know is whether step#8 represents a failure case or not. But I think as you've written it, it should be good to push out to fedora-test-list for review.
Can you carry forward your changes to the other preupgrade test also (https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release)?
Thanks, James
Wow,the case looks much better now. I've forward the changes to https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release.Thanks.
I have just tried default F12 install, it still creates only 200MB for /boot :( So it should be probably corrected in the test case. Chris' changes (https://bugzilla.redhat.com/show_bug.cgi?id=534055#c7) will probably be effective only at F13.
Replying to [comment:10 kparal]:
Right, I have to change the case back to 200MB. Thanks.
I have just used preupgrade on my bare metal (unfortunately the fixed preupgrade wasn't in updates yet). It is quite probably that we will hit preupgrade problems again, therefore we should pay more attention to it. I have a few remarks:
My old kernel (from F11) didn't stay installed. Should it stay installed? In that case the test case should mention that.
Do we have a bug number which relates to the need of using "tune2fs -m0 device"? Because that certainly should not be needed - the upgrade process must be running like root.
I propose the test case should contain this requirement:
3a) If only one current kernel is installed, the update process must not complain about insufficient space. It must not be required to delete some files (efi/, grub/splash.xmp.gz), nor to tune the filesystem (tune2fs). If that happens, it's a fault of default partitioning (/boot created too small). A workaround should be present (at least a descriptions of steps needed to take to solve that issue).
3b) When more kernels are installed and the free space is not enough, user should be offered to remove all but most recent kernel. Removing kernels is a delicate matter and all users may probably not be able to do that on their own, so the best solution would be a dialog with just Next->Next->Finish interface. I don't know if current preupgrade offers this feature, but it would be certainly desired. We should create a ticket for that in that case.
These requirements should help us catch the problems next time and call for fixes in advance if needed.
Replying to [comment:12 kparal]:
I've filed ticket#31 to track investigation of this effort. Anyone interested in owning and following up in that ticket?
Good catch. We have [https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=530541 bug#530541] which tracks the work arounds in place for F-12. Will Woods is close to this issue, I've asked for his thoughts on if there is anything to track the discussed phase#2 scripting of the current workarounds.
I propose the test case should contain this requirement: 3a) If only one current kernel is installed, the update process must not complain about insufficient space. It must not be required to delete some files (efi/, grub/splash.xmp.gz), nor to tune the filesystem (tune2fs). If that happens, it's a fault of default partitioning (/boot created too small). A workaround should be present (at least a descriptions of steps needed to take to solve that issue).
I agree, but caution that our test cases should be written to how preupgrade is intended to work, not what we believe the user experience should be. It would be nice for it to be more seemless, but that support is not presently included in preupgrade. Until it is, I wouldn't feel comfortable with our test cases being based on that assumption.
Here's what I would recommend to meet your proposal ... 1. Modify [https://fedoraproject.org/wiki/QA:Testcase_Preupgrade QA:Testcase_Preupgrade] and [https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release QA:Testcase_Preupgrade_from_older_release] so that the setup instructions have only 1 kernel installed prior to running preupgrade. Also, adjust the expected results so that '''no''' low disk-space dialogs present. If any low-disk space dialogs appear, the test can ''note'' that manual workarounds are available (see http://fedoraproject.org/wiki/How_to_use_PreUpgrade#Not_enough_space_in_.2Fboot), but the need for manual workarounds would be considered a test result of FAIL. 1. Create a new test case (perhaps [https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_low_/boot_disk_space QA:Testcase_Preupgrade_low_/boot_disk_space]). This test case intentionally will craft a /boot filesystem with insufficient free space, and ensure that preupgrade responds as expected to this scenario.
3b) When more kernels are installed and the free space is not enough, user should be offered to remove all but most recent kernel. Removing kernels is a delicate matter and all users may probably not be able to do that on their own, so the best solution would be a dialog with just Next->Next->Finish interface. I don't know if current preupgrade offers this feature, but it would be certainly desired. We should create a ticket for that in that case. These requirements should help us catch the problems next time and call for fixes in advance if needed.
Let's create bugs against preupgrade for any requests for changes in behavior. For this ticket, I want to make sure that our preupgrade tests correctly reflect the current expectations for using preupgrade.
Replying to [comment:13 jlaska]:
Here's what I would recommend to meet your proposal ... 1. Modify [https://fedoraproject.org/wiki/QA:Testcase_Preupgrade QA:Testcase_Preupgrade] and [https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release QA:Testcase_Preupgrade_from_older_release] so that the setup instructions have only 1 kernel installed prior to running preupgrade. Also, adjust the expected results so that '''no''' low disk-space dialogs present. If any low-disk space dialogs appear, the test can ''note'' that manual workarounds are available (see http://fedoraproject.org/wiki/How_to_use_PreUpgrade#Not_enough_space_in_.2Fboot), but the need for manual workarounds would be considered a test result of FAIL. Create a new test case (perhaps [https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_low_/boot_disk_space QA:Testcase_Preupgrade_low_/boot_disk_space]). This test case intentionally will craft a /boot filesystem with insufficient free space, and ensure that preupgrade responds as expected to this scenario.
Here's what I would recommend to meet your proposal ... 1. Modify [https://fedoraproject.org/wiki/QA:Testcase_Preupgrade QA:Testcase_Preupgrade] and [https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release QA:Testcase_Preupgrade_from_older_release] so that the setup instructions have only 1 kernel installed prior to running preupgrade. Also, adjust the expected results so that '''no''' low disk-space dialogs present. If any low-disk space dialogs appear, the test can ''note'' that manual workarounds are available (see http://fedoraproject.org/wiki/How_to_use_PreUpgrade#Not_enough_space_in_.2Fboot), but the need for manual workarounds would be considered a test result of FAIL.
These make sense. Would change it soon.
I've updated the cases. For the low /boot space case, the method I wrote is to install another kernel to make free space insufficient. An easier method is to create a file with certain size using dd and put the file in /boot. How do you feel the methods or do I get it wrong?
1.https://fedoraproject.org/wiki/QA:Testcase_Preupgrade 2.https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_low_/boot_disk_space 3.https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release 4.https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release_low_/boot_disk_space
Replying to [comment:15 rhe]:
On closer inspection, it seems there are several low disk-space conditions we may wish to trigger. 1. Not enough ''/boot'' space for preupgrade to download the installer kernel+initrd.img 2. Not enough ''/boot'' space for preupgrade to download the install.img (see https://fedoraproject.org/wiki/How_to_use_PreUpgrade#Method_2:_Trick_preupgrade_into_downloading_the_installer) 3. Not enough ''/boot'' space for anaconda to install a new kernel
As written, I think the low disk-space tests targets #3 above. I believe that should be sufficient for testing the real-world use case. I've made some minor adjustments to the wording of the following test cases. I think they look good.
I think we can focus our low disk-space tests on just upgrading from the previous release (not from the previous previous release). I'd be okay with not adding the following test to our matrix. Do you agree or have concerns?
Replying to [comment:16 jlaska]:
Replying to [comment:15 rhe]: I've updated the cases. For the low /boot space case, the method I wrote is to install another kernel to make free space insufficient. An easier method is to create a file with certain size using dd and put the file in /boot. How do you feel the methods or do I get it wrong? On closer inspection, it seems there are several low disk-space conditions we may wish to trigger. 1. Not enough ''/boot'' space for preupgrade to download the installer kernel+initrd.img 2. Not enough ''/boot'' space for preupgrade to download the install.img (see https://fedoraproject.org/wiki/How_to_use_PreUpgrade#Method_2:_Trick_preupgrade_into_downloading_the_installer) 3. Not enough ''/boot'' space for anaconda to install a new kernel As written, I think the low disk-space tests targets #3 above. I believe that should be sufficient for testing the real-world use case. I've made some minor adjustments to the wording of the following test cases. I think they look good.
I also feeled that the third one already covers the real-world use.
https://fedoraproject.org/wiki/QA:Testcase_Preupgrade https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_low_/boot_disk_space https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release I think we can focus our low disk-space tests on just upgrading from the previous release (not from the previous previous release). I'd be okay with not adding the following test to our matrix. Do you agree or have concerns? https://fedoraproject.org/wiki/QA:Testcase_Preupgrade_from_older_release_low_/boot_disk_space
I added this case to make the cases more integrated. But yes, Focusing on the previous release is enough and can also reflect that in the older release.
Discussed during team sprint, the test cases look great! The additional failure cases noted in comment#16 will be something to target for a preupgrade focused test day.
Hurry will post the tests to the mailing list for final review. Leaving this open for now...
Review finished so close it. Another ticket will be created for cases of different disk space checks in /boot for a test day.
Metadata Update from @adamwill: - Issue untagged with: test review - Issue tagged with: test cases
Login to comment on this ticket.