Hey All,
We now have the Fedora IoT Edition which will receive a section in the Release validation pages with a bunch of test cases which community members can write and run. This is tracker issue to track the progress of the same.
Woot! Sumantro, ping me if you need to offload some of the work getting this going. Glad to help.
Geoff Marr IRC: coremodule
On Wed, Jan 30, 2019 at 10:07 AM Sumantro Mukherjee pagure@pagure.io wrote:
sumantrom reported a new issue against the project: fedora-qa that you are following: `` Hey All, We now have the Fedora IoT Edition which will receive a section in the Release validation pages with a bunch of test cases which community members can write and run. This is tracker issue to track the progress of the same. `` To reply, visit the link below or just reply to this email https://pagure.io/fedora-qa/issue/581
sumantrom reported a new issue against the project: fedora-qa that you are following: `` Hey All,
fedora-qa
We now have the Fedora IoT Edition which will receive a section in the Release validation pages with a bunch of test cases which community members can write and run. This is tracker issue to track the progress of the same. ``
To reply, visit the link below or just reply to this email https://pagure.io/fedora-qa/issue/581
install testing - install from iso - disk image installation to SBC, virtualization - pxe installation
initial-setup - existing testcase should be sufficient with some notes - https://fedoraproject.org/wiki/QA:Testcase_base_initial_setup
package layering - add/remove packages - existing testcase looks sufficient - https://fedoraproject.org/wiki/User:Miabbott/QA/AtomicTests/RpmOstree_Package_Layering
container runtime test - podman pull, hello world or custom container
Raspberry Pi 3 hardware testing - network (wired and wifi), bluetooth
I have linked a couple of the testcases I've found that should also work for IoT testing. Are there existing tests for installation (iso, disk image) and container runtime that we could also use?
Besides the tests you mentioned, there is more regarding ostree in the documentation for Fedora IoT release criteria:
Upgrade When a new ostree is composed and made available, it must be possible to upgrade an existing host using rpm-ostree upgrade.
rpm-ostree upgrade
Rollback After upgrading to the latest ostree, it must be possible to rollback to an older version using the rpm-ostree rollback command.
rpm-ostree rollback
Rebase It must be possible to rebase to a different IoT release by using the rpm-ostree rebase command.
rpm-ostree rebase
Testcases we could reuse are:
Any thoughts?
Besides the tests you mentioned, there is more regarding ostree in the documentation for Fedora IoT release criteria: Upgrade When a new ostree is composed and made available, it must be possible to upgrade an existing host using rpm-ostree upgrade. Rollback After upgrading to the latest ostree, it must be possible to rollback to an older version using the rpm-ostree rollback command. Rebase It must be possible to rebase to a different IoT release by using the rpm-ostree rebase command.
I remember writing this for silverblue https://fedoraproject.org/wiki/User:Sumantrom/Draft/Testcase_silverblue_upgrade_%26_rebase
https://fedoraproject.org/wiki/User:Sumantrom/Draft/Testcase_silverblue_upgrade_%26_rebase
@lbrabec @pwhalen will this be apt test cases for what Lukas mentioned?
Testcases we could reuse are: Disk image: QA:Testcase_arm_image_deployment (we should mention fedora-arm-installer there) PXE boot: QA:Testcase_Boot_Methods_Pxeboot Install from ISO: QA:Testcase_install_to_SATA Container runtime testcase could be inspired by QA:Testcase_base_service_manipulation (pulling, starting, stoping and checking status) Any thoughts?
Disk image: QA:Testcase_arm_image_deployment (we should mention fedora-arm-installer there) PXE boot: QA:Testcase_Boot_Methods_Pxeboot Install from ISO: QA:Testcase_install_to_SATA Container runtime testcase could be inspired by QA:Testcase_base_service_manipulation (pulling, starting, stoping and checking status)
@alciregi will you be able to help us the Rpi3 test cases? :)
Artifacts to be tested: ostree iso for x86_64, aarch64 disk images for x86_64, aarch64 and armhfp Tests to be completed install testing - install from iso - disk image installation to SBC, virtualization - pxe installation initial-setup - existing testcase should be sufficient with some notes - https://fedoraproject.org/wiki/QA:Testcase_base_initial_setup package layering - add/remove packages - existing testcase looks sufficient - https://fedoraproject.org/wiki/User:Miabbott/QA/AtomicTests/RpmOstree_Package_Layering container runtime test - podman pull, hello world or custom container Raspberry Pi 3 hardware testing - network (wired and wifi), bluetooth I have linked a couple of the testcases I've found that should also work for IoT testing. Are there existing tests for installation (iso, disk image) and container runtime that we could also use?
Artifacts to be tested:
ostree iso for x86_64, aarch64 disk images for x86_64, aarch64 and armhfp
Tests to be completed
@coremodule can you verify if this is working ? https://fedoraproject.org/wiki/User:Roshi/QA/AtomicTests/Atomic_Image_Boots
@pwhalen @lbrabec The test day wiki is here https://fedoraproject.org/wiki/Test_Day:Fedora_IoT_Edition_2019-03-13 have a look and let me know if there is something that needs fixing. I am yet to create the meta and the test results. I will link them here in a few hours. The Fedocal is already booked and will blog post will be coming up soon!
Sure. But do you mean writing new test cases?
@alciregi will you be able to help us the Rpi3 test cases? :) Sure. But do you mean writing new test cases? @alciregi :100: No, nothing to write the regular testing like @pwhalen mentioned "Raspberry Pi 3 hardware testing - network (wired and wifi), bluetooth"
Sure. But do you mean writing new test cases? @alciregi :100: No, nothing to write the regular testing like @pwhalen mentioned "Raspberry Pi 3 hardware testing - network (wired and wifi), bluetooth"
@sumantrom Just ran the test on Fedora-IoT-ostree-x86_64-30-20190215 and Fedora-IoT-ostree-x86_64-30-20190301 and found that podman is installed by default but docker is not... So technically that particular test fails.
@pwhalen Do you have any idea why the "latest-Fedora-IoT-30" [0] links to the 20190215 image and not the 20190301 image?
[0] https://kojipkgs.fedoraproject.org/compose/iot/latest-Fedora-IoT-30/compose/
@pwhalen Do you have any idea why the "latest-Fedora-IoT-30" [0] links to the 20190215 image and not the 20190301 image? [0] https://kojipkgs.fedoraproject.org/compose/iot/latest-Fedora-IoT-30/compose/
I guess because the 20190301 compose result is DOOMED. In fact there are no the aarch64 raw image. https://kojipkgs.fedoraproject.org/compose/iot/Fedora-IoT-30-20190301.0/STATUS
As far as I know, latest F30 IoT composes don't boot on the Raspberry. Will we have a working one in time for the test day?
@alciregi will you be able to help us the Rpi3 test cases? :) As far as I know, latest F30 IoT composes don't boot on the Raspberry. Will we have a working one in time for the test day?
@pwhalen any ideas? @alciregi the test day must go on but I am worried that we will miss on Rpi
@alciregi will you be able to help us the Rpi3 test cases? :) As far as I know, latest F30 IoT composes don't boot on the Raspberry. Will we have a working one in time for the test day? @pwhalen any ideas? @alciregi the test day must go on but I am worried that we will miss on Rpi
We are working to get a new compose that includes an image that will boot on the Raspberry Pi 3. Currently the last successful compose was Feb 15th.
@coremodule can you verify if this is working ? https://fedoraproject.org/wiki/User:Roshi/QA/AtomicTests/Atomic_Image_Boots @sumantrom Just ran the test on Fedora-IoT-ostree-x86_64-30-20190215 and Fedora-IoT-ostree-x86_64-30-20190301 and found that podman is installed by default but docker is not... So technically that particular test fails.
I can take that test Roshi wrote and amend it to only test podman, as well as extend it to include a basic functionality test.
Possible podman testcase: https://fedoraproject.org/wiki/User:Pwhalen/QA/IoT_Tests/Podman_Basic
Raspberry Pi 3 hardware tests: https://fedoraproject.org/wiki/User:Pwhalen/QA/IoT_Tests/RaspberryPi_3
@pwhalen The test cases are updated on the results page http://testdays.fedorainfracloud.org/events/62
Great, thanks very much for your help getting this ready.
Metadata Update from @kparal: - Issue tagged with: iot
This one ran smoothly http://testdays.fedorainfracloud.org/events/73
Metadata Update from @sumantrom: - Issue close_status updated to: Worksforme - Issue status updated to: Closed (was: Open)
Metadata Update from @kparal: - Issue status updated to: Open (was: Closed)
Metadata Update from @kparal: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Login to comment on this ticket.