PR#244: Also run Cloud tests on x86_64 UEFI, on Rawhide only - fedora-qa/os-autoinst-distri-fedora

fedora-qa / os-autoinst-distri-fedora

#244 Also run Cloud tests on x86_64 UEFI, on Rawhide only

Merged 3 years ago by adamwill. Opened 3 years ago by adamwill.

cloud-uefi into master

Also run Cloud tests on x86_64 UEFI, on Rawhide only

Adam Williamson • 3 years ago

910fbf2

schemas/fif-version.json

file modified

+1 -1

		`@@ -3,5 +3,5 @@`
		`"$id": "fif-version.json",`
		`"title": "FIF version schema",`
		`"type": "string",`
		`- "pattern": "^([*]\|[[:digit:]]{1,3})$"`
		`+ "pattern": "^([*]\|[[:digit:]]{1,3}\|Rawhide)$"`
		`}`

templates.fif.json

file modified

+33 -1

		`@@ -121,6 +121,20 @@`
		`},`
		`"version": "*"`
		`},`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide": {`
		`+ "arch": "x86_64",`
		`+ "distri": "fedora",`
		`+ "flavor": "Cloud_Base-qcow2-qcow2",`
		`+ "settings": {`
		`+ "BOOTFROM": "c",`
		`+ "DEPLOY_UPLOAD_TEST": "",`
		`+ "+HDD_1": "%HDD_2%",`
		`+ "ISO": "cloudinit.iso",`
		`+ "ROOT_PASSWORD": "weakpassword",`
		`+ "TEST_TARGET": "HDD_1"`
		`+ },`
		`+ "version": "Rawhide"`
		`+ },`
		`"fedora-CoreOS-colive-iso-x86_64-*": {`
		`"arch": "x86_64",`
		`"distri": "fedora",`
		`@@ -411,6 +425,14 @@`
		`"machine": "64bit",`
		`"product": "fedora-Cloud_Base-qcow2-qcow2-x86_64-*"`
		`},`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-64bit": {`
		`+ "machine": "64bit",`
		`+ "product": "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide"`
		`+ },`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-uefi": {`
		`+ "machine": "uefi",`
		`+ "product": "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide"`
		`+ },`
		`"fedora-CoreOS-colive-iso-x86_64-*-64bit": {`
		`"machine": "64bit",`
		`"product": "fedora-CoreOS-colive-iso-x86_64-*"`
		`@@ -587,6 +609,8 @@`
		`"fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 20,`
		`"fedora-Cloud_Base-qcow2-qcow2-ppc64le-*-ppc64le": 20,`
		`"fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 20,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-64bit": 20,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-uefi": 20,`
		`"fedora-CoreOS-colive-iso-x86_64-*-64bit": 50,`
		`"fedora-IoT-dvd_ostree-iso-aarch64-*-aarch64": 20,`
		`"fedora-IoT-dvd_ostree-iso-x86_64-*-64bit": 20,`
		`@@ -617,6 +641,7 @@`
		`"fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 20,`
		`"fedora-Cloud_Base-qcow2-qcow2-ppc64le-*-ppc64le": 20,`
		`"fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 20,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-uefi": 20,`
		`"fedora-CoreOS-colive-iso-x86_64-*-64bit": 50,`
		`"fedora-IoT-dvd_ostree-iso-aarch64-*-aarch64": 20,`
		`"fedora-IoT-dvd_ostree-iso-x86_64-*-64bit": 20,`
		`@@ -647,6 +672,7 @@`
		`"fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 20,`
		`"fedora-Cloud_Base-qcow2-qcow2-ppc64le-*-ppc64le": 20,`
		`"fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 20,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-uefi": 20,`
		`"fedora-KDE-live-iso-x86_64-*-64bit": 22,`
		`"fedora-Minimal-raw_xz-raw.xz-arm-*-ARM": 22,`
		`"fedora-Minimal-raw_xz-raw.xz-aarch64-*-aarch64": 22,`
		`@@ -672,6 +698,7 @@`
		`"fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 40,`
		`"fedora-Cloud_Base-qcow2-qcow2-ppc64le-*-ppc64le": 40,`
		`"fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 40,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-uefi": 40,`
		`"fedora-KDE-live-iso-x86_64-*-64bit": 42,`
		`"fedora-Minimal-raw_xz-raw.xz-arm-*-ARM": 42,`
		`"fedora-Minimal-raw_xz-raw.xz-aarch64-*-aarch64": 42,`
		`@@ -697,6 +724,8 @@`
		`"fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 40,`
		`"fedora-Cloud_Base-qcow2-qcow2-ppc64le-*-ppc64le": 40,`
		`"fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 40,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-64bit": 40,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-uefi": 40,`
		`"fedora-CoreOS-colive-iso-x86_64-*-64bit": 50,`
		`"fedora-IoT-dvd_ostree-iso-aarch64-*-aarch64": 40,`
		`"fedora-IoT-dvd_ostree-iso-x86_64-*-64bit": 40,`
		`@@ -727,6 +756,7 @@`
		`"fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 40,`
		`"fedora-Cloud_Base-qcow2-qcow2-ppc64le-*-ppc64le": 40,`
		`"fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 40,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-uefi": 40,`
		`"fedora-CoreOS-colive-iso-x86_64-*-64bit": 50,`
		`"fedora-IoT-dvd_ostree-iso-aarch64-*-aarch64": 40,`
		`"fedora-IoT-dvd_ostree-iso-x86_64-*-64bit": 40,`
		`@@ -757,6 +787,7 @@`
		`"fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 40,`
		`"fedora-Cloud_Base-qcow2-qcow2-ppc64le-*-ppc64le": 40,`
		`"fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 40,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-uefi": 40,`
		`"fedora-CoreOS-colive-iso-x86_64-*-64bit": 50,`
		`"fedora-IoT-dvd_ostree-iso-aarch64-*-aarch64": 40,`
		`"fedora-IoT-dvd_ostree-iso-x86_64-*-64bit": 40,`
		`@@ -786,7 +817,8 @@`
		`"profiles": {`
		`"fedora-Cloud_Base-qcow2-qcow2-aarch64-*-aarch64": 30,`
		`"fedora-Cloud_Base-qcow2-qcow2-ppc64le-*-ppc64le": 30,`
		`- "fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 30`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-*-64bit": 30,`
		`+ "fedora-Cloud_Base-qcow2-qcow2-x86_64-Rawhide-uefi": 30`
		`},`
		`"settings": {`
		`"POSTINSTALL": "autocloud"`

adamwill commented 3 years ago

Cloud images are now BIOS/UEFI hybrid in Rawhide (but not F33
or F34), so we want to run the Cloud tests on UEFI as well, but
only on Rawhide.

Signed-off-by: Adam Williamson awilliam@redhat.com

adamwill commented 3 years ago

This should resolve https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issue/238 . I've deployed it on staging and it seems to work OK, it doesn't result in any weird groups existing, and scheduling Cloud flavor for F34 and Rawhide works as expected (we get the BIOS and UEFI tests for Rawhide, only BIOS tests for F34).

This duplicates every test. I'm not sure if that's strictly necessary. @chrismurphy , out of the set of tests you see under Cloud_Base-qcow2-qcow2 at https://openqa.stg.fedoraproject.org/tests/overview?distri=fedora&version=Rawhide&build=Fedora-Rawhide-20210721.n.0&groupid=1 , how many do you really think we need to run "both ways"? All of them? Just one? Just a few? Thanks!

Note, we'll need to remember to update this to do the same for F35 when it branches, and maybe for F36 too if that branches before F34 goes EOL (can't remember how that will go). When F34 is EOL we can just go with * version for all tests and simplify this back again.

zuul commented 3 years ago

Build failed. More information on how to proceed and troubleshoot errors available at https://fedoraproject.org/wiki/Zuul-based-ci

os-autoinst-tox : FAILURE in 3m 26s
os-autoinst-perl : SUCCESS in 5m 25s

ngompa commented 3 years ago

Aside from the odd -64bit suffix, looks good to me!

:thumbsup:

Edited 3 years ago by ngompa

ngompa commented 3 years ago

In the future, I think we want all tests to run in UEFI and reduce BIOS to a basic boot test.

chrismurphy commented 3 years ago

@adamwill I'm thinking test them all for now? But then pare down the BIOS test to just seeing if it can successfully boot. What is a good test for that? The login test is strictly speaking much later than a boot test, but if we get to that point then we definitely booted. If it fails then it could have failed due to boot or anything startup related. But if we see that this single BIOS test fails, and UEFI tests at least pass for login, then we can infer, with a halfway decent chance of being correct, that it's BIOS boot failure related.

Edited 3 years ago by chrismurphy

adamwill commented 3 years ago

The -64bit suffix is necessary for...openQA reasons (it's the name of the x86_64 BIOS "machine", the x86_64 UEFI "machine" is called "uefi").

OK, I can tweak this to reduce the BIOS set to one test for Rawhide. I guess the reboot test might be a good one to use, as we can check rebooting works OK...

rebased onto 910fbf2

3 years ago

adamwill commented 3 years ago

OK, in the end I left two tests - base_reboot_unmount and base_services_start, as they sort of felt like the ones that might most likely diverge between BIOS and UEFI. So in Rawhide only those two run on BIOS, all the tests run on UEFI. On previous releases, all tests run on BIOS.

zuul commented 3 years ago

Build succeeded.