#616 [F32] CoreOS Test Day
Closed: Fixed 3 years ago by sumantrom. Opened 4 years ago by kparal.

We'll want to have a CoreOS test day for F32. This involves figuring out important changes in F32 FCOS, so that people can use that information as focus areas. @dustymabe said he would create such a list. Also, we might need some new FCOS test cases, that's reported in #617.


Yes. We'll need to get a next stream up first so we've got some work to do. Ping me again in two weeks.

@dustymabe two things we from your end:
1. The changeset which will help us writes test cases for the FCOS Test Day
2. A date - as we are closing in on Beta, it will help us fix up a cal. Even some possible dates, I can toss out are March (27th,31st) April (02nd,9th).

@sumantrom Please try to figure out whether a test day will be happening before F32 FCOS is out and update this ticket, thanks.

We had a chat with @dustymabe and we'll try to come up with a few simple test cases (targeted specifically at a test day, so not general test cases that would be used for release validation without modifications) and have a test day before F32 FCOS is stable (next stream should go into testing in 2 weeks from now, then it will take some time before it's pushed stable). We created a temporary place for test cases brainstorming here:
https://hackmd.io/lX41BH4TSlqDGqgwr_xWVg?edit

@coremodule @sumantrom I polished the hackmd document a little and started sketching up the test cases. Please have a look at it to get an idea, and help me sketch up more test cases (either those mentioned or some additional ones) in that document. We need to set up the test day pretty soon, so we should have some at least basic work done before our next meeting on Wednesday. Thanks!

@dustymabe mentioned that he prefers to have test cases which have a human element in them (things that people can misunderstand, mess up, act unpredictably) or there is a non-trivial environment configuration element (you can't run automation against all possible bare metal configurations, qemu configurations, network setup configurations, etc). And it makes sense. So a test case which will try to start and stop a particular service is not helpful, because that can be easily automated (and FCOS folks have a lot of such tests automated already). But let's say a testcase where the user writes his own ignition file is helpful, because humans can mess it up in infinite ways, and even if it's written exactly against spec, no automation can cover all possible cases, so the input randomness can help us uncover interesting issues. What is also valuable is finding UX issues and documentation issues (so we can point users to some existing documentation and see how well it works for them).

@kparal I have created the https://fedoraproject.org/wiki/Test_Day:Fedora_32_CoreOS and a meta https://fedoraproject.org/wiki/Test_Day:Fedora_32_CoreOS_Test_Day_TestdayApp_Metadata and event : http://testdays.fedorainfracloud.org/events/84.
I am working towards creating the wiki and all of the above pages are work in progress and will be updated.
As for dates, I am looking at end of this week OR starting of next week .. I am good with 2020-06-02 or a couple of days after? @dustymabe @kparal @coremodule WDY'all think?

@dustymabe do you feel deploying on digital ocean will be a good manual test case to have? I remember when you showed us the automation test run, it built the digital ocean but never deployed on the digital ocean.

I updated the wiki page to use #fedora-coreos irc channel. I also updated the hackmd document to link to your wiki test cases. I don't have a preference on the test day date.

In the Fedora CoreOS meeting of 2020-05-27 it was decided that, we will stick to 2020-06-08 as the tentative Test Day date and general request for "come up with more manual test scenarios" was ack'ed by the CoreOS folks. CoreOS folks will help review the test cases and help us expand more.

I created some new test cases and edited existing ones, all of them in Sumantro's home space, I assume you don't mind, @sumantrom ;-)

@kparal not at all :D

I created some new test cases and edited existing ones, all of them in Sumantro's home space, I assume you don't mind, @sumantrom ;-)

@kparal @coremodule The test day date is finalized to be 2020-06-08. I am going to start announcing the test day on ML and blogs

[0]https://github.com/coreos/fedora-coreos-tracker/issues/491#issuecomment-637240830

@sumantrom Please don't forget to add a fedocal event (while fedocal is still up), thanks.

@sumantrom Please don't forget to add a fedocal event (while fedocal is still up), thanks.

was just about to do that :D

@sumantrom Please don't forget to add a fedocal event (while fedocal is still up), thanks.
https://apps.fedoraproject.org/calendar/QA/2020/6/8/#m9767

FCOS folks added a few more testcases and amended the existing ones. I moved all the testcases to an official name and updated the testday app. Everything should be hopefully now prepared. See you on Monday!

thanks all for the really hard work on this! See you on Monday!

The Fedora Magazine Announcement is out and this means it will spread all around Fedora media(twitter,facebook,linkedin and reddit)
https://fedoramagazine.org/contribute-at-the-fedora-coreos-test-day/

Thanks a LOT to all for making this happen

@sumantrom Please ensure that all test day results have been transferred to the wiki and then I believe we can close this ticket, wdyt?

there are some more additions to the results, I will again transfer
once again and then take care of this

On Tue, Jul 7, 2020 at 7:23 PM Kamil P=C3=A1ral pagure@pagure.io wrote:

kparal added a new comment to an issue you are following:
@sumantrom Please ensure that all test day results have been transferred = to the wiki and then I believe we can close this ticket, wdyt?

To reply, visit the link below or just reply to this email
https://pagure.io/fedora-qa/issue/616

--=20
//sumantro
Fedora QE
TRIED AND PERSONALLY TESTED, ERGO TRUSTED

there are some more additions to the results, I will again transfer
once again and then take care of this
On Tue, Jul 7, 2020 at 7:23 PM Kamil P=C3=A1ral pagure@pagure.io wrote:

kparal added a new comment to an issue you are following:
@sumantrom Please ensure that all test day results have been transferred =
to the wiki and then I believe we can close this ticket, wdyt?
To reply, visit the link below or just reply to this email
https://pagure.io/fedora-qa/issue/616

--=20
//sumantro
Fedora QE
TRIED AND PERSONALLY TESTED, ERGO TRUSTED

Done.
The results are up to date

Metadata Update from @sumantrom:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Login to comment on this ticket.

Metadata