The anaconda team would like to have a special Test Day, on 2015-02-12, with the purpose of testing anaconda's DNF backend and trying to identify all cases where it is broken compared to the yum backend, especially release-blocking issues.
A secondary goal is to develop tests for anaconda's CI system to cover the packaging-related functions.
Here are some known issues as of anaconda-22.17:
Added a placeholder to the QA calendar: https://apps.fedoraproject.org/calendar/meeting/2028/
Note - we can handle actually getting the tests into CI. What we are most interested in here is tests in the form of kickstart files and expected outcomes, which can just be a sentence or two. We can help with the kickstart portion too. Interactive tests (things that involve lots of clicking back and forth) can't really be represented by kickstart, so there's no need to get those into CI.
I've thrown up the first version of the test day page:
https://fedoraproject.org/wiki/Test_Day:2015-02-12_Anaconda_DNF
still needs a bunch of stuff, it's just to have something there. I'll add the known issues and figure something out about getting people to submit kickstarts later. There's a webapp we can use for reporting results for the planned tests which is marginally better than editing the test day page directly, but it seems to be down ATM - the URL is http://testdays.qa.fedoraproject.org/testdays/ . If it starts working again soon I'll add an event in that and update the page appropriately, if not I'll just throw together a manual-edit table.
We will send out announcements over the weekend or on Monday - it's usually best to do it not too long before the event, or else people feel it's too far in the future and just forget about it. Once I've put up a blog post or something it'd be great if others can post on their own blogs or twitter or G+ or facebook or whatever and link to it.
Replying to [comment:4 adamwill]:
There's a webapp we can use for reporting results for the planned tests which is marginally better than editing the test day page directly, but it seems to be down ATM - the URL is http://testdays.qa.fedoraproject.org/testdays/ . If it starts working again soon I'll add an event in that and update the page appropriately, if not I'll just throw together a manual-edit table.
Tim has rebooted that machine, it seems to be back up.
OK, I've put some test cases (mostly installer validation ones, but also some older ones from the same categories that we don't usually bother with any more) in a results page at http://testdays.qa.fedoraproject.org/testdays/show_event?event_id=22 , and updated the test day page to reference that and also to add an 'exploratory testing' section explaining roughly what we want folks to do and adding a table to list bugs and kickstarts.
I've sent out a blog post - https://www.happyassassin.net/2015/02/10/fedora-22-anacondadnf-test-day-coming-2015-02-12/ - about the event, and I'll throw that out on social media, forums and the Fedora Magazine soon. If you folks could G+ it, write your own blog posts (linking to mine or just improving on it :>) etc, that'd be a great help. thanks!
So the Test Day happened, and it was pretty good I think. I'll move the test day app results to the wiki page and send a post-event summary mail out probably on Monday.
Results are on the wiki. Thanks for organization.
Login to comment on this ticket.