testdays

testdays is a small (at present) CLI for interacting with Fedora Test Day wiki pages, comparable to relval (which interacts with release validation test pages). Like relval, it uses the python-wikitcms module. Try it out:

testdays stats --release 25
testdays stats --filter nouveau --list-bugs

Source

You can visit the testdays project page on Pagure, and clone with https://pagure.io/fedora-qa/testdays.git. Tarballs are available for download. If you have its dependencies - setuptools, wikitcms, and, for pre-Python 2.7 systems, argparse - installed, you can run it directly from the git checkout as ./testdays.py.

Repository

Fedora users can download this repository file and place it in /etc/yum.repos.d/. yum install testdays will install testdays after adding the repository definition, and you will get updates along with all your other system updates.

Installation on other distributions

To install the module system-wide on other distributions, simply run python(2) setup.py install as root.

Usage

The examples above should give a good idea how to use testdays. You can use --release to limit the set of test days queried to those associated with a single release, or --since and/or --until to specify a date range. Note that --since defaults to 2010-09-01, as earlier Test Days rarely used the {{result}} template and consequently the counts of testers and tests will be inaccurate. You cannot use --release and --since/--until together; --release will take precendence if you try.

You can use --filter to limit the set of pages queried further; only those whose names match the string passed will be queried (the match is case-insensitive). --filter can be combined with --release or --until/--since.

For each Test Day, several statistics will be printed: the number of individual testers who submitted at least one result, the number of tests run in total by all testers, the number of bugs referenced, the ratio of bugs referenced per tester, and a "Fixed %" value which represents an attempt to approximate what percentage of valid, unique bugs associated with the Test Day have since been fixed. See this mailing list post for more details on the specific statistics calculated, the formulae used, and the intended benefit of each statistic. If you pass --list-bugs, a one-line summary of each bug referenced will be printed above the statistics.

If more than one Test Day is queried, some total statistics across the whole set of Test Days will be reported.

There is a second subcommand, modify, which can perform two modification operations on Test Day pages. testdays modify --results (pagename) will modify the Test Day page with name (pagename) and try to change results like {{result|fail}}<ref>{{bz|1384440}} </ref> - the form used by the testdays webapp - into a form like {{result|fail||1384440}}. testdays modify --longrefs (pagename) will modify the Test Day page with name (pagename) and try to move long comments out of the results table(s) 'References' column(s) into a separate section at the bottom of the page, to help make the table(s) more readable. --longrefs and --results can be used together to perform both modifications at once. All use of the modify subcommand requires a wikitcms credentials file for authentication (to keep this app's code simple, there is no provision for interactive username / password entry like in relval).

License

testdays is released under the GPL, version 3 or later.