JATS--Just A Test System is a test system that is intended to be extensible, compatible across the most common GNU/Linux distros and most importantly, oriented towards Quality Assurance needs: simple API, cleaner focus on System Under Test, ability to write robust tests on all levels (from Unit to System to Integration), usable and extensible result format and built-in support for things like granular test and test case identification and specification reference management.
This repository contains early version of jattool, which is a toolset designed to work with JATS tests: suite management and packaging, test development, test planning, test running (harness) and result management.
Development stage of this tool and most of its libraries is very early, that is, most parts of intended functionality are missing, buggy and/or subject to change.
Branch last always contains the last released version.
See Notes section for details.
Apart from core things like Bash and coreutils, and minus things I'm forgetting right now, jattool uses:
Shellfu for internal function management and some of its core libraries,
Saturnin Shellfu library for support of configuration, versioning and meta-command structure,
jat Shellfu library for wrapping and running tests (currently only Shellfu/Bash tests are supported,
python2-jinja2 for exporting test results to other formats such as HTML.
On Fedora, following should work:
# dnf copr enable netvor/shellfu # dnf copr enable netvor/jats # dnf install jattool
Older distros like CentOS 6 and 7, don't have
dnf copr or
yum copr plugin,
so you'll have to pick correct .repo file from COPR site and download it
under /etc/yum.repos.d. (You can use this convenience script, though.)
Note that on testing machines, where you only want to run test (but not eg. export to HTML), you only need to install jattool-minimal:
# yum install jattool-minimal
This will help you avoid excess dependencies. Test suites should also normally depend only on jattool-minimal.
Debian builds are available per-request; there's no repository, though, and auhor of jattool is not aware of easy (and reasonably proper) way to create one.
To install manually, clone this repository and use "old-school" make commands:
git clone https://pagure.io/jattool make sudo make install
Note that you will need to do this also for dependencies; all Shellfu dependencies
use the same method (
make && sudo make install), for the rest, use appropriate
jattool [common-options] SUBCOMMAND [subcommand-options]
Common options for jattool are
-d for debugging and
for verbosity. Note that these are options are for
jattool -d foo and
jattool foo -d are completely different things!
Here we'll just briefly introduce the available subcommands:
jattool sfind, jattool tfind - discover test suites and individual tests.
jattool qrun - quick&dirty in-place test run.
jattool runtest - proper test run, ie. copy test to temporary directory and run it with identification data.
jattool export - export test report ftom YAML log to other formats such as HTML.
Please refer to
jattool --help for general help and or
SUBCOMMAND --help (some subcommands).
You first need to obtain some tests. At this moment, jattool does not provide necessary tools to create tests, however, it's recommended to start with demo test suite called jats-demo. This is available in form of builds; in same repositories as jattool itself:
sudo dnf install jats-demo
Of course you can just clone it:
git clone https://pagure.io/jats-demo jattool tfind jats-demo jattool runtest jats-demo/src/foo
Follow jats-demo README for details about how typical JATS test suite is organized.
We're committed to following SemVer 2.0 scheme. However, note that for early stages when Major version is zero, SemVer allows for custom rules. We fill this "gap" with SemVer extension, which (at this stage) means:
x=0, y=0, only z is changing; ie. version is
Breaking API changes are allowed between any released versions.
Once the project reaches 0.1.0, breaking API changes will only be allowed on the Y level (e.g. 0.1.5 -> 0.2.0 may not be safe) but only safe, additive ones are allowed on Z level (e.g. 0.1.5 -> 0.1.6 should be safe).
For filing bug requests, use the Issue tracker on project's Pagure page, or ping the auhor directly:
Alois Mahdal <email@example.com> https://netvor.info/