JaT v0.0.31 - • 4 years ago
Overview of changes:
* Fixed formatting of normalized params
JaT v0.0.30 - • 4 years ago
Overview of changes:
* Improved harness connector reporting granularity
Connectors will see fullest case ID available, ie. including test
params and phase case id's. Previously, params were not included.
Newly connectors will also see original event type, allowing them to
skip types that do not fit harness reporting model.
* Minor cleanups and improvements
JaT v0.0.29 - • 4 years ago
Overview of changes:
* Harness connectors will now run in own dir
We need to allow them some persistence, ie. they can now save/load
local files.
JaT v0.0.28 - • 4 years ago
Overview of changes:
* Clean up relic name before _jath submission
Don't always pass full (local) file path as its "name" for the
harness connector. Instead, remove the local path part, which is
irrelevant to test reader.
JaT v0.0.27 - • 4 years ago
Overview of changes:
* Hotfixed infite loop in jat__submit*() when parsing arguments
JaT v0.0.26 - • 4 years ago
Overview of changes:
* log.yaml will now also include test code URL
Of course URL contents will have to be prepared by other
means but if CI can make it work, test reviewer can get to the test
code really fast.
* jat__submit*() are now more consistent
They don't emit RELIC.FILE event if submission is unsuccessful,
instead they all emit MESSAGE.WARNING or MESSAGE.INFO if -s is
provided.
JaT v0.0.25 - • 4 years ago
Overview of changes:
* Show and log test doc URL if baseurl is provided
jat.sh now takes over doc URL logic from jattool. Ie. if
JAT__TEST_DOC_BASEURL is non-empty, it will use new module
shellfu-bash-jats to create URL to test doc.
The URL is shown immediately as well as added to log.yaml for later
use by `jattool export` or other tools.
JaT v0.0.24 - • 4 years ago
Overview of changes:
* Fixed missing SFPI dependencies
JaT v0.0.23 - • 4 years ago
Overview of changes:
* Added (experimental) support for harness connector plugins
Relic submissions and partial and final results can now be synced to
external tool (such as a Beaker lab or beaker/restraint) already
during run of the test.
* Added warning if result submission fails
* Updated MKit to v0.0.40
JaT v0.0.22 - • 4 years ago
Overview of changes:
* Added jat__abort() for aborting tests
Preferred to jat__die(), this new function allows more effective
reporting of failure modes to upper levels.
* Added jat__seen() to simplify run status checking
Typically test libraries want to check if there were any errors
or failures in boolean context, without regarss to the count.
Testing jat__stat() output is cumbersome, jat__seen() should be
able to serve these cases better.
* jat__die() will now always ask jat__rfinish() for final exit status
jat__die() always used just die(), which returns 3, which is
inconsistent with how tests behave normally; normally jat__rfinish()
is consulted, which has own exit status polict (4 for errors, 3 for
warnings, etc.)
Technically, this is backwards-incompatible change, but should only
affect cases where test harness would distinguish between non-zero
exit statuses.
* Fixed several ambiguous grep calls
* Polished few usage messages
JaT v0.0.21 - • 4 years ago
Overview of changes:
* Fixed YAML storing original char after quoted char
* Moved to new branch naming scheme
Latest releases will be published to 'latest' branch, not 'last'.
* Keep dumped vars in single block for readability
* Codebase maintenance
JaT v0.0.20 - • 4 years ago
Overview of changes:
* Fixed inability to use jat__cmp() to compare empty strings
jat__cmp() would die with usage messaage if one of arguments was
empty string. While this is certainly invalid for OP, it should be
valid for RVAL and OVAL.
* Added exit status validity checks into jat__cmd() and jat__eval()
Unexpected exit statuses that are neither 0 or 1 now raise warning.
This should help identify some hidden problems.
Statuses passed via `-S` will automatically be deemed valid, and
the default valid list can be overriden locally and globally.
* Fixed incorrect jat__submit*() variable docstrings
* Updated MKit to v0.0.38
* Small fixes and cleanups
JaT v0.0.19 - • 5 years ago
Overview of changes:
* Ported variable dumper from library(distribution/dump)
JaT v0.0.18 - • 5 years ago
Overview of changes:
* Added support for double dash in jat__submit()
* Fixed various porting errors from distribution/dump
JaT v0.0.17 - • 5 years ago
Overview of changes:
* Ported submission policy overriding options for `jat_dump`
JaT v0.0.16 - • 5 years ago
Overview of changes:
* Abandoned "session" term in favor of "run"
The term "session" is confusing; "run" should be pretty obvious in
most contexts.
* jat_dump now allows controlling submission policy
Scheduler can now force jat_dump submit all files regardless of
size. This will make access to common attachments easier in
some cases.
JaT v0.0.15 - • 5 years ago
Overview of changes:
* Minor reshaping of phases
Phases don't have names anymore, instead they have hints, very
similar to asserts. This change also extends to syntax of jat__pstart?()
family of functions, which now won't treat first positional argument as
phase name, instead will acceot `-h HINT` just like asserts,
Additionally, if phase hint is not provided, it will be composed of
phase type and CASEID, if available. (Phase type must always be
available so there will always be some hint.)
* Fixed typo in docstring
JaT v0.0.14 - • 5 years ago
Overview of changes:
* Added support for quoted storage of strings in YAML log
Because testers can (and will!) output various strange characters,
we sometimes hit not only chars that are prohibited from YAML, but
also chars that merely trigger parsing bugs in tools down the line.
This release changes how strings are saved in YAML log, to make the
storage robust and readable, while preserving way to extract the
original data.
* Terminal output will switch to hexdump in case of binary data
* Added support for embedding data with direct asserts
Direct asserts (jat__fail() and jat__pass()) now allow passing set
of KEY=VALUE pairs.
JaT v0.0.13 - • 5 years ago
Overview of changes:
* Fixed jat__param() exit status logic
JaT v0.0.12 - • 5 years ago
Overview of changes:
* Renamed relic storage path subdirectory to 'relics'
* Removed useless hint when submitting multiple files
* Changed root for RELIC.FILE path to within 'relics' subdirectory
JaT v0.0.11 - • 5 years ago
Overview of changes:
* Fix small logging error when submitting file
JaT v0.0.10 - • 5 years ago
Overview of changes:
* Added support for test parametrization
* Added support for RELIC event types
* Fixed variable loggers logging quoted versions of data
* Fixed grep spam on first write of session key
JaT v0.0.9 - • 5 years ago
Overview of changes:
* Fixed several copu/pasting errors in docstrings
* Fixed jat__filerestore() restoring files multiple times in some cases
* Fixed variable loggers showing in the console output as error messages
JaT v0.0.8 - • 5 years ago
Overview of changes:
* Fixed several critical bugs in the submit API
Somehow I managed to merge a very buggy version of the jat__submit?()
functions.
JaT v0.0.7 - • 5 years ago
Overview of changes:
* Removed log.ansi
The file was never really useful: it would not contain whole output
anyway (only parts produced by jat module), so if user wants to
preserve that, keeping `jattool runtest` outputs is the way to go.
Hence, the file is not worth documenting and keeping.
* Few visual improvements to terminal output
Colors are turned off if stderr is redirected to file, header is a
bit cleaner and colors are more consistent.
* Removed CMD and CODE from jat__cmd() and jat__eval()
These make simple logs actually less readable and take away the
control over the simplicity from test writer.
JaT v0.0.6 - • 6 years ago
Overview of changes:
* Fixed several risky grep calls
* Improved usage error messaging
* Updated MKit to last version
JaT v0.0.5 - • 6 years ago
Overview of changes:
* Added variable value logging functions
You can call `jat__log_infov VARNAME` to log contents of VARNAME
directly.
* Fixed data corruption on usage error
Due to bad call to event logger, data could be corrupted on usage
errors.
* Fixed usage messages being silent (issue #1)
* Smaller cleanups and docstring updates
JaT v0.0.4 - • 6 years ago
Overview of changes:
* jat__filebackup() now silently (more-or-less) overwrites existing backups
If the backup has been used, the overwrite is silent (with just
subtle hint in the info message. Otherwise there is warning.
Note that this only affects double backup of the same file, you can
still add different files to namespace.
* Dropped support for BEIDs
There was not really any support, so no one should miss this. Also
CASEIDs probably already solve the issue BEIDs wanted to solve. Well
I'm not entirely sure of that statement, but we can always add BEIDs
if real use case comes up. For now let's keep the code smaller.
* Added new submission utility functions
jat__submit() can become chores with many files, so it's now
accompanied with two new friends: jat__submitn() to submit multiple
files, and jat__submitb() to submit multiple files as tarball.
* Fixed and added many docstrings
* Fixed URL after moving to pagure.io
* Updated TFKit and tests to reflect small convention change
JaT v0.0.3 - • 6 years ago
Overview of changes:
* Ensured log.yaml won't contain unacceptable characters
Just filter the whole thing, replacing unacceptable characters by �
(U+FFFD AKA 'REPLACEMENT CHARACTER').
* jat__stat() will now treat empty phases properly
* Smaller cleanups
JaT v0.0.2 - • 6 years ago
Overview of changes:
* Fixed caseid merging
caseids were not working at all; now it's the "preview" stage.
* Re-introduced session id
In order to be able to re-run tests and avoid conflict without much
hassle, ids are re-introduced; in a bit cleaner way this time,
though.
* Made phase and session verdicts a bit smarter
Don't consider empty phases PASS, propagate ERROR or WARNING to
phase verdict...
* Added jat__die() suicide pill
Will finish test in a way that jattool report is still correct but
it should be clear that the test "panicked".
* Switched to non-deterministic by default
Non-deterministic is easier, and timestamps may be useful.
* Used 'assert' term in ANSI output
* Codebase maintenance
JaT v0.0.1 - • 6 years ago
Overview of changes:
* Revamped event data model a bit
Removed lots of repetition and useless cruft, simplified events.
* Added support for multi-line log messages
Every argument to logging functions will now constitute one line of
the final message.
Thx @ester and @magda.
* jat__sfinish() will now return informative exit status
Roughly following TFKit convention
Case ids dynamically extend test ids from within the test, adding
further granularity, feature linkage and promise/delivery control.
* Added support for t., o., and r. prefixes for assert data
Allowed these prefixes in data key name to set convention where
relevancy to test, oracle or and result is made more clear.
* Added support for saving exit status into file
Useful in xcase when we need for the result to stay from one handler
than other. In that case local var cannot suffice but since context
is already driven by working directory, using file is safe---so it's
than a global variable.
* Made reloading more robust
Now jat__sinit() Won't reload session unless test id and version match
* Added support for case ids
* Fixed error when submitting without altname
* Added test id and version to stderr and log
* Codebase cleanups and maintenance