#138 distinguish actions that provide results from actions that don't
Closed: Fixed None Opened 9 years ago by kparal.

This has been discussed in D104. Mainly:

b) The check doesn't have to be executed by the python directive. That is not true at the moment, but can/should easily become true in the future (bash checks, for example).
c) The python directives doesn't have to execute checks. It can be some other code, for example some environment preparation scripts (and then, in another python directive can the actual check be run).
d) We currently print to stdout anything that we find in Runner.working_data and assume they are check results. That's a bit shortsighted, because working_data should be used for arbitrary data transfer, not just check results. We will need to change that as well and find a better way to distinguishing check results from other data.

We need some way to tell which directive returned check results and which didn't. I can imagine several solutions (e.g. adding a custom attribute to a directive), but maybe the easiest way would be to let the check maintainer be responsible for that. We can simply provide some directive called report (essentially just a renamed resultsdb directive) and it would report to ResultsDB if enabled, and always print to stdout.

The task author is the best person to know which directive output to save and forward to report, and which directive output to ignore. Maybe we don't need any automagic happening. We just need to ensure that all tasks use the report directive, and we can stop guessing or printing out all variables at the end (our current practice).

Think about this and propose the best way how to make sure only relevant data (check results) get logged and reported by default.


This ticket had assigned some Differential requests:
D208

After discussion the current idea is:
stop printing all variable values automatically at the end of runner
print the variable content during export directive if --debug is enabled
* in resultsdb directive print TAP rather than CheckDetail object, and strip all output except for the first and the last line (advise to use --debug if the user wants to see it all, and have him look at export output then)

D208 will hopefully be sufficient for our purposes. Closing now.

Login to comment on this ticket.

Metadata