#77 synchronize TAP output from CheckDetail with TAP13 parser
Closed: Fixed None Opened 9 years ago by tflink.

At the moment, the TAP output from CheckDetail doesn't parse with pytap13. The things I've found thus far are:
* Missing "TAP version 13"
* Missing "test plan" (ie, a 1..n line somewhere in the output where n is the number of cases run)

There are multiple, somewhat related levels to this:
# Should it be the tasks' responsibility to ensure that the TAP output is 100% valid and ready for reporting?
* If so, we should create some better convenience methods for creating TAP output
* Also, we'd need to stop returning results as lists of mostly-TAP strings
# Do we want the output from CheckDetail to be passable into directive_resultsdb without farther modification?
* This might work if we intend to only have one CheckDetail object per check run
# Does the pytap13 parser need to be made more flexible so that it can read not-quite-complete TAP snippets?
* This is the way we'd likely have to go if we construct the complete TAP output in the reporting mechanisms


This ticket had assigned some Differential requests:
D68

This is IMHO addressed in D68 - the TAP output of CheckDetail's export_TAP is capable of exporting multiple CheckDetails and adds the proper TAPv13 headers.

ad 1) Creating TAP outputs via CheckDetail export is IMHO quite convenient. Do you think otherwise?
ad 2) As it is, resultsdb directive is created in a way that expects the direct output from CheckDetail (or multiple check details exported in one proper TAPv13 "stream"
ad 3) I do not see a need to support "invalid" TAPv13 data. The standard is not that complicated to adhere to, and we have the convenience methods (CheckDetails + export_TAP).

This is IMHO addressed in D68

Agreed. The biggest problems I was hitting should be addressed with D68

ad 1) Creating TAP outputs via CheckDetail export is IMHO quite convenient. Do you think otherwise?

Yesterday, I would have said yes. D68 should make it much easier, though.

ad 2) As it is, resultsdb directive is created in a way that expects the direct output from CheckDetail (or multiple check details exported in one proper TAPv13 "stream"

But it's still effectively limited to one CheckDetail export per reporting action, no? Granted, I'm not sure this is a big problem for the tasks that we currently have but it's something that we may want to explore in the future.

ad 3) I do not see a need to support "invalid" TAPv13 data. The standard is not that complicated to adhere to, and we have the convenience methods (CheckDetails + export_TAP).

The point wasn't that the TAP13 standard was complicated - just that one way of constructing a complete TAP13 result would be to create incomplete TAP in CheckDetail and assemble before reporting.

This is effectively resolved with D68 and the fix for #120. Closing.

Login to comment on this ticket.

Metadata