This is something that I still don't understand well but from what I do know, I suspect that it may not belong in libtaskotron. Is there any documentation on what the end goal is with this or can you go into more detail on where/how it would be used and why it's important?

After talking with acarter, this may need some work in libtaskotron but will likely feed into our long term plan of having an intermediate rules engine to determine "real pass" or not.

That being said, there may be a need to revisit the logic of how the job status is computed based on whether the child test cases are "blocking" or not. We'll know more once we get more details figured out

IIRC we've done some work in the new TaskResult YAML file and resultsdb directive that will allow you to specify different testcase names for different "subchecks". So for example you can report rpmlint/foo: PASSED, rpmlint/bar: FAILED, rpmlint: PASSED. That could be sufficient for this use case. It then depends on the system that's //listening// for these fedmsgs to decide which testcases to listen for, which are important and which are not, and act on that information accordingly.