adcff8f don't fail playbook if there was error running test

Authored and Committed by bgoncalv 4 years ago
    don't fail playbook if there was error running test
    
    avoid confusion to distinguish test failure from infrastructure failure
    
    Without this fix when test reports error the playbook will exit with error and report infrastructure error, but this is actually a test error and package maintainer should be informed betterthat the failure was not some infra failure.
    
    The CI pipeline sends different topic in case of test or infra issue, in case the test exits with ERROR we are sending infra topic because the playbook exits with error, but for package maintainer is much harder to see what failed and he can think the pipeline failed because of infra outage instead of fixing the test.
    
    We currently consider as Infra Error: If any Beaker test ends in ERROR?
    
    The fix says from you says:
    1. Continue running other tests.
    2. Do not report beaker execution test as infra error.
    
    If yes, then this this makes sense.
    
        
file modified
+0 -2
file modified
+0 -2