| |
@@ -0,0 +1,28 @@
|
| |
+ name: result
|
| |
+ type: string
|
| |
+ purpose: specify how test result should be interpreted
|
| |
+
|
| |
+ motivation:
|
| |
+ Even if a test fails it might makes sense to execute it to be able
|
| |
+ to manually review the results (ignore test result) or ensure the
|
| |
+ behaviour has not unexpectedly changed and the test is still failing
|
| |
+ (expected fail).
|
| |
+
|
| |
+ stories:
|
| |
+ - As a developer I have a test, I know it's failing for now, yet I
|
| |
+ want to have it for future reference.
|
| |
+ - As a tester I want to regularly execute the test but temporarily
|
| |
+ ignore test result until more investigation is done and the test
|
| |
+ can be fixed properly.
|
| |
+
|
| |
+ notes:
|
| |
+ The following values should be supported:
|
| |
+ - respect: test result is respected (fails when test failed)
|
| |
+ - ignore: ignore the test result (test always passes)
|
| |
+ - xfail: expected fail (pass when test fails, fail when test passes)
|
| |
+
|
| |
+ examples: |
|
| |
+ result: ignore
|
| |
+
|
| |
+ notes:
|
| |
+ The default value is: "result: respect".
|
| |
psss
commented 5 years ago
As discussed on the Best Practices meeting, this pull request is proposing a new attribute to handle different ways how to interpret test results. I am not completely sure about the attribute name
result. Does anyone have a suggestion for a better name?