#34 roles: More rigorous parsing of PASS/FAIL in beakerlib tests
Merged 6 years ago by merlinm. Opened 6 years ago by stefw.
stefw/standard-test-roles beakerlib-pass-fail-check  into  master

@@ -121,7 +121,11 @@ 

    - name: Make the master test summary log artifact

      shell: |

        logfile={{ remote_artifacts }}/test.$(echo {{ item }} | sed -e 's/\//-/g').log

-       if grep -q FAIL "$logfile"; then

+       if grep -q '\[ *FAIL *\]' "$logfile"; then

+         echo "FAIL {{ item }}" >> {{ remote_artifacts }}/test.log

+       elif grep -q '\[ *PASS *\]' "$logfile"; then

+         echo "PASS {{ item }}" >> {{ remote_artifacts }}/test.log

+       elif grep -q FAIL "$logfile"; then

          echo "FAIL {{ item }}" >> {{ remote_artifacts }}/test.log

        elif grep -q PASS "$logfile"; then

          echo "PASS {{ item }}" >> {{ remote_artifacts }}/test.log

The parsing of PASS or FAIL status from a beakerlib test
output needs to be more rigorous, or tests with lines like
FAILED in their output will be triggered.

As noted at https://pagure.io/standard-test-roles/issue/32#comment-450692, false UNKNOWNs are now reported.

Perhaps it would help to prepend the more granular pass/fail logic to what was already there, maybe something line:

      if grep -q '\[ *FAIL *\]' "$logfile"; then
        echo "FAIL {{ item }}" >> {{ remote_artifacts }}/test.log
      elif grep -q '\[ *PASS *\]' "$logfile"; then
        echo "PASS {{ item }}" >> {{ remote_artifacts }}/test.log
      elif grep -q FAIL "$logfile"; then
        echo "FAIL {{ item }}" >> {{ remote_artifacts }}/test.log
      elif grep -q PASS "$logfile"; then
        echo "PASS {{ item }}" >> {{ remote_artifacts }}/test.log
      else
        echo "UNKNOWN {{ item }}" >> {{ remote_artifacts }}/test.log
      fi

rebased

6 years ago

Tested, and that works much better now. Thanks.

Pull-Request has been merged by merlinm

6 years ago
Metadata